Lab Management: In-House or Cloud?
October 16th, 2009 Posted in Virtual Lab Automation and ManagementOver the last couple of years, we’ve spent lots of time working with prospects as they make the decision whether to build their own lab infrastructure (using vendors such as VMware lab manager, VMlogix, Surgient or the upcoming Microsoft Test and Lab Manager) or choose Skytap’s cloud-based lab. There are situations where either approach can be the right one: in some cases it makes sense to utilize an external cloud, and in other cases to build internally. Here are some of the top take-aways from those conversations:
Agility
Any lab management solution will enable an application development and test team to be far more productive. Bugs can be reproduced much faster with rich contextual information, new test environments set-up and torn-down in seconds and a new level of automation can be achieved in the build and deployment process. Furthermore, add tool integration (e.g. Visual Studio Team System or HP Software), and team members can achieve development and testing tasks far more quickly than before. A cloud-based lab can further increase agility because it can be scaled on demand (vs. an in-house lab that has a fixed capacity), and gives team the illusion of infinite capacity should it be required for a busy test period. This consideration becomes important for dev/test teams that experience variability in the demand for infrastructure.
Security
Some organizations, such as defence/military vendors, have policies which prevent any information being stored outside their firewall. In this case, an in-house lab management solution is the only option. However, we’ve also seen some of these organizations choose Skytap for the very reason it is outside the firewall. It enables them to create isolated sandboxes that can be shared with external vendors, such as distributed testing organizations. However, for the vast majority of organizations, security is not a major issue with external clouds, especially now secure IPsec VPN capabilities are available. Because test data should not include any production data, this also overcomes most regulatory concerns (HIPPA, PCI etc.).
Cost
We’ve been through lots of TCO analysis exercises with customers and typically we see a 40-70% lower cost with Skytap than an in-house lab. Gartner gave us a cool vendor of the year award for 2009 and stated Skytap has the ‘‘potential to dramatically reduce total cost of ownership for test labs by 25% to 50% annually’. The reason for such dramatic numbers is that an external cloud is multi-tenant, and when virtual infrastructure isn’t deployed, a customer doesn’t incur cost. Most test environments and build servers only run for a few hours a day, so by implementing an in-house lab, you are essentially paying a premium for the time that lab infrastructure lies idle. The only case where this isn’t true is where test configurations are required to run 24×7.
Given the proven cost savings, we have only seen decisions go the other way for one of three reasons (and this is rare!): (1) Organizations have a lot of money (e.g. investment banks) and want to ‘build and own’ their lab internally and can afford this luxury, (2) there are political dynamics where an IT operations team doesn’t want to use an external cloud and builds a case against it and (3) an organization falls for the ’sunk cost fallacy’ where they have spare older hardware and think it’s cheaper to go down the in-house route (ignoring the fact in a couple of years they will be hit with a big hardware refresh cycle).
Administration
For the vast majority of organizations, IT operations staff are overstretched and focused on production uptime. The idea of using a cloud-based service for the lab environments is very compelling (we hear again and again ‘lab environments are 20% of the server footprint and 80% of the work’). However, a minority of organizations have spare IT ops capacity and prefer to build out a lab internally. A typical implementation can take 6 months to a year and once a solution is in place, and there are often performance issues (here’s a good example). It’s very much similar to the ’salesforce.com vs. Siebel’ decision – some organizations prefer to move to a cloud vendor with a core competency delivering a service and others choose to build in-house.
Team Collaboration
Finally, team collaboration is often an important consideration. In some cases, we’ve seen Internet connectivity in very remote parts of the world be an issue. This not only prevents the use of a cloud-based lab, it also prevents a remote teams accessing an in-house lab in another part of the world. Fortunately, countries like India and China, where a lot of offshore testing is done, now have excellent connectivity and this is rarely an issue. The other consideration is latency between a Skytap data center and distributed locations and the same test for an in-house lab location. At Skytap we’re directly connected to 15 major ISPs that route the majority of the Internet’s traffic, so our connectivity speeds usually second to none. We also find that using a single cloud platform for sharing virtual machines and builds can greatly improve team productivity vs. utilized transfer on an internal network.
We’re obviously big advocates of the external cloud model and we are seeing great traction and customer adoption (100+ so far). But there are some reasons to build an in-house lab, including security (for some specific organizations) and cultural reasons. We use the ’salesforce.com vs. Siebel analogy’ frequently, because it often comes down to the same types of considerations.
Ultimately, it’s up to every organization to decide their approach. If you’re evaluating a cloud-based vs. internal implementation, please drop us a line – one of our virtual lab experts is always available to discuss your options!
Ian
