November 2011

HPC Data Center Relocations

Top 10 best practices for HPC Data Center Relocations

By Todd Bone, XS International

There are many reasons that a company or institution might need to move a large computer system. New equipment might necessitate the currently installed systems being relocated within the same data center to provide space. Leases expire, facilities close, corporate headquarters move, grants end, and new data centers are built making it either necessary or desirable to move a system to a new location. Whatever the reason, moving a large HPC installation is a massive undertaking that can fall anywhere between a nightmarish disaster to a relatively painless process.

Following a few simple best practices can eliminate many of the traps often faced by teams who take on this task.

1. During requirements capture, interview the end user and all key stakeholders to finalize responsibilities under contract. Change orders can cost you more time than it would take if the proper plan was in place to begin with.

2. Construct a schedule with identifiable crash points. Be realistic in your scheduling. When the administrator of the system requires that the system has an inflexible maximum downtime don’t forget to compensate for the extra time it takes to reconfigure the system each time a new phase begins or ends. Allow for shipping or order fulfillment delays and for unavoidable delays such as weather or traffic.

3. Produce redundant comprehensive backups. The Administrator should handle this, but her responsibility for the data sets and the integrity of all data structures should be clearly laid out. This is especially important when an HPC storage device is being physically moved, in the age of high-speed high-capacity data appliances performance often comes at the expense of physical sturdiness.

4. Identify each and every piece of hardware in the existing configuration and specify a successful end state for each part, whether it is a server, storage array or a power cable.

5. Get the right tools for the job. If there is no loading dock at the Data Center, make sure to bring a truck with a lift-gate. How high are the doorways? Is the wiring under floor, in the ceiling, in trays above the systems? What type screws are being used in the racks? Make allowances for especially large or fragile items. Some switches being used in HPC today are mammoth contraptions that weigh a lot and which have hundreds of tiny connections that must be handled carefully.

6. Contingency planning – identify as many potential pitfalls as possible and plan a comprehensive solution to assure success. Have backup labor standing by. Carry insurance adequate to cover a complete equipment loss and any lost profit incurred because of unexpected system downtime.

7. Distribute the cell numbers and email addresses of all key stakeholders. Open and consistent communication between everyone with a stake in the outcome of a data center move is absolutely essential. For the technicians performing the move, a lack of communication can kill productivity. Techs who are stuck waiting for someone to unlock a door or turn on a breaker or who can’t make another move until an unresolved administrative disagreement is resolved are a massive waste of resources. Communication is also essential to synchronize system downtime and assure all proper backups and security precautions have been made.

8. Check the infrastructure requirements with a site visit prior to the start of work. Nothing slows a Data Center move like waiting for new power to be installed or repairing a malfunctioning HVAC system. Also verify all connection types and assure there is physically enough space to house the incoming system.

9. Fastidiously label every cable on both ends and put a label on every server, storage array and switch in the front and in the back of the rack. Create a set of documents to identify and locate every component of the system in its new data center.

10. The best chance for successfully implementing your plan is to utilize professional technicians whose day-to-day responsibilities include moving static-sensitive equipment and keeping massive groups of cables functionally organized and aesthetically pleasing. Inexperienced attempts at data center moves usually end up costing more when professionals have to be called in to clean up an amateur’s mess.

Make sure all of the key players in your move are included in the plan constructed following these suggestions. Whether you’re moving a small rack with a few servers across the room or your team is tasked with moving a 300 node cluster across the country, these best practices can save you many headaches.

XSi specializes in Data Center Relocation, Data Center Moves, Installation and Deinstallation of HPC systems including SGi, CRAY, IBM, HP, Dell, Super Micro and other HPC systems, storage and networking equipment.

For more information about XS International, Inc. Data Center Relocation and Data Center Move services, contacting xs@xsnet.com or call 1.800.256.6133 x 106 today!

Share