Posted on | August 30, 2008 | No Comments
A server with a 1.2 TB SATA RAID-5 is housing 15 paravirtualized Xen guests via local storage. Each one has at least one LV of 100 – 400 GB. Move them to an identical server with a more sensibly configured RAID without pissing off each user for more than 30 – 40 minutes. All servers have use of a private gigabit link that can be used for the copy.
The Senior Integrator:
We’ll use iSCSI or AoE (depending on the size of dom-0). We’ll stop one guest at a time, export then create their LV’s on the new box and use ‘dd’ to move the data. After that we’ll update the guest’s kernel modules as needed. If the secondary switch says Linksys, its not my fault that people get upset. BTW, why the ^$(#*& aren’t we using a NAS or SAN, anyway?
The Junior System Administrator:
Can’t we just use rsync, somehow? How come we’re not using ZFS? Wait, is this Linux?
Easy, we just hack ext3 so that every process has its own unique and disposable view of a file system scoped via temporal inodes in vfs. This lets the guests keep running while we copy without fear of copying 10 revisions of the same database into one file, since we can just mount them from dom-0 and use NFS, scoping each FS just as it was when we mounted it. By the way, I could write a tool to do that as an emacs plug-in if the boss agrees to let me AGPL3 it. Some bastard might make emacs as a service, you never know!
When you get really, really bored watching dd run, sometimes all you can do is blog.