Rainbow Scenery
Valid HTML 4.01 Transitional

CompuLab (Fit-PC) Intense PC
Setting Up the Virtual Machine

Jim Carter, 2013-04-06

The virtual machine Baobei runs Microsoft Windows 7 and is used for Windows-only financial software, specifically TurboTax and Microsoft Access. These are mission-critical functions and getting the virtual machine working reliably is important.

Moving the Virtual Disc

On the old machine, Baobei is hosted on Sun's VirtualBox. This software has been discontinued, as has Sun Microsystems, and I'm going to host the VM on the KVM framework, which uses libvirt and qemu. VirtualBox has an idiosyncratic disc image format called VDI that only it can read. Thus it's going to be a struggle to copy the virtual disc onto the new machine.

You do not want a pending update, which will mess up the following procedure. On the old machine go to Windows Update and make sure that all updates are downloaded and installed and that all necessary reboots have been done. Reboot again for good measure; this is Windows, after all.

In one forum post by ajd4096 (2007-11-22) he says he finally booted an Ubuntu rescue CD inside VirtualBox. Being inside it had access to the raw devices, and he could cat the whole disc to a trans-net pipe ending on his new machine. Later in the thread, bis0n (2009-03-04) suggests:

VBoxManage internalcommands converttoraw file.vdi file.raw

Reference for this to wikibooks.org for this non-user-level tidbit. That is the method I used. I created a directory /scr/jimc on the new machine (temporarily called Orion). On the old machine I used these commands:

sshfs jimc@orion:/scr/jimc /mnt # Give jimc's password
VBoxManage internalcommands converttoraw /dev/sda2 /mnt/baobei.sda2.raw
fusermount -u /mnt

The data transfer speed was about 11.3Mb/sec, saturating the 100baseT Ethernet. It used Using 35% CPU on Diamond and 50% CPU on Orion (sum of 2 threads) (100% would refer to one CPU saturated). file baobei.sda2.raw reports these partitions (all sizes are in 512 byte sectors):

Start Size Type Content
2048 204800 7 A booter, no payload files
206848 102131712 7 The payload image

Jiggering the Partition Table

Initially I laid out the new machine's disc like this:

Nbr Start End Size Type Fsys Flags Mount Pt
1 1049kB 16.1GB 16.1GB primary ext4 boot, type=83 / (root)
2 16.1GB 24.7GB 8587MB primary swap type=82 Swap
3 24.7GB 46.2GB 21.5GB primary ext4 type=83 /home
4 46.2GB 500GB 454GB extended lba, type=0f
5 46.2GB 154GB 107GB logical type=07 Baobei
6 154GB 500GB 347GB logical ext4 type=83 /s1 (distro)

However, Baobei actually uses only 27Gb of its former 49.9Gb, and 107Gb is a total waste. I'm going to split the partition into two of approx 53.5Gb, let's say 54Gb for Baobei and the rest as a second area for special projects, as /dev/sda7. You aren't required to have the partition numbers in order by block number.

Some minor details cropped up:

I did these commands in parted:

rm 5
mkpart logical ntfs 46.2GB 98.2GB
mkpart logical ext4 98.2GB 154GB
Nbr Start End Size Type Fsys Flags Mount Pt
1 1049kB 16.1GB 16.1GB primary ext4 boot, type=83 / (root)
2 16.1GB 24.7GB 8587MB primary swap type=82 Swap
3 24.7GB 46.2GB 21.5GB primary ext4 type=83 /home
4 46.2GB 500GB 454GB extended lba, type=0f
5 154GB 500GB 347GB logical ext4 type=83 /s1 (distro)
6 46.2GB 98.2GB 52.0GB logical type=07 Baobei
7 98.2GB 154GB 55.3GB logical ext4 type=83 /s2 (extra)

Since /dev/sda6 was mounted at the time (I should have unmounted it), the kernel's copy of the partition table was not updated. /etc/fstab already mounts the discs by label, so I just rebooted. Then I put a filesystem on /dev/sda7:

mkfs -t ext4 -L "diamond-s2" /dev/sda7

And I added a line in /etc/fstab to mount it:
LABEL=diamond-s2 /s2 ext4 acl,user_xattr 1 4

Squeezing the Virtual Disc

Step by step, here's what I did. First I made a copy of the original image that I could trash.

dd if=/scr/jimc/baobei.sda2.raw of=/scr/jimc/baobei.sda2.copy bs=1M

Here is its partition table:

losetup -f # To pick a vacant loop device; says /dev/loop0
losetup /dev/loop0 /scr/jimc/baobei.sda2.copy
parted /dev/loop0
units B # Shows everything in bytes
Nbr Start End Size Type Fsys Flags Mount Pt
1 1048576B 105906175B 104857600B primary ntfs type=07, boot Booter
2 105906176B 52397342719B 52291436544B primary ntfs type=07 Windows

Now we'll check the filesystem using ntfsresize from the ntfsprogs package.

losetup -d /dev/loop0
losetup -o 103424K /dev/loop0 /scr/jimc/baobei.sda2.copy
ntfsresize -n --info /dev/loop0

It says: filesystem check OK; Used 23665 MB; device 52292485120 bytes. If the whole disc image has to shrink by 368050176 B (let's shrink 4096 B more for good measure) then the payload partition will end up at size 51923382272 B. These commands do the shrinkage (or expansion). It's strongly recommended to run it with -n first.

ntfsresize -n --size 51923382272 /dev/loop0
ntfsresize --size 51923382272 /dev/loop0 #Succeeded

Now I copy the initial segment onto /dev/sda6. Fortunately the destination size is an exact multiple of 1M (2^20 bytes).

losetup -d /dev/loop0
dd if=/scr/jimc/baobei.sda2.copy of=/dev/sda6 bs=1M count=49620

Check that the copy came out OK.

losetup -o 103424K /dev/loop0 /dev/sda6
ntfsresize -n -f --info /dev/loop0 # -f to ignore chkdsk flag
losetup -d /dev/loop0

If I had been expanding, I would have first copied the raw image onto /dev/sda6 and resized that copy, rather than making a file copy and shrinking it first. Otherwise the procedure is the same.

Defining the Virtual Machine

The key problem here is that the disc (HDA) has paravirtual drivers for VirtualBox, and it doesn't have paravirtual drivers for KVM. (Actually the bus is virtio, which is not exactly paravirtual but is also unknown to Windows.) The procedure for getting these drivers onto the virtual machine is given in this OpenSuSE application writeup. To summarize:

Here is the initial virtual machine definition that I used. Note which bus each of the two discs is on. It's also important that the CDROM is on the IDE bus, since Windows is going to be reading the virtio drivers off it.

<domain type='kvm'>
  <memory unit='GB'>4</memory>
  <currentMemory unit='MB'>1024</currentMemory>
    <type arch='i686' machine='pc-0.14'>hvm</type>
    <bootmenu enable='no'/>
  <clock offset='utc'>
    <timer name='rtc' tickpolicy='catchup'/>
    <timer name='pit' tickpolicy='delay'/>
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw'/>
      <source file='/dev/sda6'/>
      <target dev='hda' bus='ide'/>
      <boot order='1'/>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/usr/share/qemu-kvm/win-virtio-drivers.iso'/>
      <target dev='hdb' bus='ide'/>
      <boot order='2'/>
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw'/>
      <source file='/s1/kvm/baobei-disc2.raw'/>
      <target dev='hdc' bus='virtio'/>
    <controller type='ide' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
    <controller type='usb' index='0'/>
    <interface type='bridge'>
      <mac address='52:54:00:09:c8:c5'/>
      <source bridge='br0'/>
      <model type='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    <input type='mouse' bus='ps2'/>
    <graphics type='vnc' port='-1' autoport='yes'/>
      <model type='cirrus' vram='9216' heads='1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    <memballoon model='virtio'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>

Key features in this machine:

Name Baobei
UUID Must be unique per machine, create with uuidgen (no options needed)
Memory 4Gb
VCPU (cores) 2 cores
OS i686, no boot menu, with ACPI and PAE
Clock offset UTC
Power management Destroy on everything including reboot. There is something wacked and it cannot restart after an ACPI reboot.
Disk #1 /dev/sda6 on IDE bus initially
Disk #2 A temporary file (with a VFAT filesystem on it) on virtio bus
CDROM /usr/share/qemu-kvm/win-virtio-drivers.iso from the kvm package, on IDE bus
Interface (net) Bridge, 52:54:00:09:c8:c5 on br0

For the net interface's MAC address I have a local convention to use the required vendor prefix for the first 3 bytes followed by the ending 3 octets of the machine's fixed IPv4 address (converted to hex).

Commands to define the virtual machine:

virsh undefine baobei # if already defined
virsh define /home/baobei/baobei.xml
virt-viewer -w -r baobei &
virsh start baobei

Here's what happened then. This is a synthesis of several failed attempts, skipping over false trails and breaking the machine.

Windows Experience Index:

Category Index Comments
Processor 4.7 2 CPUs allowed to the VM
RAM 7.5 4Gb allowed to the VM
Graphics (Aero) 1.0 By VNC to the virtual framebuffer
Gaming Graphics(3D) 1.0 Pathetic
Disc 5.9 Using virtio

To start TurboTax took 20 seconds flat. On the old machine this would take as long as 5 minutes.

Remaining Issues

Rainbow Scenery