CompuLab (Fit-PC) Intense PC
Setting Up the Virtual Machine

The virtual machine Baobei runs Microsoft Windows 7 and is used for Windows-only financial software, specifically TurboTax and Microsoft Access. These are mission-critical functions and getting the virtual machine working reliably is important.

Moving the Virtual Disc

On the old machine, Baobei is hosted on Sun's VirtualBox. This software has been discontinued, as has Sun Microsystems, and I'm going to host the VM on the KVM framework, which uses libvirt and qemu. VirtualBox has an idiosyncratic disc image format called VDI that only it can read. Thus it's going to be a struggle to copy the virtual disc onto the new machine.

You do not want a pending update, which will mess up the following procedure. On the old machine go to Windows Update and make sure that all updates are downloaded and installed and that all necessary reboots have been done. Reboot again for good measure; this is Windows, after all.

In one forum post by ajd4096 (2007-11-22) he says he finally booted an Ubuntu rescue CD inside VirtualBox. Being inside it had access to the raw devices, and he could cat the whole disc to a trans-net pipe ending on his new machine. Later in the thread, bis0n (2009-03-04) suggests:

VBoxManage internalcommands converttoraw file.vdi file.raw

Reference for this to wikibooks.org for this non-user-level tidbit. That is the method I used. I created a directory /scr/jimc on the new machine (temporarily called Orion). On the old machine I used these commands:

sshfs jimc@orion:/scr/jimc /mnt # Give jimc's password
VBoxManage internalcommands converttoraw /dev/sda2 /mnt/baobei.sda2.raw
fusermount -u /mnt

The data transfer speed was about 11.3Mb/sec, saturating the 100baseT Ethernet. It used Using 35% CPU on Diamond and 50% CPU on Orion (sum of 2 threads) (100% would refer to one CPU saturated). file baobei.sda2.raw reports these partitions (all sizes are in 512 byte sectors):

Jiggering the Partition Table

Initially I laid out the new machine's disc like this:

However, Baobei actually uses only 27Gb of its former 49.9Gb, and 107Gb is a total waste. I'm going to split the partition into two of approx 53.5Gb, let's say 54Gb for Baobei and the rest as a second area for special projects, as /dev/sda7. You aren't required to have the partition numbers in order by block number.

Some minor details cropped up:

• Windows thinks its virtual disc is an actual physical disc with two partitions: a small one for the booter and a second one covering the rest of the disc for the payload. The recursive partition structure adds complication.

• I was planning to make the Windows partition a lot larger than it had on the old machine, and expand it. The plan is now changed to make it only slightly larger. However, some sizes are calculated in GB (2^30 bytes), while others are in GiB (10^9 bytes), differing by about 7%. I ended up making the partition about 0.36Gb too small. The procedure to shrink a partition is a bit more complicated.

• If you delete a partition and later re-create it, following partitions are renumbered to be contiguous, and these numbers are used as the minor device numbers. In this case, /dev/sda6 turns into /dev/sda5, and Baobei's smaller partition will be dev/sda6, with a new /dev/sda7 after it.

I did these commands in parted:

rm 5
mkpart logical ntfs 46.2GB 98.2GB
mkpart logical ext4 98.2GB 154GB
print

Since /dev/sda6 was mounted at the time (I should have unmounted it), the kernel's copy of the partition table was not updated. /etc/fstab already mounts the discs by label, so I just rebooted. Then I put a filesystem on /dev/sda7:

mkfs -t ext4 -L "diamond-s2" /dev/sda7

LABEL=diamond-s2 /s2 ext4 acl,user_xattr 1 4

Squeezing the Virtual Disc

Step by step, here's what I did. First I made a copy of the original image that I could trash.

dd if=/scr/jimc/baobei.sda2.raw of=/scr/jimc/baobei.sda2.copy bs=1M

Here is its partition table:

losetup -f # To pick a vacant loop device; says /dev/loop0
losetup /dev/loop0 /scr/jimc/baobei.sda2.copy
parted /dev/loop0

units B # Shows everything in bytes
print
quit

Now we'll check the filesystem using ntfsresize from the ntfsprogs package.

losetup -d /dev/loop0
losetup -o 103424K /dev/loop0 /scr/jimc/baobei.sda2.copy
ntfsresize -n --info /dev/loop0

It says: filesystem check OK; Used 23665 MB; device 52292485120 bytes. If the whole disc image has to shrink by 368050176 B (let's shrink 4096 B more for good measure) then the payload partition will end up at size 51923382272 B. These commands do the shrinkage (or expansion). It's strongly recommended to run it with -n first.

ntfsresize -n --size 51923382272 /dev/loop0
ntfsresize --size 51923382272 /dev/loop0 #Succeeded

Now I copy the initial segment onto /dev/sda6. Fortunately the destination size is an exact multiple of 1M (2^20 bytes).

losetup -d /dev/loop0
dd if=/scr/jimc/baobei.sda2.copy of=/dev/sda6 bs=1M count=49620

Check that the copy came out OK.

losetup -o 103424K /dev/loop0 /dev/sda6
ntfsresize -n -f --info /dev/loop0 # -f to ignore chkdsk flag
losetup -d /dev/loop0

If I had been expanding, I would have first copied the raw image onto /dev/sda6 and resized that copy, rather than making a file copy and shrinking it first. Otherwise the procedure is the same.

Defining the Virtual Machine

The key problem here is that the disc (HDA) has paravirtual drivers for VirtualBox, and it doesn't have paravirtual drivers for KVM. (Actually the bus is virtio, which is not exactly paravirtual but is also unknown to Windows.) The procedure for getting these drivers onto the virtual machine is given in this OpenSuSE application writeup. To summarize:

• Define the main disc initially to use an IDE bus, which Windows does have drivers for, plus a second disc with the virtio bus, for which drivers will then be wanted.

• Boot the machine and provide the virtio drivers. Shut down.

• Now put the main disc on the virtio bus, and get rid of the second disc.

• When Windows boots again it will automatically use the virtio drivers for the main disc.

Here is the initial virtual machine definition that I used. Note which bus each of the two discs is on. It's also important that the CDROM is on the IDE bus, since Windows is going to be reading the virtio drivers off it.

<domain type='kvm'>
  <name>baobei</name>
  <uuid>ef8a1ea9-4261-49dc-8083-b1f24e05c7a2</uuid>
  <memory unit='GB'>4</memory>
  <currentMemory unit='MB'>1024</currentMemory>
  <vcpu>2</vcpu>
  <os>
    <type arch='i686' machine='pc-0.14'>hvm</type>
    <bootmenu enable='no'/>
  </os>
  <features>
    <acpi/>
    <pae/>
  </features>
  <clock offset='utc'>
    <timer name='rtc' tickpolicy='catchup'/>
    <timer name='pit' tickpolicy='delay'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>destroy</on_reboot>
  <on_crash>destroy</on_crash>
  <devices>
    <emulator>/usr/bin/qemu-kvm</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw'/>
      <source file='/dev/sda6'/>
      <target dev='hda' bus='ide'/>
      <boot order='1'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/usr/share/qemu-kvm/win-virtio-drivers.iso'/>
      <target dev='hdb' bus='ide'/>
      <readonly/>
      <boot order='2'/>
    </disk> 
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw'/>
      <source file='/s1/kvm/baobei-disc2.raw'/>
      <target dev='hdc' bus='virtio'/>
    </disk>
    <controller type='ide' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
    </controller>
    <controller type='usb' index='0'/>
    <interface type='bridge'>
      <mac address='52:54:00:09:c8:c5'/>
      <source bridge='br0'/>
      <model type='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </interface>
    <input type='mouse' bus='ps2'/>
    <graphics type='vnc' port='-1' autoport='yes'/>
    <video>
      <model type='cirrus' vram='9216' heads='1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </video>
    <memballoon model='virtio'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </memballoon>
  </devices>
</domain>

Key features in this machine:

For the net interface's MAC address I have a local convention to use the required vendor prefix for the first 3 bytes followed by the ending 3 octets of the machine's fixed IPv4 address (converted to hex).

Commands to define the virtual machine:

virsh undefine baobei # if already defined
virsh define /home/baobei/baobei.xml
virt-viewer -w -r baobei &
virsh start baobei

Here's what happened then. This is a synthesis of several failed attempts, skipping over false trails and breaking the machine.

• It shows the Starting Windows splash screen.

• When the NTFS filesystem is resized it is marked to be checked on the next boot. It drops into Windows Error Recovery; pick Launch Startup Repair.

• It shows a pretty GUI with system recovery options. Pick your keyboard.

• Select the repair procedure (the first option).

• Do not ever revert to a restore point in this procedure. (Reversion is offered several places including the initial repair screen, the second option.) There is nothing wrong with Windows that the correct drivers won't cure.

• I think that if the main disc is on the IDE bus from the beginning, it will not need to load drivers, but if it does not show or try to search for the Windows instance, hit Install Drivers.

  • Open the file browser next to the line for the pathname of the driver .INF file.
  • Click on Computer, then double click CD Drive which you have provided from the ISO image in the KVM package that contains the drivers. Go through the containing directories: viostor (disc driver; you'll want the net driver later), Windows7 (or pick your correct version), x86 or X86_64 architecture, and double click on the viostor item described as setup information.
  • Confirm to install the 3 drivers.
  • It does so with no error messages.

• It now searches for windows installations, rather slowly. Eventually it finds one instance of Windows 7, already selected. Hit Next.

• It attempts repairs. This is not quick but not as awful as the warning message says. After 2-3 minutes it finishes. Under Details it lists a lot of things it checked with no error found. Finally it ran chkdsk, taking a lot of time but finding no errors.

• Hit Finish to reboot. Remember that rebooting doesn't work; the virtual machine will destroy itself. Restart the machine (virsh start baobei).

• It boots right up and gives the login screen. (No network yet.)

• Follow the instructions in the reference given previously, to supply the viostor and network drivers to the devices lacking them. To summarize:

  • Start Device Manager. In the Win7 Start menu use the search feature to find it.
  • Right click on one of the uncategorized Novell devices (SCSI or network). Pick Update Drivers.
  • Find the driver's .INF file in the same way you did in the repair console.
  • Repeat for the other device.
  • You're done. Reboot.

• Shut down the virtual machine. Edit the XML definition to change the main disc to the virtio bus, and delete the second disc. Undefine and redefine the virtual machine. Start it up again.

• I'm not sure if it always wants to do system repair after a change to the XML file, but it often does. At this point you should be able to Start Windows Normally and it will work.

• The disc works, the network works, including IPv6 with a random address, but it also responds to incoming packets sent to its RFC 2462 address.

Windows Experience Index:

To start TurboTax took 20 seconds flat. On the old machine this would take as long as 5 minutes.

Remaining Issues

• Printing: The URL http://diamond.cft.ca.us:631/printers/lp2 can be used for the printer, but the server responds: windows-ext client-error-bad-request; Print-Job client-error-not-authorized.

  Most likely this is a configuration error on the server, which is new.

• When you tell Windows to shut down, it reboots instead. With shutdown or intentional reboot, it pops a screen saying Windows has been shut down to prevent damage to the filesystem, and it then reboots and attempts system repair. To avoid all these issues, I changed the VM's power management action upon reboot to destroy. If you really want to reboot, you need to start the VM by hand.

• Where are we going to put the financial records, and how do we back them up?

  • Originally we had Windows running on a physical machine. The records were hosted on that machine and were exported by CIFS and mounted on the Linux backup server, which copied them to the backup media.

  • With VirtualBox, it turned out to be impossible to reach into the virtual disc to back up the records. We put them in a directory on the host (from which they were easy to back up) and exported this directory to the Windows machine using Samba (CIFS). One objection to this method was that Windows frequently complained that trans-net content was likely to be a security hazard, and we had to blow off this warning every time we recorded a transaction.

  • With KVM it is possible to define a disk of type dir representing a directory on the host, but it appears to Windows as a readonly VFAT partition, which is not useful.

  • KVM also has a filesystem object. I experimented with a Linux VM and was able to mount it, but I was not able to set up the permissions and ownership so the guest could write on it. I never did try it on Windows.

  • I reverted to the original scheme: hosting the financial records on the Windows machine. For backup I mount the virtual disc's partition (readonly); see below for the systemd unit(s) to do this.

• If the host has the disc mounted and the virtual machine changes anything, like creating, renaming or deleting a file, and the host has cached the affected content, it will not be aware of the change, which could have a baleful effect on the host. It is best to do backups when the guest is entirely shut down, and to unmount the virtual disc right away after backups finish, i.e. before starting the guest again.

  To make the unmounting happen, I set up three systemd units. These files go in /etc/systemd/system . This is over-complicated and kludgey.

  • baobei.automount (needs to be enabled). While /usr/sbin/automount automatically unmounts unused objects after a timeout, systemd's automounter doesn't, adding a lot of complication.

    [Unit]
    Description=Automount Baobei's virtual disc
    
    [Automount]
    Where=/baobei
    
    [Install]
    WantedBy=local-fs.target
    

  • baobei.mount . The offset is found by doing file -s /dev/sda6, finding where the wanted partition starts, and converting 512 byte sectors to 1024 byte K's (or use bytes).

    [Unit]
    Description=Mount Baobei's virtual disc
    
    [Mount]
    What=/dev/sda6
    Where=/baobei
    Type=ntfs
    Options=ro,loop,offset=103424K,umask=222,uid=alice
    

  • baobei-unmount.service (must be enabled).

    [Unit]
    Description=Unmounts Baobei's virtual disc
    
    # When baobei.mount is started or stopped, this unit is started/stopped too.  
    PartOf=baobei.mount
    
    # If baobei.mounts stops for any reason this one also stops.
    BindsTo=baobei.mount
    
    After=baobei.mount
    
    [Service]
    Type=oneshot
    
    ExecStartPre=/usr/bin/sleep 300
    ExecStartPre=/bin/echo Unmounting /baobei
    ExecStart=/usr/bin/umount /baobei
    
    # When this process unmounts /baobei, BindsTo applies, and this unit is
    # stopped, and the still-running umount is killed, by default with SIGTERM.
    # Let's use a signal that's ignored by default.
    KillSignal=SIGWINCH
    
    Restart=always
    RestartSec=300
    # This is the return code if the mtpt is not mounted or is busy.
    SuccessExitStatus=32
    
    [Install]
    WantedBy=baobei.mount