Mounting LVM Volumes inside a VMWare Disk Image

So, the situation is this. I’m sitting remotely at a client site, and suddendly, due to some sleep-deprived slip-up, ended up erasing part of the configuration for their local server.

Being the kind of guy to plan things through before taking action (usually), I had previously made a test set-up with all the configs in vmware on my local workstation at home, before heading over and installing them. So I pop open my laptop, fire up the Cisco VPN Client, connect over there, leapfrog some routers in between and land on my workstation. Turns out I hadn’t left the vm running, so I can’t access it from the shell by SSH’ing in.

“Hey, no problem!” I thought. “I’ll just mount the disk images with vmware-mount.pl, fetch the configuration files, send them over with a convulted mess of ssh-within-ssh-within-ssh and unix pipes to my laptop! Piece of cake!”

But then something hits me. The Virtual Disk Image (vmdk) file had LVM Volumes as partitions. Which I can’t directly mount from my Ubuntu workstation. Usually, you can get away with issuing the following:

$ sudo /opt/vmware/bin/vmware-mount.pl /path/to/disk-image.vmdk patition_number -t ext3 /mnt/mountpoint

But to much of my dismay, partition 2 on this particular disk image is an LVM Volume, so it can’t be directly mounted. I has to be mapped and a bunch things has to be done before I can get to the data. I’m not even sure I have LVM support on the Ubuntu machine at the moment.

Curses. Unless I fuck around with it to make it work. Which I did. Read on for the details.

So, I light up a cigarette and get to work. I first issue the command described above, regardless of the outcome. vmware-mount.pl works by firing up part of vmware, accessing the disk, and then feeds the data through a Network Block Device (/dev/nb0), which, in turn, is then mounted wherever you specify.

The output is optimistic.

--------------------------------------------
VMware for Linux - Virtual Hard Disk Mounter
Version: 1.0 build-45731
Copyright 1998 VMware, Inc.  All rights reserved. -- VMware Confidential
--------------------------------------------

It has been reported that this program does not work correctly with 2.4+ Linux 
kernels in some cases, and you are currently running such a kernel. Do you 
really want to continue? [N] y

No Network Block Device detected.

There is no Network Block Device defined on this machine. This script is about 
to create the /dev/nb0 Network Block Device. Continue? [Y] 

Creating the /dev/nb0 Network Block Device

No Network Block Device driver detected.

Trying to load the Network Block Device driver kernel module... Success.

Client: The partition is now mapped on the /dev/nb0 Network Block Device.
mount: unknown filesystem type 'LVM2_member'

If you know the filesystem of the partition you want to mount, you can provide 
it using the -t command line option. Since you haven't done so, this script is 
going to try to determine the filesystem of the partition based on the partition
type and id.

Unable to retrieve the filesystem of the partition (the partition type is BIOS 
and the partition Id is 8E). Please file an incident with VMware at 
http://www.vmware.com/forms/Incident_Login.cfm by copying this error message.

So here I think, “I’m pretty fucked. I’ll have to hack vmware-mount.pl, and god knows vmware’s Perl scripts are complex.”

So, out of the blue, I try again. But then I notice the vmware disk image is locked. And /dev/nb0 is still present. So my vmware Disk Image is still attached to the network block device! Great!

So, I ignore the error messages, and install LVM support on my ubuntu box.

$ sudo apt-get install lvm2

Note: On Ubuntu Feisty, according to bug #96802, the lvm package has some packaging errors. You'll end up getting the error message "no program found for your current version of LVM" if you try to issue any of the LVM management commands. Long story short, issue the following to fix it:

$ sudo ln -s /lib/lvm-200 /lib/lvm-0

Next step, detect and mount the LVM volume you just added

Step 1: Load LVM kernel modules:

$ sudo modprobe dm-mod

Step 2: Detect new LVM Physical Volumes

$ sudo vgscan

It should appear as /dev/nb0. You may safely ignore errors about other devices. The last line should be: Found volume group "VolGroup00" using metadata type lvm2, or something similar.

Step 3: Activate the newly found vgroup

$ sudo vgchange -ay VolGroup00

(Substitute "VolGroup00" appropriately if you named your Volume Group differently)

Step 4: Display available volumes and mount them as necessary

$ sudo lvdisplay

Step 5: Mount logical Volume needed:

$ sudo mount -t ext3 /dev/VolGroup00/LogVol00 /mnt/lvm1 -o ro

Step 6: SUCCESS! (Profit)

I can now access my RHEL5 Disk from my Ubuntu machine. Now, time to go back to the convulted mess of ssh-within-ssh-within-ssh-and-unix-pipes, but that's a story for another day.

Also, an important note, I have no idea how to unmount that volume. Well, I know how to unmount the LVM volume, but I have no idea how to get rid of /dev/nb0 without a reboot. Also, do note that this may make your machine majorly unstable. I had to call a friend to walk over to my place and reboot my workstation because it hung while reading files. But in the end, it worked.

I hope this helps.

This entry was posted on Friday, September 7th, 2007 at 1:18 pm and is filed under Computers, English, Tutorials/How-To, Unix/Linux. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.