It is advised to have at least one processing core or hyper-thread for each virtual machine.

You system will also require hardware virtualization extensions to use fully virtualized guest operating systems. The steps to identify whether your system has virtualization extensions can be found at Hardware virtualization extensions.

Hardware virtualization extensions

Full virtualization requires CPUs with hardware virtualization extensions. This section describes how to identify hardware virtualization extensions and enable them in your BIOS if they are disabled. If hardware virtualization extensions are not present you can only use para-virtualization with Red Hat Virtualization.

The virtualization extensions can not be disabled in the BIOS for AMD-V capable processors installed in a Rev 2 socket. The Intel® VT extensions can be disabled in the BIOS. Certain laptop vendors have disabled the Intel® VT extensions by default in their CPUs.

These instructions enable Intel® VT virtualization extensions If they are disabled in BIOS:

Do not proceed if the output of xm dmesg | grep VMX is not VMXON is done for each CPU. Please visit the BIOS if other messages are reported.

The following commands verify that the virtualization extensions are enabled on AMD-V architectures:

Enabling Intel® VT and AMD-V virtualization hardware extensions

To run unmodified guest operating systems (also known as fully virtualized or HVM guests) you require a Intel® VT or AMD-V capable system. The steps below will provide an overview about how you can verify whether the virtualization extensions have been enabled, and that Red Hat Enterprise Linux 5 can use them:

First, verify the Intel® VT or AMD-V capabilities are enabled via the BIOS. The BIOS settings for Intel® VT or AMD-V are usually in the Chipset or Processor menus. However, they can sometimes be hidden under obscure menus, such as Security Settings or other non standard menus.

For Intel® VT architectures perform these steps to make sure Intel Virtualization Technology is enabled. Some menu items may have slightly different names:

Adding Red Hat Virtualization packages to a Kickstart file

This section describes how to use kickstart files to install Red Hat Enterprise Linux. Kickstart files allow for large, automated installations without a user manually installing each individual system. The steps in this section will assist you in creating and using a Kickstart file to install Red Hat Enterprise Linux with the Red Hat Virtualization packages.

More information on kickstart files can be found on Red Hat's website, redhat.com, in the Installation Guide for your Red Hat Enterprise Linux version.

The section describes the steps necessary to install Red Hat Virtualization on a working copy of Red Hat Enterprise Linux.

Adding packages to your list of Red Hat Network entitlements

This section will describe the procedure for enabling Red Hat Network (RHN) entitlements to install the Red Hat Virtualization packages on your systems. You need to have these entitlements enabled and the packages installed to set up host systems.

Your system is now entitled to receive the Red Hat Virtualization packages. The next section covers installing these packages.

Installing Red Hat Virtualization with yum

To commence using virtualization on Red Hat Enterprise Linux you will need the xen and kernel-xen packages. The xen package contains the Xen hypervisor and Xen tools. The kernel-xen package contains a modified linux kernel which runs as a virtual machine guest on the Xen hypervisor.

To install the xen and kernel-xen packages, run:

# yum install xen kernel-xen

Fully virtualized guests on the Itanium® architecture require the guest firmware image package(xen-ia64-guest-firmware) from the supplementary installation DVD. This package can also be can be installed from RHN with the yum command:

# yum install xen-ia64-guest-firmware

To install the other recommended virtualization packages, use the command below:

# yum install virt-manager libvirt libvirt-python libvirt-python python-virtinst

Instead of virt-manager you can use the virt-install command. virt-install can either be used interactively or in a script to automate the creation of virtual machines. Using virt-install with Kickstart files an unattended installation of virtual machines can be achieved.

If you are using the virt-install CLI command and you select the --vnc option for a graphical installation you will also see the graphical installation screen as shown below.

The virt-install script provides a number of options one can pass on the command line. Below is the output from virt-install -help:

 virt-install -help
usage: virt-install [options]
options:
   -h, --help            show this help message and exit
   -n NAME, --name=NAME  Name of the guest instance
   -r MEMORY, --ram=MEMORY
                         Memory to allocate for guest instance in megabytes
   -u UUID, --uuid=UUID  UUID for the guest; if none is given a random UUID
                         will be generated. If you specify UUID, you should use
                         a 32-digit hexadecimal number.
   --vcpus=VCPUS         Number of vcpus to configure for your guest
   --check-cpu           Check that vcpus do not exceed physical CPUs and warn
                         if they do.
   --cpuset=CPUSET       Set which physical CPUs Domain can use.
   -f DISKFILE, --file=DISKFILE
                         File to use as the disk image
   -s DISKSIZE, --file-size=DISKSIZE
                         Size of the disk image (if it doesn't exist) in
                         gigabytes
   --nonsparse           Don't use sparse files for disks.  Note that this will
                         be significantly slower for guest creation
   --nodisks             Don't set up any disks for the guest.
   -m MAC, --mac=MAC     Fixed MAC address for the guest; if none or RANDOM is
                         given a random address will be used
   -b BRIDGE, --bridge=BRIDGE
                         Bridge to connect guest NIC to; if none given, will
                         try to determine the default
   -w NETWORK, --network=NETWORK
                         Connect the guest to a virtual network, forwarding to
                         the physical network with NAT
   --vnc                 Use VNC for graphics support
   --vncport=VNCPORT     Port to use for VNC
   --sdl                 Use SDL for graphics support
   --nographics          Don't set up a graphical console for the guest.
   --noautoconsole       Don't automatically try to connect to the guest
                         console
   -k KEYMAP, --keymap=KEYMAP
                         set up keymap for a graphical console
   --accelerat
   --accelerate          Use kernel acceleration capabilities
   --connect=CONNECT     Connect to hypervisor with URI
   --livecd              Specify the CDROM media is a LiveCD
   -v, --hvm             This guest should be a fully virtualized guest
   -c CDROM, --cdrom=CDROM
                         File to use a virtual CD-ROM device for fully
                         virtualized guests
   --pxe                 Boot an installer from the network using the PXE boot
                         protocol
   --os-type=OS_TYPE     The OS type for fully virtualized guests, e.g.
                         'linux', 'unix', 'windows'
   --os-variant=OS_VARIANT
                         The OS variant for fully virtualized guests, e.g.
                         'fedora6', 'rhel5', 'solaris10', 'win2k', 'vista'
   --noapic              Disables APIC for fully virtualized guest (overrides
                         value in os-type/os-variant db)
   --noacpi              Disables ACPI for fully virtualized guest (overrides
                         value in os-type/os-variant db)
   --arch=ARCH           The CPU architecture to simulate
   -p, --paravirt        This guest should be a paravirtualized guest
   -l LOCATION, --location=LOCATION
                         Installation source for paravirtualized guest (eg,
                         nfs:host:/path, http://host/path, ftp://host/path)
   -x EXTRA, --extra-args=EXTRA
                         Additional arguments to pass to the installer with
                         paravirt guests
   -d, --debug           Print debugging information
   --noreboot            Disables the automatic rebooting when the installation
                         is complete.
   --force               Do not prompt for input. Answers yes where applicable,

If you are using the virt-install command line interface tool and you select the --vnc option for a graphical installation you will also see the graphical installation screen as shown below.

If you used virt-install or virt-manager the following command line would start a guest installation with the same parameters as selected above via the GUI:

virt-install -n rhel5PV -r 500 -f /var/lib/xen/images/rhel5PV.dsk -s 3 --vnc -p -l\
ftp://10.1.1.1/trees/RHEL5-B2-Server-i386/

The window will show the initial boot of your guest:

After your guest has completed its initial bootstrap you will be dropped into the standard installation process for Red Hat Enterprise Linux or the operating system specific installer you select to install. For most installations the default answers will be acceptable

6.1.1. Graphical Red Hat Enterprise Linux 5 installation

After you have answered the basic questions to get the installation process started and all of the required files have been retrieved you will be greeted by the graphical installation screen of Red Hat Enterprise Linux 5 (or the installation screen of the operating system you decided to install):

1. If you are installing a Beta or early release distribution you need to confirm that you really want to install the operating system:

   

2. The next step is to enter a valid registration code. If you have a valid RHN subscription key please enter in the Installation Number field:

   

3. The release notes for Red Hat Enterprise Linux 5 have temporary subscription keys for Server and Client installations. If you are installing a local environment with no need to access RHN enter V for virtualization, S for clustered storage (CLVM or GFS) and C for clustering (Red Hat Cluster Suite) if required. In most cases entering V is sufficient:

   

   Note

   Depending on your version of Red Hat Enterprise Linux the setup numbers above may not work. In that case you can skip this step and confirm your Red Hat Network account details after the installation using the rhn_register command.

4. The installation will now confirm you want to erase the data on the storage you selected for the installation:

   

5. This screen will allow you to review the storage configuration and partition layout. You can also chose to select the advanced storage configuration if you want to use iSCSI as guest storage:

   

6. After you have reviewed the storage choice just confirm that you indeed want to use the drive for the installation:

   

7. Next is the guest networking configuration and hostname settings. The information will be populated with the data you entered earlier in the installation process:

   

8. Select the appropriate time zone for your environment:

   

9. Select a root password for your guest:

   

10. Now you get to select the software packages you want to install. You can also chose to customize the software selection by selecting the Customize Now button:

    

11. For our installation we chose the office packages and web server packages:

    

12. After you have selected the packages to install, dependencies and space requirements will be verified:

    

13. After all of the installation dependencies and space requirements have been verified you need to press the Next button to start the actual installation:

    

14. Now the installation will automatically install all of the selected software packages:

    

15. After the installation has finished you need to reboot your guest:

    

16. Your newly installed guest will not reboot, instead it will shutdown:

    

6.1.2. The first boot after the guest installation of Red Hat Enterprise Linux 5

To reboot your new guest use the command xm create GuestName where GuestName is the name you entered during the initial installation. Guest configuration files are located in /etc/xen/.

Once you have started your guest using the command above you can use virt-manager to open a graphical console window for your guest. Start virt-manager, select your guest from the list and click the Open tab, you will see a window similar to the one below:

6.1.3. First boot configuration

During the first boot after the installation has finished you will see the First Boot configuration screen. This screen will walk you through some basic configuration choices for your guest:

1. First you need to agree to the license agreement:

   

2. Next is the Firewall configuration:

   

3. If you select to disable the Firewall configuration you need to confirm your choice one more time:

   

4. Next is SELinux. It is strongly recommended to run SELinux in enforcing mode but you can chose to either run SELinux in permissive mode or completely disable it:

   

5. Choosing to disable SELinux causes the following warning to appear:

   

6. You can enable kdump:

   

7. Confirm time and date are set correctly for your guest. If you install a para-virtualized guest time and date should be in sync with dom0/Hypervisor as the guest will get its time from dom0. A fully-virtualized guest may need additional configuration such as NTP to avoid skew in time from dom0:

   

8. If you have a Red Hat Network subscription or would like to try one you can use the screen below to register your newly installed guest in RHN:

   

9. Confirm your choices for RHN:

   

10. Once setup has finished you may see one more screen if you opted out of RHN at this time:

    

11. This screen will allow you to create a user besides the standard root account:

    

12. The following warning appears if you do not choose to create a personal account:

    

13. If the install detects a sound device you would be asked to calibrate it on this screen:

    

14. If you want to install any additional software packages from CD you could do so on this screen. It it often more efficient to not install any additional software at this point but add it later using yum:

    

15. After you press the Finish button your guest will reconfigure any settings you may have changed and continue with the boot process:

    

16. And finally you will be greeted with the Red Hat Enterprise Linux 5 login screen:

    

17. After you have logged in you will see the standard Red Hat Enterprise Linux 5 desktop environment:

    

virt-manager can install Windows XP as a fully-virtualized guest. However there are some extra steps which must be followed in order to complete the installation successfully. This section explains the installation process and extra steps needed to get Windows XP fully virtualized guests installed.

virt-manager can install Windows XP as a fully-virtualized guest. However there are some extra steps which must be followed in order to complete the installation successfully. This section explains the installation process and extra steps needed to get Windows XP fully virtualized guests installed.

It may be easier to use virt-install for installing Windows Server 2003 as the console for the Windows guest will open quicker and allow for F5 to be pressed which is required to select a new HAL.

An example of using the virt-install for installing a Windows Server 2003 guest:

virt-install -hvm -s 5 -f /var/lib/xen/images/windows2003spi1.dsk -n windows2003sp1\
-cdrom=/xen/trees/ISO/WIN/en_windows_server_2003_sp1.iso -vnc -r 1024

Presently, physical floppy disks can not be mapped to virtualized guests, however, you can access a virtualized floppy drive created using an image file of a floppy disk and a modified configuration file. The section will guide you through the process.

This section uses a guest system created with virt-manager running a fully virtualized Red Hat Enterprise Linux installation with an image located in /var/lib/xen/images/rhel5FV-1.img.

Create the XML configuration file for your guest image using the following command on a running guest. This will save the configuration settings as an XML file which can be edited to customize the operations the guest performs when the guest is started. For another example of editing the virsh XML files, read Chapter 27, Creating custom Red Hat Virtualization scripts.

# virsh dumpxml rhel5FV > rhel5FV.xml

Execute this instruction to create a floppy image to use on the guest.

dd if=/dev/zero of=/var/lib/xen/images/rhel5FV-1flop.img bs=512 count=2880

Add the content below, changing where appropriate, to your guest's configuration XML file. This example creates a guest with a floppy device as a file based virtual device.

<disk type='file' device='floppy'>
	<source file='/var/lib/xen/images/rhel5FV-1flop.img'/>
	<target dev='fda'/>
</disk>

Stop the guest system, and restart the guest using the XML configuration file.

# virsh create rhel5FV.xml

The floppy device is now available in the guest and stored as an image file on the host.

After your guests are installed and configured you can add additional storage to your virtualization machines. You can use multiple different types of storage in your guests, these include:

Adding a file based container as additional storage to a guest

To add a file based container to a guest you must perform the following steps:

Adding a block device as additional storage to a guest

To present a block device from your host to a guest you must perform the following steps:

The same procedure can be used to allow a guest machine access to other physical block devices, for example a CD-ROM or DVD drive.

Dynamically adding storage to a virtual machine

In some cases one may want to add additional storage to a virtual machine without the need to reboot it. Red Hat Enterprise Linux 5/Virt provides this capability using the command line toolset xm. In order to dynamically add storage to a virtual machine/domain you need to perform the following steps:

In an environment where external storage (for example, Fibre Channel or iSCSI) is used it is advised to configure persistent device names on your hosts. This will also aid in using Red Hat Virtualization's (live) migration feature to implement consistent device names across multiple systems.

Single Path Configuration

On systems not using multipath, udev is a good way to implement LUN persistence. For more information on scsi_id and its various options please consult the man page.

Multi-path configuration

To implement LUN persistence in a multipath environment, you need to define alias names for your multipath devices. To identify a device's UUID or WWID follow the steps from the single path configuration section. The multipath devices will be created in the /dev/mpath directory. In our example below we will define 4 devices in /etc/multipath.conf:

multipaths { 
	multipath { 
	wwid		3600805f30015987000000000768a0019 
	alias		oramp1 
	} 
	multipath { 
	wwid		3600805f30015987000000000d643001a 
	alias		oramp2 
	} 
	mulitpath { 
	wwid		3600805f3001598700000000086fc001b 
	alias		oramp3 
	} 
	mulitpath { 
	wwid		3600805f300159870000000000984001c 
	alias		oramp4 
	} 
}

This configuration will create 4 LUNs named /dev/mpath/oramp1, /dev/mpath/oramp2, /dev/mpath/oramp3 and /dev/mpath/oramp4. Once entered, the mapping of the devices' WWIDs/UUIDs to their new names will now be persistent even after rebooting.

You may be prompted for Red Hat Enterprise Linux Installation CD-ROMs during the initial installation to add additional software components or packages. This method can make any ISO, or file based image, available as a device for your guest. This might be useful for installing software from an ISO file or other media source(for example, a downloaded image).

The syntax for adding an ISO image as a new device to a guest is as following:

['file:/var/lib/xen/images/win2003sp1.dsk,hda,w',\
'file:/xen/trees/ISO/WIN/en_windows_server_2003_with_sp1_standard.iso,hdc:cdrom,r', ]

In the example above you can see how the file en_windows_server_2003_with_sp1_standard.iso has been added to the guest configuration. Once you reboot your guest this ISO image will be configured as a CD Rom driver inside guest.

Process to setup multiple Red Hat Virtualization bridges:

The challenge in running Red Hat Virtualization on a laptop is that most laptops will connected to the network via wireless network or wired connections. Often these connections are switched multiple times a day. In such an environment Red Hat Virtualization does not behave well as it assumes it has access to the same interface all the time and it also can perform ifup or ifdown calls to the network interface it is using. In addition wireless network cards do not work well in a Red Hat Virtualization environment due to Red Hat Virtualization's (default) bridged network usage.

This setup will also enable you to run Red Hat Virtualization in offline mode when you have no active network connection on your laptop. The easiest solution to run Red Hat Virtualization on a laptop is to follow the procedure outlined below:

Configuring a dummy network interface

Perform the following configuration steps on your host/Dom0:

Configuring NAT(network address translation) for Red Hat Virtualization

Network address translation(NAT) allows multiple network address to connect through a single IP address by intercepting packets and passing them to the private IP addresses. You can copy the following script to /etc/init.d/xenLaptopNAT and create a soft link to /etc/rc3.d/S99xenLaptopNAT. this automatically starts NAT at boot time.

#!/bin/bash
PATH=/usr/bin:/sbin:/bin:/usr/sbin
export PATH
GATEWAYDEV=`ip route | grep default | awk {'print $5'}`
iptables -F
case "$1" in
start)
	if test -z "$GATEWAYDEV"; then
	echo "No gateway device found"
    else
	echo  "Masquerading using $GATEWAYDEV"
	/sbin/iptables -t nat -A POSTROUTING -o $GATEWAYDEV -j MASQUERADE
fi
	echo "Enabling IP forwarding"
	echo 1 > /proc/sys/net/ipv4/ip_forward
	echo "IP forwarding set to `cat /proc/sys/net/ipv4/ip_forward`"
	echo "done."
	;;
*)
echo "Usage: $0 {start|restart|status}"
;;
esac

Configuring dnsmasq for the DNS, DHCP and tftpboot services

One of the challenges in running Red Hat Virtualization on a laptop (or any other computer which is not connected by a single or stable network connection) is the change in network interfaces and availability. Using a dummy network interface helps to build a more stable environment but it also brings up new challenges in providing DHCP, DNS and tftpboot services to your virtual machines/guests. The default DHCP daemon shipped with Red Hat Enterprise Linux and Fedora Core will not listen on dummy interfaces, your DNS forwarded information may change as you connect to different networks and VPNs.

One solution to the above challenges is to use dnsmasq which can provide all of the above service in a single package and will also allow you to control its service only being available to requests from your dummy interface. Below is a short write up on how to configure dnsmasq on a laptop running Red Hat Virtualization:

This section provides the requirements for para-virtualized drivers with Red Hat Enterprise Linux.

Installation

Before you install the para-virtualized drivers the following requirements (listed below) must be met.

You will need the following RPM packages for para-virtualized drivers for each guest operating system installation.

Red Hat Enterprise Linux 5 requires:

Red Hat Enterprise Linux 4 requires:

Red Hat Enterprise Linux 3 requires:

Minimum host operating system version

Minimum guest operating system version

You require at least 50MB of free disk space in the /lib filesystem

This section outlines support restrictions and requirements for using para-virtualized drivers on Red Hat Enterprise Linux. What we support and the restrictions put upon support can be found in the sections below.

Supported Guest Operating Systems

Support for para-virtualized drivers is available for the following operating systems and versions:

You are supported for running a 32 bit guest operating system with para-virtualized drivers on 64 bit Red Hat Enterprise Linux 5 Virtualization.

The table below indicates the kernel variants supported with the para-virtualized drivers. You can use the command shown below to identify the exact kernel revision currently installed on your host. Compare the output against the table to determine if it is supported.

# rpm -q --queryformat '%{NAME}-%{VERSION}-%{RELEASE}.%{ARCH}\n' kernel

The Red Hat Enterprise Linux 5 i686 and x86_64 kernel variants include Symmetric Multiprocessing(SMP), no separate SMP kernel RPM is required.

Take note of processor specific kernel requirements for Red Hat Enterprise Linux 3 Guests in the table below.

Important Restrictions

Para-virtualized device drivers can be installed after successfully installing a guest operating system. You will need a functioning host and guest before you can install these drivers.

After installing the para-virtualized drivers on a guest operating system you should only use the xm command or virsh to start the guests. If xm is not used the network interfaces (for example, eth1 and so on) will not be connected correctly during boot. This problem is known and the Bugzilla number is 300531 and a bug fix is in progress. The bug connects the network interface to qemu-dm and subsequently limits the performance dramatically.

Red Hat Enterprise Linux 3 kernel variant architecture dependencies

For Red Hat Enterprise Linux 3 based guest operating systems you must use the processor specific kernel and para-virtualized driver RPMs as seen in the tables below. If you fail to install the matching para-virtualized driver package loading of the xen-pci-platform module will fail.

The table below shows which host kernel is required to run a Red Hat Enterprise Linux 3 guest on if the guest was compiled for an Intel processor.

The table below shows which host kernel is required to run a Red Hat Enterprise Linux 3 guest on if the guest was compiled for an AMD processor.

The following three chapters describe how to install and configure your fully virtualized guests to run on Red Hat Enterprise Linux 5.1 or above with para-virtualized drivers.

#!/bin/sh
module="xvd"
mode="664"
major=`awk "\\$2==\"$module\" {print \\$1}" /proc/devices`
# < mknod for as many or few partitions on xvd disk attached to FV guest >
# change/add xvda to xvdb, xvbd, etc. for 2nd, 3rd, etc., disk added in
# in xen config file, respectively.
mknod /dev/xvdb b $major 0
mknod /dev/xvdb1 b $major 1
mknod /dev/xvdb2 b $major 2
chgrp disk /dev/xvd*
chmod $mode /dev/xvd*

mknod /dev/xvdc b $major 16
mknod /dev/xvdc1 b $major 17

mknod /dev/xvdd b $major 32
mknod /dev/xvdd1 b $major 33

[root@rhel3]# cat /proc/partitions
major   minor     #blocks   name

  3        0      10485760  hda
  3        1        104391  hda1
  3        2      10377990  hda2
202        0         64000  xvdb
202        1         32000  xvdb1
202        2         32000  xvdb2
253        0       8257536  dm-0
253        1       2031616  dm-1

[root@rhel3]# mkdir /mnt/pvdisk_p1
[root@rhel3]# mkdir /mnt/pvdisk_p2
[root@rhel3]# mount /dev/xvdb1 /mnt/pvdisk_p1
[root@rhel3]# mount /dev/xvdb2 /mnt/pvdisk_p2

[root@rhel3]# df /mnt/pvdisk_p1
Filesystem           1K-blocks      Used   Available Use%  Mounted on
/dev/xvdb1               32000        15       31985   1%  /mnt/pvdisk_p1

[root@rhel4]# cat /proc/partitions
major    minor   #blocks   name

   3        0    10485760  hda
   3        1      104391  hda1
   3        2    10377990  hda2
 202        0       64000  xvdb
 202        1       32000  xvdb1
 202        2       32000  xvdb2
 253        0     8257536  dm-0
 253        1     2031616  dm-1

[root@rhel4]# mkdir /mnt/pvdisk_p1
[root@rhel4]# mkdir /mnt/pvdisk_p2
[root@rhel4]# mount /dev/xvdb1 /mnt/pvdisk_p1
[root@rhel4]# mount /dev/xvdb2 /mnt/pvdisk_p2

[root@rhel4]# df /mnt/pvdisk_p1
Filesystem           1K-blocks      Used   Available Use%  Mounted on
/dev/xvdb1               32000        15       31985   1%  /mnt/pvdisk_p1

[root@rhel5]# ifconfig eth0

[root@rhel5]# cat /proc/partitions
major minor  #blocks    name
   3     0   10485760   hda
   3     1     104391   hda1
   3     2   10377990   hda2
 202     0      64000   xvdb
 202     1      32000   xvdb1
 202     2      32000   xvdb2
 253     0    8257536   dm-0
 253     1    2031616   dm-1

[root@rhel5]# mkdir /mnt/pvdisk_p1
[root@rhel5]# mkdir /mnt/pvdisk_p2
[root@rhel5]# mount /dev/xvdb1 /mnt/pvdisk_p1
[root@rhel5]# mount /dev/xvdb2 /mnt/pvdisk_p2
[root@rhel5]# df /mnt/pvdisk_p1
Filesystem           1K-blocks      Used   Available Use%  Mounted on
/dev/xvdb1               32000        15       31985   1%  /mnt/pvdisk_p1

Once the para-virtualized network driver is loaded you may need to reconfigure the guest's network interface to reflect the driver and virtual ethernet card change.

Perform the following steps to reconfigure the network interface inside the guest.

This section will explain how to add additional virtual network or storage to a guest operating system. For more details on configuring network and storage resources on Red Hat Enterprise Linux 5 Virtualization read the document available on Emerging Technologies, Red Hat.com

vif = [ "mac=00:16:3e:2e:c5:a9,bridge=xenbr0" ]

vif = [ "mac=00:16:3e:2e:c5:a9,bridge=xenbr0",
    "mac=00:16:3e:2f:d5:a9,bridge=xenbr0" ]

# echo 'import virtinst.util ; print virtinst.util.randomMAC()' | python

alias eth1 xen-vnif

# ifconfig eth1

disk = [ "file:/var/lib/xen/images/rhel5_64_fv.dsk,hda,w"]

disk = [ "file:/var/lib/xen/images/rhel5_64_fv.dsk,hda,w",
    "tap:aio:/var/lib/xen/images/UserStorage1.dsk,xvda,w" ]

disk = [ "file:/var/lib/xen/images/rhel5_64_fv.dsk,hda,w",
    "tap:aio:/var/lib/xen/images/UserStorage1.dsk,xvda,w" ] 
    "tap:aio:/var/lib/xen/images/UserStorage2.dsk,xvdb,w" ]

# cat /proc/partitions
major minor  #blocks    name
   3     0   10485760   hda
   3     1     104391   hda1
   3     2   10377990   hda2
 202     0      64000   xvda
 202     1      64000   xvdb
 253     0    8257536   dm-0
 253     1    2031616   dm-1

# mkdir /mnt/pvdisk_xvda
# mkdir /mnt/pvdisk_xvdb
# mount /dev/xvda /mnt/pvdisk_xvda
# mount /dev/xvdb /mnt/pvdisk_xvdb
# df /mnt/pvdisk_p1
Filesystem           1K-blocks      Used   Available Use%  Mounted on
/dev/xvda                64000        15       63985   1%  /mnt/pvdisk_xvda
/dev/xvdb                64000        15       63985   1%  /mnt/pvdisk_xvdb

Below is an example of how to setup a simple environment for live migration. This configuration is using NFS for the shared storage. NFS is suitable for demonstration environments but for a production environment a more robust shared storage configuration using Fibre Channel or iSCSI and GFS is recommended.

The configuration below consists of two servers (et-virt07 and et-virt08), both of them are using eth1 as their default network interface hence they are using xenbr1 as their Red Hat Virtualization networking bridge. We are using a locally attached SCSI disk (/dev/sdb) on et-virt07 for shared storage using NFS.

Setup for live migration

Create and mount the directory used for the migration:

# mkdir /xentest
# mount /dev/sdb /xentest

# cat /etc/exports
/xentest  *(rw,async,no_root_squash)

Verify it is exported via NFS:

# showmount -e et-virt07
Export list for et-virt07:
/xentest *

Install the virtual machine

The install command in the example used for installing the virtual machine:

# virt-install -p -f /xentest/xentestTravelVM01.dsk -s 5 -n\
xentesttravelvm01 --vnc -r 1024 -l http://porkchop.devel.redhat.com/rel-eng/RHEL5-\
Server-20070105.0/4.92/x86_64/os/ -b xenbr1

Verify environment for migration

Make sure the Xen networking bridges are configured correctly and have the same name on both hosts:

[et-virt08 ~]# brctl show
bridge name     bridge id               STP enabled     interfaces
xenbr1          8000.feffffffffff       no              peth1
vif0.1

[et-virt07 ~]# brctl show
bridge name     bridge id               STP enabled     interfaces
xenbr1          8000.feffffffffff       no              peth1
vif0.1

Check the relocation parameters have been configured in the Xen config file on both hosts:

[et-virt07 ~]# grep xend-relocation /etc/xen/xend-config.sxp |grep -v '#'
(xend-relocation-server yes)
(xend-relocation-port 8002)
(xend-relocation-address '')
(xend-relocation-hosts-allow '')

[et-virt08 ~]# grep xend-relocation /etc/xen/xend-config.sxp |grep -v '#'
(xend-relocation-server yes)
(xend-relocation-port 8002)
(xend-relocation-address '')
(xend-relocation-hosts-allow '')

Make sure the Xen relocation server has started and is listening on the dedicated port for Xen migrations (8002):

[et-virt07 ~]# lsof -i :8002
COMMAND  PID  USER   FD   TYPE  DEVICE SIZE NODE NAME
python 3445 root 14u IPv4 10223 TCP *:teradataordbms (LISTEN)

[et-virt08 ~]# lsof -i :8002
COMMAND  PID USER   FD   TYPE DEVICE SIZE NODE NAME
python 3252 root 14u IPv4 10901 TCP *:teradataordbms (LISTEN)

Verify the NFS directory has been mounted on the other host and you can see and access the virtual machine image and file system:

[et-virt08 ~]# df /xentest
Filesystem           1K-blocks      Used Available Use% Mounted on
et-virt07:/xentest    70562400   2379712  64598336   4% /xentest

[et-virt08 ~]# file /xentest/xentestTravelVM01.dsk 
/xentest/xentestTravelVM01.dsk: x86 boot sector; partition 1: ID=0x83,
active, starthead 1, startsector 63, 208782 sectors; partition 2: ID=0x8e, 
starthead 0, startsector 208845, 10265535 sectors, code offset 0x48

[et-virt08 ~]# touch /xentest/foo
[et-virt08 ~]# rm -f /xentest/foo

Verification of save and restore on local host

Start up the virtual machine (if it has not yet):

[et-virt07 ~]# xm li
Name             ID Mem(MiB) VCPUs State   Time(s)
Domain-0         0     1880  8     r-----  50.7

[et-virt07 ~]# xm create xentesttravelvm01
Using config file "/etc/xen/xentesttravelvm01".
Going to boot Red Hat Enterprise Linux Server (2.6.18-1.2961.el5xen)
kernel: /vmlinuz-2.6.18-1.2961.el5xen
initrd: /initrd-2.6.18-1.2961.el5xen.img
Started domain xentesttravelvm01

Verify the virtual machine is running:

[et-virt07 ~]# xm li
Name                     ID Mem(MiB) VCPUs State   Time(s)
Domain-0                 0  983      8     r-----  58.2
xentesttravelvm01        1  1024     1     -b----  9.2

Save the virtual machine on the local host:

[et-virt07 xentest]# time xm save xentesttravelvm01 xentesttravelvm01.sav
real    0m15.744s
user    0m0.188s
sys     0m0.044s

[et-virt07 xentest]# ls -lrt xentesttravelvm01.sav
-rwxr-xr-x 1 root root 1075657716 Jan 12 06:46 xentesttravelvm01.sav

[et-virt07 xentest]# xm li
Name                  ID Mem(MiB) VCPUs State   Time(s)
Domain-0              0  975      8     r-----  110.7

Restore the virtual machine on the local host:

[et-virt07 xentest]# time xm restore xentesttravelvm01.sav
real    0m12.443s
user    0m0.072s
sys     0m0.044s

[et-virt07 xentest]# xm li
Name                   ID Mem(MiB) VCPUs State   Time(s)
Domain-0               0  975      8     r-----  118.5
xentesttravelvm01      3  1023     1     -b----  0.0

Live migration test

Run a simple loop inside the guest to print out time and hostname every 3 seconds.

# while true
> do
> hostname ; date
> sleep 3
> done
dhcp78-218.lab.boston.redhat.com
Fri Jan 12 06:50:16 EST 2007
dhcp78-218.lab.boston.redhat.com
Fri Jan 12 06:50:19 EST 2007
dhcp78-218.lab.boston.redhat.com
Fri Jan 12 06:50:22 EST 2007
dhcp78-218.lab.boston.redhat.com
Fri Jan 12 06:50:25 EST 2007
dhcp78-218.lab.boston.redhat.com
Fri Jan 12 02:22:24 EST 2007
dhcp78-218.lab.boston.redhat.com
Fri Jan 12 02:22:27 EST 2007
dhcp78-218.lab.boston.redhat.com
Fri Jan 12 02:22:30 EST 2007

Verify the virtual machine is running:

[et-virt08 xen]# xm li
Name                      ID Mem(MiB) VCPUs State   Time(s)
Domain-0                  0  975      4     r-----  45.9
xentesttravelvm01         1  1023     1     -b----  1.3

Initiate the live migration to et-virt08. in the example below et-virt07 is the hostname you are migrating to and <domain-id> muyst be replaced with a guest domain available to the host system.

[et-virt08 ~]# xm migrate --live <domain-id> et-virt07

Verify the virtual machine has been shut down on et-virt07

[et-virt07 xentest]# xm li
Name                        ID Mem(MiB) VCPUs State   Time(s)
Domain-0                    0  975      8     r-----  161.1

Verify the virtual machine has been migrated to et-virt08:

[et-virt08 ~]# xm li
Name                        ID Mem(MiB) VCPUs State   Time(s)
Domain-0                    0  975      4     r-----  46.3
xentesttravelvm01           1  1023     1     -b----  1.6

Testing the progress and initiating the live migration

Create the following script inside the virtual machine to log date and hostname during the migration. This script performs I/O tasks on the virtual machine's file system.

#!/bin/bash

while true
do
touch /var/tmp/$$.log
echo `hostname` >>  /var/tmp/$$.log
	echo `date`     >>  /var/tmp/$$.log
	cat  /var/tmp/$$.log
	df /var/tmp
	ls -l  /var/tmp/$$.log
	sleep 3
	done

Remember, that script is only for testing purposes and unnecessary for a live migration in a production environment.

Verify the virtual machine is running on et-virt08 before we try to migrate it to et-virt07:

[et-virt08 ~]# xm li
Name                      ID Mem(MiB) VCPUs State   Time(s)
Domain-0                  0  975      4     r-----  46.3
xentesttravelvm01         1  1023     1     -b----  1.6

Initiate a live migration to et-virt07. You can add the time command to see how long the migration takes:

[et-virt08 ~]# time xm migrate --live xentesttravelvm01 et-virt07
real    0m10.378s
user    0m0.068s
sys     0m0.052s

run the script inside the guest:

# ./doit
dhcp78-218.lab.boston.redhat.com
Fri Jan 12 02:26:27 EST 2007
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/mapper/VolGroup00-LogVol00
2983664   2043120    786536  73% /
-rw-r--r-- 1 root root 62 Jan 12 02:26 /var/tmp/2279.log
dhcp78-218.lab.boston.redhat.com
Fri Jan 12 02:26:27 EST 2007
dhcp78-218.lab.boston.redhat.com
Fri Jan 12 02:26:30 EST 2007
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/mapper/VolGroup00-LogVol00
2983664   2043120    786536  73% /
-rw-r--r-- 1 root root 124 Jan 12 02:26 /var/tmp/2279.log
dhcp78-218.lab.boston.redhat.com
Fri Jan 12 02:26:27 EST 2007
dhcp78-218.lab.boston.redhat.com
Fri Jan 12 02:26:30 EST 2007
dhcp78-218.lab.boston.redhat.com
Fri Jan 12 02:26:33 EST 2007
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/mapper/VolGroup00-LogVol00
2983664   2043120    786536  73% /
-rw-r--r-- 1 root root 186 Jan 12 02:26 /var/tmp/2279.log
Fri Jan 12 02:26:45 EST 2007
dhcp78-218.lab.boston.redhat.com
Fri Jan 12 02:26:48 EST 2007
dhcp78-218.lab.boston.redhat.com
Fri Jan 12 02:26:51 EST 2007
dhcp78-218.lab.boston.redhat.com
Fri Jan 12 06:54:57 EST 2007
dhcp78-218.lab.boston.redhat.com
Fri Jan 12 06:55:00 EST 2007
dhcp78-218.lab.boston.redhat.com
Fri Jan 12 06:55:03 EST 2007
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/mapper/VolGroup00-LogVol00
2983664   2043120    786536  73% /
-rw-r--r-- 1 root root 744 Jan 12 06:55 /var/tmp/2279.log
dhcp78-218.lab.boston.redhat.com
Fri Jan 12 02:26:27 EST 2007

Verify the virtual machine has been shut down on et-virt08:

[et-virt08 ~]# xm li
Name                    ID Mem(MiB) VCPUs State   Time(s)
Domain-0                0  975      4     r-----  56.3

Verify the virtual machine has started up on et-virt07:

[et-virt07 xentest]# xm li
Name                    ID Mem(MiB) VCPUs State   Time(s)
Domain-0                0  975      8     r-----  198.1
xentesttravelvm01       4  1023     1     -b----  1.0

Run through another cycle migrating from et-virt07 to et-virt08. Initiate a migration from et-virt07 to et-virt08:

[et-virt07 xentest]# time xm migrate --live xentesttravelvm01 et-virt08
real    0m11.490s
user    0m0.084s
sys     0m0.028s

Verify the virtual machine has been shut down:

[et-virt07 xentest]# xm li
Name                      ID Mem(MiB) VCPUs State   Time(s)
Domain-0                  0  975      8     r-----  221.7

Before initiating the migration start the simple script in the guest and note the change in time when migrating the guest:

# ./doit
dhcp78-218.lab.boston.redhat.com
Fri Jan 12 06:57:53 EST 2007
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/mapper/VolGroup00-LogVol00
2983664   2043120    786536  73% /
-rw-r--r-- 1 root root 62 Jan 12 06:57 /var/tmp/2418.log
dhcp78-218.lab.boston.redhat.com
Fri Jan 12 06:57:53 EST 2007
dhcp78-218.lab.boston.redhat.com
Fri Jan 12 06:57:56 EST 2007
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/mapper/VolGroup00-LogVol00
2983664   2043120    786536  73% /
-rw-r--r-- 1 root root 124 Jan 12 06:57 /var/tmp/2418.log
dhcp78-218.lab.boston.redhat.com
Fri Jan 12 06:57:53 EST 2007
dhcp78-218.lab.boston.redhat.com
Fri Jan 12 06:57:56 EST 2007
dhcp78-218.lab.boston.redhat.com
Fri Jan 12 06:58:00 EST 2007
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/mapper/VolGroup00-LogVol00
2983664   2043120    786536  73% /
-rw-r--r-- 1 root root 186 Jan 12 06:57 /var/tmp/2418.log
dhcp78-218.lab.boston.redhat.com
Fri Jan 12 06:57:53 EST 2007
dhcp78-218.lab.boston.redhat.com
Fri Jan 12 06:57:56 EST 2007
dhcp78-218.lab.boston.redhat.com
Fri Jan 12 06:58:00 EST 2007
dhcp78-218.lab.boston.redhat.com
Fri Jan 12 02:30:00 EST 2007
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/mapper/VolGroup00-LogVol00
2983664   2043120    786536  73% /
-rw-r--r-- 1 root root 248 Jan 12 02:30 /var/tmp/2418.log
dhcp78-218.lab.boston.redhat.com
Fri Jan 12 06:57:53 EST 2007
dhcp78-218.lab.boston.redhat.com
Fri Jan 12 06:57:56 EST 2007
dhcp78-218.lab.boston.redhat.com
Fri Jan 12 06:58:00 EST 2007
dhcp78-218.lab.boston.redhat.com
Fri Jan 12 02:30:00 EST 2007
dhcp78-218.lab.boston.redhat.com
Fri Jan 12 02:30:03 EST 2007
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/mapper/VolGroup00-LogVol00
2983664   2043120    786536  73% /
-rw-r--r-- 1 root root 310 Jan 12 02:30 /var/tmp/2418.log
dhcp78-218.lab.boston.redhat.com
Fri Jan 12 06:57:53 EST 2007
dhcp78-218.lab.boston.redhat.com
Fri Jan 12 06:57:56 EST 2007
dhcp78-218.lab.boston.redhat.com
Fri Jan 12 06:58:00 EST 2007
dhcp78-218.lab.boston.redhat.com
Fri Jan 12 02:30:00 EST 2007
dhcp78-218.lab.boston.redhat.com
Fri Jan 12 02:30:03 EST 2007
dhcp78-218.lab.boston.redhat.com
Fri Jan 12 02:30:06 EST 2007
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/mapper/VolGroup00-LogVol00
2983664   2043120    786536  73% /
-rw-r--r-- 1 root root 372 Jan 12 02:30 /var/tmp/2418.log

After the migration command completes on et-virt07 verify on et-virt08 that the virtual machine has started:

[et-virt08 ~]# xm li
Name                       ID Mem(MiB) VCPUs State   Time(s)
Domain-0                   0  975      4     r-----  67.3
xentesttravelvm01          2  1023     1     -b----  0.4

and run another cycle:

[et-virt08 ~]# time xm migrate --live xentesttravelvm01 et-virt07
real    0m10.378s
user    0m0.068s
sys     0m0.052s

At this point you have successfully performed an offline and a live migration test.

The ssh tool can be used to manage remote virtual machines. The method described uses the libvirt management connection securely tunneled over an SSH connection to manage the remote machines. All the authentication is done using SSH public key cryptography and passwords or passphrases gathered by your local SSH agent. In addition the VNC console for each guest virtual machine will be tunneled over SSH.

SSH is usually configured by default so you probably already have SSH keys setup and no extra firewall rules needed to access the management service or VNC console.

Be aware of the issues with using SSH for remotely managing your virtual machines, including:

Configuring SSH access for virt-manager

The following instructions assume you are starting from scratch and do not already have SSH keys set up.

The libvirt daemon (libvirtd)

The libvirt daemon provide an interface for managing virtual machines. You must have the libvirtd daemon installed and running on every remote host that you need to manage. Using Red Hat Virtualization may require a special kernel or CPU hardware support, see Chapter 1, System requirements for details.

$ ssh root@somehost
# chkconfig libvirtd on
# service libvirtd start

After libvirtd and SSH are configured you should be able to remotely access and manage your virtual machines. You should also be able to access your guests with VNC at this point.

You can manage virtual machines using TLS and SSL. TLS and SSL provides greater scalability but is more complicated than ssh (see Section 18.1, “Remote management with ssh”. TLS and SSL is the same technology used by web browsers for secure connections. The libvirt management connection will open a TCP port for incoming connections, which is securely encrypted and authenticated based on x509 certificates. In addition the VNC console for each guest virtual machine will be setup to use TLS with x509 certificate authentication.

Using this method you will not need to give users shell accounts on the remote machines being managed. However, extra firewall rules are needed to access the management service or VNC console. Certificate revocation lists can be used to revoke access to users.

Steps to setup TLS/SSL access for virt-manager

The following short guide assuming you are starting from scratch and you do not have any TLS/SSL certificate knowledge. If you are lucky enough to have a certificate management server you can probably skip the first steps.

To enable SSL and TLS for VNC, it is necessary to put the certificate authority and client certificates into $HOME/.pki, that is the following three files:

Red Hat Virtualization is a collection of software components that work together to host and manage virtual machines. The Virtual Machine Manager (VMM) gives you a graphical view of the virtual machines on your system. You can use VMM to define both para-virtual and full virtual machines. Using Virtual Machine Manager, you can perform any number of virtualization management tasks including assigning memory, assigning virtual CPUs, monitoring operational performance, and save, restore, pause, resume, and shutdown virtual systems. It also allows you to access the textual and graphical console. Red Hat Virtualization abstracts CPU and memory resources from the underlying hardware and network configurations. This enables processing resources to be pooled and dynamically assigned to applications and service requests. Chip-level virtualization enables operating systems with Intel VT and AMD-V hardware to run on hypervisors.

This window appears first and prompts the user to choose a hypervisor session. Non-privileged users can initiate a read-only session. Root users can start a session with full blown read-write status. For normal use, select the Local Xen host option. You start the Virtual Machine Manager test mode by selecting the Other hypervisor and then type test:///default in the URL field beneath. Once in test mode, you can connect to a libvirt dummy hypervisor. Note that although the Remote Xen host screen is visible, the functionality to connect to such a host is not implemented into Red Hat Enterprise Linux 5.

Figure 21.1. Virtual Machine Manager connection window

This main window displays all the running virtual machines and resources currently allocated to them (including domain0). You can decide which fields to display. Double-clicking on the desired virtual machine brings up the respective console for that particular machine. Selecting a virtual machine and double-click the Details button to display the Details window for that machine. You can also access the File menu to create a new virtual machine.

Figure 21.2. Virtual Machine Manager main window

This window displays graphs and statistics of a guest's live resource utilization data available from the Red Hat Virtualization Virtual Machine Manager. The UUID field displays the globally unique identifier for the virtual machines.

Figure 21.3. Virtual Machine Manager details window

This window displays a virtual machine's graphical console. Para-virtualized and fully virtualized guests use different techniques to export their local virtual framebuffers, but both technologies use VNC to make them available to the Virtual Machine Manager's console window. If your virtual machine is set to require authentication, the Virtual Machine Graphical console prompts you for a password before the display appears.

Figure 21.4. Graphical console window

Your local desktop can intercept key combinations (for example, Ctrl+Alt+F11) to prevent them from being sent to the guest machine. You can use virt-managersticky key' capability to send these sequences. You must press any modifier key (Ctrl or Alt) 3 times and the key you specify gets treated as active until the next non-modifier key is pressed. Then you can send Ctrl-Alt-F11 to the guest by entering the key sequence 'Ctrl Ctrl Ctrl Alt+F1'.

To start virt-manager session, from the Applications menu, click System Tools and select Virtual Machine Manager(virt-manager).

The virt-manager main window appears.

Figure 21.5. Starting virt-manager

Alternatively, virt-manager can be started remotely using ssh as demonstrated in the following command:

ssh -X host's address
[remotehost]# virt-manager

Using ssh to manage virtual machines and hosts is discussed further in Section 18.1, “Remote management with ssh”.

virt-manager is the desktop application which can be used to manage guests.

You can use Red Hat's Virtual Machine Manager to:

Before creating new guest virtual machines you should consider the following options. This list is a summery of the installation process using the Virtual Machine Manager.

VNC is used for graphical installations.

These are the steps required to install a guest operating system on Red Hat Enterprise Linux 5 using the Virtual Machine Monitor:

After you start the Virtual Machine Manager, all virtual machines on your system are displayed in the main window. Domain0 is your host system. If there are no machines present, this means that currently there are no machines running on the system.

To restore a previously saved session:

The saved virtual system appears in the Virtual Machine Manager main window.

Figure 21.20. A restored virtual machine manager session

You can use the Virtual Machine Monitor to view activity data information for any virtual machines on your system.

To view a virtual system's details:

You can use the Virtual Machine Manager to modify the virtual system Status monitoring.

To configure Status monitoring, and enable Consoles:

To view the domain IDs for all virtual machines on your system:

To view the amount of virtual CPUs for all virtual machines on your system:

To view the CPU usage for all virtual machines on your system:

To view the memory usage for all virtual machines on your system:

To configure a virtual network on your system:

To create a virtual network on your system:

The virsh command is an alternative command to manage a Red Hat Enterprise Linux 5 Xen environment using a Command Line Interface(CLI). The virsh command is provided as part of the libvirt API which provides a common API to applications requiring standardized access to interact with Xen using a stable interface. Besides providing virsh the libvirt API also provide a higher level language API for C, Python, and Perl (via the CPAN network) The virsh command is an interactive shell which can either be used in scripts or interactively. The following virsh commands assume you have already run virsh and are at a virsh# prompt.

Basic management options

The following are basic and commonly used virsh commands:

Resource management options

Use the following virsh commands to manage resources:

Monitoring and troubleshooting options

Use the following virsh commands for monitoring and troubleshooting:

Presently unsupported options

The connect command that is used to connect or reconnect to a hypervisor.

virsh command output

The following are example outputs from common virsh commands:

The xm command is used to manage your Red Hat Virtualization environment using a CLI interface. Most operations can be performed by the virt-manager application, including a CLI interface which is part of virt-manager. However, there are a few operations which currently can not be performed using virt-manager. As the xm command is part of the Xen environment a few options available with the xm command will not work in a Red Hat Enterprise Linux 5 environment. The list below provides an overview of command options available (and unavailable) in a Red Hat Enterprise Linux 5 environment. As an alternative to using the xm command one can also use the virsh command which is provided as part of the Red Hat Enterprise Linux 5 Xen environment . The virsh command is layered on top of the libvirt API which can provide a number of benefits over using the xm command. Namely the ability to use virsh in scripts and the ability to manage other hypervisors as they are integrated into the libvirt API.

Basic management options

The following are basic and commonly used xm commands:

Resource management options

Use the following xm commands to manage resources:

Monitoring and troubleshooting options

Use the following xm commands for monitoring and troubleshooting:

Currently unsupported options

The xm vnet-list is currently unsupported.

This section explains how to make guest systems boot automatically during the host system's boot phase. You need to configure soft links in /etc/xen/auto to point to the guest configuration file of the guests you want to automatically boot. It is recommended to keep the number of guests small because the boot order of guests is serialized. More guests automatically started at boot cause the boot sequence to take a significantly longer time.

The example below shows how you can configure the softlink for a guest image named example to automatic boot during the system boot.

# cd /etc/xen/auto
# ls
# ln -s /var/lib/xen/images/example .
# ls -l
lrwxrwxrwx 1 root root 14 Dec 14 10:02 example -> ../example

This section describes how to safely and correctly change your /etc/grub.conf file to use the virtualization kernel. You must use the virtualization kernel for domain0 in order to successfully run the hypervisor. Copy your existing virtualized kernel entry make sure you copy all of the important lines or your system will panic upon boot (initrd will have a length of '0'). You need specify hypervisor specific values you have to add them to the xen line of your grub entry.

The output below is an example of a grub.conf entry from a Red Hat Virtualization system. The grub.conf on your system may vary. The important part in the example below is the section from the title line to the next new line.

#boot=/dev/sda
default=0
timeout=15
#splashimage=(hd0,0)/grub/splash.xpm.gz hiddenmenu
serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1
terminal --timeout=10 serial console

title Red Hat Enterprise Linux Server (2.6.17-1.2519.4.21.el5xen)
	root (hd0,0)
	kernel /xen.gz-2.6.17-1.2519.4.21.el5 com1=115200,8n1
	module /vmlinuz-2.6.17-1.2519.4.21.el5xen ro root=/dev/VolGroup00/LogVol00
	module /initrd-2.6.17-1.2519.4.21.el5xen.img

To set the amount of memory assigned to your host system at boot time to 256MB you need to append dom0_mem=256M to the xen line in your grub.conf. A modified version of the grub configuration file in the previous example:

#boot=/dev/sda
default=0
timeout=15
#splashimage=(hd0,0)/grub/splash.xpm.gz
hiddenmenu
serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1
terminal --timeout=10 serial console

title Red Hat Enterprise Linux Server (2.6.17-1.2519.4.21.el5xen)
	root (hd0,0)
	kernel /xen.gz-2.6.17-1.2519.4.21.el5 com1=115200,8n1 dom0_mem=256MB
	module /vmlinuz-2.6.17-1.2519.4.21.el5xen ro
	root=/dev/VolGroup00/LogVol00
	module /initrd-2.6.17-1.2519.4.21.el5xen.img

The following configuration files can be used as reference examples. Normally the configuration will be created by virt-install or virt-manager during the installation of a guest. However sometimes it might be useful to have a reference available in case a new configuration needs manual creation.

Example: para-virtualized guest's configuration file

An example of a para-virtualized guest's configuration file:

name = "rhel5b2vm01"
memory = "2048"
disk = [ 'tap:aio:/var/lib/xen/images/rhel5b2vm01.dsk,xvda,w', ]
vif = [ 'mac=00:16:3e:33:79:3c, bridge=xenbr0', ]
vnc=1
vncunused=1
uuid = "302bd9ce-4f60-fc67-9e40-7a77d9b4e1ed"
bootloader="/usr/bin/pygrub"
vcpus=2
on_reboot   = 'restart'
on_crash    = 'restart'

An example of a fully virtualized guest's configuration file:

name = "rhel4u4-x86_64"
builder = "hvm"
memory = "500"
disk = [ 'file:/var/lib/xen/images/rhel4u4-x86_64.dsk,hda,w', ]
vif = [ 'type=ioemu, mac=00:16:3e:09:f0:12, bridge=xenbr0', 'type=ioemu, 
mac=00:16:3e:09:f0:13, bridge=xenbr1' ]
uuid = "b10372f9-91d7-a05f-12ff-372100c99af5"
device_model = "/usr/lib64/xen/bin/qemu-dm"
kernel = "/usr/lib/xen/boot/hvmloader"
vnc=1
vncunused=1
apic=1
acpi=1
pae=1
vcpus=1
serial = "pty" # enable serial console
on_reboot   = 'restart'

This section outlines copying an existing configuration file to create a new guest. There are key parameters in your guest's configuration file you must be aware of, and modify, to successfully duplicate a guest.

Now, adjust the system configuration settings on your guest:

The script below can identify if the environment an application or script is running in is a para-virtualized, a fully virtualized guest or on the hypervisor. The script below can be used to identify running on:

#!/bin/bash
declare -i IS_HVM=0
declare -i IS_PARA=0
check_hvm()
{
	IS_X86HVM="$(strings /proc/acpi/dsdt | grep int-xen)"
	  if [ x"${IS_X86HVM}" != x ]; then
	   echo "Guest type is full-virt x86hvm"
	   IS_HVM=1
	fi
}
check_para()
{
	if $(grep -q control_d /proc/xen/capabilities); then
	  echo "Host is dom0"
	  IS_PARA=1
	else
	  echo "Guest is para-virt domU"
	  IS_PARA=1
	fi
}
if [ -f /proc/acpi/dsdt ]; then 
	check_hvm
fi

if [ ${IS_HVM} -eq 0 ]; then
	if [ -f /proc/xen/capabilities ] ; then
		check_para
	fi
     fi
if [ ${IS_HVM} -eq 0 -a ${IS_PARA} -eq 0 ]; then
	echo "Baremetal platform"
fi

In some case you will need to generate a new and unique MAC address for a guest. There is no command line tool available to generate a new MAC address at the time of writing. The script provided below can generate a new MAC address for your guests. Save the script to your guest as macgen.py. Now from that directory you can run the script using ./macgen.py . and it will generate a new MAC address. A sample output would look like the following:

$ ./macgen.py 
00:16:3e:20:b0:11
	
#!/usr/bin/python
# macgen.py script to generate a MAC address for Red Hat Virtualization guests
#
import random
#
def randomMAC():
	mac = [ 0x00, 0x16, 0x3e,
		random.randint(0x00, 0x7f),
		random.randint(0x00, 0xff),
		random.randint(0x00, 0xff) ]
	return ':'.join(map(lambda x: "%02x" % x, mac))
#
print randomMAC()

Another method to generate a new MAC and for your guest

You can also use the built-in modules of python-virtinst to generate a new MAC address and UUID for use in a guest configuration file:

# echo  'import virtinst.util ; print\
 virtinst.util.uuidToString(virtinst.util.randomUUID())' | python
# echo  'import virtinst.util ; print virtinst.util.randomMAC()' | python

The script above can also be implemented as a script file as seen below.

#!/usr/bin/env python
#  -*- mode: python; -*-
print ""
print "New UUID:"
import virtinst.util ; print virtinst.util.uuidToString(virtinst.util.randomUUID())
print "New MAC:"
import virtinst.util ; print virtinst.util.randomMAC()
print ""

In some environments it may be required to limit the network bandwidth available to certain guests. This can be used to implement basic Quality of Service on a host running multiple virtual machines. By default the virtual machine will be able to use any bandwidth setting available on your physical network card supports. The physical network card must be mapped to one of virtual machine's virtual network interfaces. In Red Hat Virtualization the “rate” parameter part of the VIF entries can be used to achieve the goal of throttling certain virtual machines.

This list covers the variables

Examples of rate parameter values and uses.

In the virtual machine configuration a sample VIF entry would look like the following:

vif = [ 'rate=10MB/s , mac=00:16:3e:7a:55:1c, bridge=xenbr1']

This rate entry would limit the virtual machine's interface to 10MB/s for outgoing traffic

You can configure your guests to start automatically when you boot the system. To do this, you must modify the symbolic links that resides in /etc/xen/auto . This file points to the guest configuration files that you need to start automatically. The start up process is serialized, meaning that the higher the number of guests, the longer the boot process will take. This example shows you how to use symbolic links for the guest rhel5vm01 :

# cd /etc/xen
# cd auto
# ls
# ln -s ../rhel5vm01 .
# ls -l
lrwxrwxrwx 1 root root 14 Dec 14 10:02 rhel5vm01 -> ../rhel5vm01
#

To use Red Hat Virtualization to manage domain0, you will constantly making changes to the grub.conf configuration file, that resides in the /etc directory. Because of the large number of domains to manage, many system administrators prefer to use the 'cut and paste' method when editing grub.conf . If you do this, make sure that you include all five lines in the Virtualization entry (or this will create system errors). If you require Xen hypervisor specific values, you must add them to the 'xen' line. This example represents a correct grub.conf Virtualization entry:

# boot=/dev/sda/
default=0
timeout=15
#splashimage=(hd0, 0)/grub/splash.xpm.gz
hiddenmenu
serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1
terminal --timeout=10 serial console
title Red Hat Enterprise Linux Server (2.6.17-1.2519.4.21. el5xen)
root (hd0, 0)
kernel /xen.gz-2.6.17-1.2519.4.21.el5 com1=115200,8n1
module /vmlinuz-2.6.17-1.2519.4.21el5xen ro root=/dev/VolGroup00/LogVol00
module /initrd-2.6.17-1.2519.4.21.el5xen.img

For example, if you need to change your dom0 hypervisor's memory to 256MB at boot time, you must edit the 'xen' line and append it with the correct entry, 'dom0_mem=256M' . This example represents the respective grub.conf xen entry:

# boot=/dev/sda
default=0
timeout=15
#splashimage=(hd0,0)/grubs/splash.xpm.gz
hiddenmenu
serial --unit=0 --speed =115200 --word=8 --parity=no --stop=1
terminal --timeout=10 serial console
title Red Hat Enterprise Linux Server (2.6.17-1.2519.4.21. el5xen)
root (hd0,0)
kernel /xen.gz-2.6.17-1.2519.4.21.el5 com1=115200,8n1 dom0_mem=256MB
module /vmlinuz-2.6.17-1.2519.4.21.el5xen ro
root=/dev/VolGroup00/LogVol00
module /initrd-2.6.17-1.2519.4.21.el5xen.img

Red Hat Virtualization can migrate virtual machines between other servers running Red Hat Virtualization. Further, migration is performed in an offline method (using the xm migrate command). Live migration can be done from the same command. However there are some additional modifications that you must do to the xend-config configuration file. This example identifies the entries that you must modify to ensure a successful migration: