Edition 5.8
1801 Varsity Drive
Raleigh, NC 27606-2072 USA
Phone: +1 919 754 3700
Phone: 888 733 4281
Fax: +1 919 754 3701
ftpd Mono-spaced Bold
To see the contents of the filemy_next_bestselling_novelin your current working directory, enter thecat my_next_bestselling_novelcommand at the shell prompt and press Enter to execute the command.
Press Enter to execute the command.Press Ctrl+Alt+F2 to switch to the first virtual terminal. Press Ctrl+Alt+F1 to return to your X-Windows session.
mono-spaced bold. For example:
File-related classes includefilesystemfor file systems,filefor files, anddirfor directories. Each class has its own associated set of permissions.
Choose → → from the main menu bar to launch Mouse Preferences. In the Buttons tab, click the Left-handed mouse check box and click to switch the primary mouse button from the left to the right (making the mouse suitable for use in the left hand).To insert a special character into a gedit file, choose → → from the main menu bar. Next, choose → from the Character Map menu bar, type the name of the character in the Search field and click . The character you sought will be highlighted in the Character Table. Double-click this highlighted character to place it in the Text to copy field and then click the button. Now switch back to your document and choose → from the gedit menu bar.
Mono-spaced Bold ItalicProportional Bold Italic
To connect to a remote machine using ssh, typesshat a shell prompt. If the remote machine isusername@domain.nameexample.comand your username on that machine is john, typessh john@example.com.Themount -o remountcommand remounts the named file system. For example, to remount thefile-system/homefile system, the command ismount -o remount /home.To see the version of a currently installed package, use therpm -qcommand. It will return a result as follows:package.package-version-release
Publican is a DocBook publishing system.
mono-spaced roman and presented thus:
books Desktop documentation drafts mss photos stuff svn books_tests Desktop1 downloads images notes scripts svgs
mono-spaced roman but add syntax highlighting as follows:
package org.jboss.book.jca.ex1; import javax.naming.InitialContext; public class ExClient { public static void main(String args[]) throws Exception { InitialContext iniCtx = new InitialContext(); Object ref = iniCtx.lookup("EchoBean"); EchoHome home = (EchoHome) ref; Echo echo = home.create(); System.out.println("Created Echo"); System.out.println("Echo.echo('Hello') = " + echo.echo("Hello")); } }
5.
NFS, FTP or HTTP protocols.
/var/lib/libvirt/images/ directory by default. If you use a different directory you must label the new directory according to SELinux policy. Refer to Section 18.2, “SELinux and virtualization” for details.
yum for more information.
grub.conf file.
virtio drivers, are PCI devices. Presently, guests are limited to a maximum of 32 PCI devices. Some PCI devices are critical for the guest to run and these devices cannot be removed. The default, required devices are:
fstab file, the initrd file or used by the kernel command line. If less privileged users, especially virtualized guests, have write access to whole partitions or LVM volumes the host system could be compromised.
/dev/sdb). Virtualized guests with access to block devices may be able to access other block devices on the system or modify volume labels which can be used to compromise the host system. Use partitions (for example, /dev/sdb1) or LVM volumes to prevent this issue.
phy devices it can have if it has sufficient resources.
/etc/modprobe.conf add the following line:
options loop max_loop=64
# rmmod loop # modprobe loop
Table of Contents
yum command and the Red Hat Network (RHN).
virt-manager, libvirt and virt-viewer and all dependencies for installation.
Customize the packages (if required)
%packages section of your Kickstart file, append the following package group:
%packages @xen
xen-ia64-guest-firmware
rhn_register command and follow the prompts.
yumxen and kernel-xen packages. The xen package contains the hypervisor and basic virtualization tools. The kernel-xen package contains a modified Linux kernel which runs as a virtual machine guest on the hypervisor.
xen and kernel-xen packages, run:
# yum install xen kernel-xen
xen-ia64-guest-firmware) from the supplementary installation DVD. This package can also be can be installed from RHN with the yum command:
# yum install xen-ia64-guest-firmware
python-virtinstvirt-install command for creating virtual machines.
libvirtlibvirt is an API library for interacting with hypervisors. libvirt uses the xm virtualization framework and the virsh command line tool to manage and control virtual machines.
libvirt-pythonlibvirt API.
virt-managervirt-manager, also known as Virtual Machine Manager, provides a graphical tool for administering virtual machines. It uses libvirt library as the management API.
# yum install virt-manager libvirt libvirt-python python-virtinst
virt-manager, libvirt and virt-viewer for installation.
Customize the packages (if required)
%packages section of your Kickstart file, append the following package group:
%packages @kvm
rhn_register command and follow the prompts.
yumkvm package. The kvm package contains the KVM kernel module providing the KVM hypervisor on the default Red Hat Enterprise Linux kernel.
kvm package, run:
# yum install kvm
python-virtinstvirt-install command for creating virtual machines.
libvirtlibvirt is an API library for interacting with hypervisors. libvirt uses the xm virtualization framework and the virsh command line tool to manage and control virtual machines.
libvirt-pythonlibvirt API.
virt-managervirt-manager, also known as Virtual Machine Manager, provides a graphical tool for administering virtual machines. It uses libvirt library as the management API.
# yum install virt-manager libvirt libvirt-python python-virtinst
virt-install. Both methods are covered by this chapter.
virt-install command to create virtualized guests from the command line. virt-install is used either interactively or as part of a script to automate the creation of virtual machines. Using virt-install with Kickstart files allows for unattended installation of virtual machines.
virt-install tool provides a number of options one can pass on the command line. To see a complete list of options run:
$ virt-install --help
virt-install man page also documents each command option and important variables.
qemu-img is a related command which may be used before virt-install to configure storage options.
--vnc option which opens a graphical window for the guest's installation.
rhel3support, from a CD-ROM, with virtual networking and with a 5 GB file-based block device image. This example uses the KVM hypervisor.
# virt-install --accelerate --hvm --connect qemu:///system \ --network network:default \ --name rhel3support --ram=756\ --file=/var/lib/libvirt/images/rhel3support.img \ --file-size=6 --vnc --cdrom=/dev/sr0
# virt-install --name fedora11 --ram 512 --file=/var/lib/libvirt/images/fedora11.img \ --file-size=3 --vnc --cdrom=/var/lib/libvirt/images/fedora11.iso
virt-manager, also known as Virtual Machine Manager, is a graphical tool for creating and managing virtualized guests.
Open virt-manager
virt-manager. Launch the application from the menu and submenu. Alternatively, run the virt-manager command as root.
Optional: Open a remote hypervisor
Create a new guest
New guest wizard
Name the virtual machine
Choose virtualization method
kernel-xen is not the kernel running presently.
Select the installation method
.iso file).
HTTP, FTP or NFS.
HTTP, FTP or NFS. The installation media URL must contain a Red Hat Enterprise Linux installation tree. This tree is hosted using NFS, FTP or HTTP.
Installation media selection
ISO image or physical media installation
Network install tree installation
NFS, FTP or HTTP. Optionally, a kickstart file can be specified to automated the installation. Kernel parameters can also be specified if required.
Network boot (PXE)
Storage setup
/var/lib/libvirt/images/ directory by default. In the default configuration, other directory locations for file-based images are prohibited by SELinux. If you use a different directory you must label the new directory according to SELinux policy. Refer to Section 18.2, “SELinux and virtualization” for details.
/var/lib/libvirt/images/. If you are using a different location (such as /images/ in this example) make sure it is added to your SELinux policy and relabeled before you continue with the installation (later in the document you will find information on how to modify your SELinux policy).
Network setup
Memory and CPU allocation
Verify and start guest installation
virt-manager. Chapter 8, Guest operating system installation procedures contains step-by-step instructions to installing a variety of common operating systems.
Create a new bridge
/etc/sysconfig/network-scripts/ directory. This example creates a file named ifcfg-installation which makes a bridge named installation.
# cd /etc/sysconfig/network-scripts/ # vim ifcfg-installation DEVICE=installation TYPE=Bridge BOOTPROTO=dhcp ONBOOT=yes
TYPE=Bridge, is case-sensitive. It must have uppercase 'B' and lower case 'ridge'.
ifup installation command can start the individual bridge but it is safer to test the entire network restarts properly.
# service network restart
brctl show command to view details about network bridges on the system.
# brctl show bridge name bridge id STP enabled interfaces installation 8000.000000000000 no virbr0 8000.000000000000 yes
virbr0 bridge is the default bridge used by libvirt for Network Address Translation (NAT) on the default Ethernet device.
Add an interface to the new bridge
BRIDGE parameter to the configuration file with the name of the bridge created in the previous steps.
# Intel Corporation Gigabit Network Connection DEVICE=eth1 BRIDGE=installation BOOTPROTO=dhcp HWADDR=00:13:20:F7:6E:8E ONBOOT=yes
# service network restart
brctl show command:
# brctl show bridge name bridge id STP enabled interfaces installation 8000.001320f76e8e no eth1 virbr0 8000.000000000000 yes
Security configuration
iptables to allow all traffic to be forwarded across the bridge.
# iptables -I FORWARD -m physdev --physdev-is-bridged -j ACCEPT # service iptables save # service iptables restart
iptables rules. In /etc/sysctl.conf append the following lines:
net.bridge.bridge-nf-call-ip6tables = 0 net.bridge.bridge-nf-call-iptables = 0 net.bridge.bridge-nf-call-arptables = 0
sysctl.
# sysctl -p /etc/sysctl.conf
Restart libvirt before the installation
libvirt daemon.
# service libvirtd reload
virt-install append the --network=bridge:installation installation parameter where installation is the name of your bridge. For PXE installations use the --pxe parameter.
# virt-install --accelerate --hvm --connect qemu:///system \
--network=bridge:installation --pxe\
--name EL10 --ram=756 \
--vcpus=4
--os-type=linux --os-variant=rhel5
--file=/var/lib/libvirt/images/EL10.img \
Select PXE
Select the bridge
Start the installation
virsh update-device Guest1 ~/Guest1.xml (substituting your guest's name and XML file), and select OK to continue past this step.
kernel-xen kernel.
virt-manager, refer to the procedure in Section 7.2, “Creating guests with virt-manager”.
virt-install tool. The --vnc option shows the graphical installation. The name of the guest in the example is rhel5PV, the disk image file is rhel5PV.dsk and a local mirror of the Red Hat Enterprise Linux 5 installation tree is ftp://10.1.1.1/trees/RHEL5-B2-Server-i386/. Replace those values with values accurate for your system and network.
# virt-install -nrhel5PV-r 500 \ -f /var/lib/libvirt/images/rhel5PV.dsk-s 3 --vnc -p \ -lftp://10.1.1.1/trees/RHEL5-B2-Server-i386/
DHCP (as shown below) or a static IP address:
Installation Number field:
rhn_register command. The rhn_register command requires root access.
virt-install in Section 8.1, “Installing Red Hat Enterprise Linux 5 as a para-virtualized guest”. If you used the default example the name is rhel5PV.
virsh to reboot the guest:
# virsh reboot rhel5PVvirt-manager, select the name of your guest, click , then click .
kdump. The use of kdump is unsupported on para-virtualized guests.
yum command or RHN. Click .
Open virt-manager
virt-manager. Launch the application from the menu and submenu. Alternatively, run the virt-manager command as root.
Select the hypervisor
qemu.
Start the new virtual machine wizard
Name the virtual machine
Choose a virtualization method
Select the installation method
Locate installation media
/var/lib/libvirt/images/ directory. Any other location may require additional configuration for SELinux, refer to Section 18.2, “SELinux and virtualization” for details.
Storage setup
/var/lib/libvirt/images/ directory. Assign sufficient space for your virtualized guest and any applications the guest requires.
Network setup
Memory and CPU allocation
Verify and start guest installation
Installing Red Hat Enterprise Linux
Starting virt-manager
Naming your virtual system
Choosing a virtualization method
Choosing an installation method
/var/lib/libvirt/images/ directory. Any other location may require additional configuration for SELinux, refer to Section 18.2, “SELinux and virtualization” for details.
Choose installation image
/var/lib/libvirt/images/ directory by default. In the default configuration, other directory locations for file-based images are prohibited by SELinux. If you use a different directory you must label the new directory according to SELinux policy. Refer to Section 18.2, “SELinux and virtualization” for details.
/var/lib/libvirt/images/. If you are using a different location (such as /images/ in this example) make sure it is added to your SELinux policy and relabeled before you continue with the installation (later in the document you will find information on how to modify your SELinux policy)
Network setup
HAL, once you get the dialog box in the Windows install select the 'Generic i486 Platform' tab. Scroll through selections with the Up and Down arrows.
# virsh start WindowsGuestWindowsGuest is the name of your virtual machine.
virsh reboot WindowsGuestName. When you restart the virtual machine, the Setup is being restarted message displays:
virt-install command. virt-install can be used instead of virt-manager This process is similar to the Windows XP installation covered in Section 8.3, “Installing Windows XP as a fully virtualized guest”.
virt-install for installing Windows Server 2003 as the console for the Windows guest opens the virt-viewer window promptly. The examples below installs a Windows Server 2003 guest with the virt-install command.
Xen virt-install
# virt-install --virt-type=xen -hvm \ --name windows2003sp1 --file=/var/lib/libvirt/images/windows2003sp2.img \ --file-size=6 \ --cdrom=/var/lib/libvirt/images/ISOs/WIN/en_windows_server_2003_sp1.iso \ --vnc --ram=1024
KVM virt-install
# virt-install --accelerate --hvm --connect qemu:///system \ --name rhel3support \ --network network:default \ --file=/var/lib/libvirt/images/windows2003sp2.img \ --file-size=6 \ --cdrom=/var/lib/libvirt/images/ISOs/WIN/en_windows_server_2003_sp1.iso \ --vnc --ram=1024
Standard PC as the Computer Type. Changing the Computer Type is required for Windows Server 2003 virtualized guests.
Open virt-manager
virt-manager. Launch the application from the menu and submenu. Alternatively, run the virt-manager command as root.
Select the hypervisor
qemu.
Start the new virtual machine wizard
Name the virtual machine
Choose a virtualization method
Select the installation method
Locate installation media
/var/lib/libvirt/images/ directory. Any other location may require additional configuration for SELinux, refer to Section 18.2, “SELinux and virtualization” for details.
Storage setup
/var/lib/libvirt/images/ directory. Assign sufficient space for your virtualized guest and any applications the guest requires.
Network setup
Memory and CPU allocation
Verify and start guest installation
Installing Windows
Table of Contents
/dev/xvd[a to z][1 to 15]
/dev/xvdb13
/dev/xvd[a to i][a to z][1 to 15]
/dev/xvdbz13
/dev/sd[a to p][1 to 15]
/dev/sda1
/dev/hd[a to t][1 to 63]
/dev/hdd3
dd command. Replace /dev/fd0 with the name of a floppy device and name the disk appropriately.
# dd if=/dev/fd0 of=~/legacydrivers.img
virt-manager running a fully virtualized Red Hat Enterprise Linux installation with an image located in /var/lib/libvirt/images/rhel5FV.img. The Xen hypervisor is used in the example.
virsh command on a running guest.
# virsh dumpxml rhel5FV > rhel5FV.xml
# dd if=/dev/zero of=/var/lib/libvirt/images/rhel5FV-floppy.img bs=512 count=2880
<disk type='file' device='floppy'> <source file='/var/lib/libvirt/images/rhel5FV-floppy.img'/> <target dev='fda'/> </disk>
virsh shutdown command instead.
# virsh destroy rhel5FV# virsh create rhel5FV.xml
dd command. Sparse files are not recommended due to data integrity and performance issues. Sparse files are created much faster and can used for testing but should not be used in production environments.
# dd if=/dev/zero of=/var/lib/libvirt/images/FileName.img bs=1M seek=4096 count=0
# dd if=/dev/zero of=/var/lib/libvirt/images/FileName.img bs=1M count=4096
Guest1 and the file is saved in the users home directory.
# virsh dumpxmlGuest1> ~/Guest1.xml
Guest1.xml in this example) in a text editor. Find the <disk> elements, these elements describe storage devices. The following is an example disk element:
<disk type='file' device='disk'>
<driver name='tap' type='aio'/>
<source file='/var/lib/libvirt/images/Guest1.img'/>
<target dev='xvda'/>
</disk>
<disk> element. Ensure you specify a device name for the virtual block device attributes. These attributes must be unique for each guest configuration file. The following example is a configuration file section which contains an additional file-based storage container named FileName.img.
<disk type='file' device='disk'>
<driver name='tap' type='aio'/>
<source file='/var/lib/libvirt/images/Guest1.img'/>
<target dev='xvda'/>
</disk>
<disk type='file' device='disk'>
<driver name='tap' type='aio'/>
<source file='/var/lib/libvirt/images/FileName.img'/>
<target dev='hda'/>
</disk>
# virsh create Guest1.xml
FileName.img as the device called /dev/sdb. This device requires formatting from the guest. On the guest, partition the device into one primary partition for the entire device then format the device.
n for a new partition.
# fdisk /dev/sdb Command (m for help):
p for a primary partition.
Command action e extended p primary partition (1-4)
1.
Partition number (1-4): 1
Enter.
First cylinder (1-400, default 1):
Enter.
Last cylinder or +size or +sizeM or +sizeK (2-400, default 400):
t.
Command (m for help): t
1.
Partition number (1-4): 1
83 for a linux partition.
Hex code (type L to list codes): 83
Command (m for help):wCommand (m for help):q
ext3 file system.
# mke2fs -j /dev/sdb1
# mount /dev/sdb1 /myfiles
fstab file, the initrd file or used by the kernel command line. If less privileged users, especially virtualized guests, have write access to whole partitions or LVM volumes the host system could be compromised.
/dev/sdb). Virtualized guests with access to block devices may be able to access other block devices on the system or modify volume labels which can be used to compromise the host system. Use partitions (for example, /dev/sdb1) or LVM volumes to prevent this issue.
multipath and persistence on the host if required.
virsh attach command. Replace: myguest with your guest's name, /dev/sdb1 with the device to add, and sdc with the location for the device on the guest. The sdc must be an unused device name. Use the sd* notation for Windows guests as well, the guest will recognize the device correctly.
--type cdrom parameter to the command for CD-ROM or DVD devices.
--type floppy parameter to the command for floppy devices.
# virsh attach-diskmyguest/dev/sdb1sdc--driver tap --mode readonly
/dev/sdb on Linux or D: drive, or similar, on Windows. This device may require formatting.
multipath must use Single path configuration. Systems running multipath can use Multiple path configuration.
udev. Only use this procedure for hosts which are not using multipath.
/etc/scsi_id.config file.
options=-b is line commented out.
# options=-b
options=-g
udev to assume all attached SCSI devices return a UUID.
scsi_id -g -s /block/sd* command. For example:
# scsi_id -g -s /block/sd* 3600a0b800013275100000015427b625e
/dev/sdc.
scsi_id -g -s /block/sd* command is identical from computer which accesses the device.
20-names.rules in the /etc/udev/rules.d directory. Add new rules to this file. All rules are added to the same file using the same format. Rules follow this format:
KERNEL=="sd[a-z]", BUS=="scsi", PROGRAM="/sbin/scsi_id -g -s /block/%k", RESULT="UUID", NAME="devicename"
UUID and devicename with the UUID retrieved above, and a name for the device. This is a rule for the example above:
KERNEL="sd*", BUS="scsi", PROGRAM="/sbin/scsi_id -g -s", RESULT="3600a0b800013275100000015427b625e", NAME="rack4row16"
udev daemon now searches all devices named /dev/sd* for the UUID in the rule. Once a matching device is connected to the system the device is assigned the name from the rule. In the a device with a UUID of 3600a0b800013275100000015427b625e would appear as /dev/rack4row16.
/etc/rc.local:
/sbin/start_udev
/etc/scsi_id.config, /etc/udev/rules.d/20-names.rules, and /etc/rc.local files to all relevant hosts.
/sbin/start_udev
multipath package is used for systems with more than one physical path from the computer to storage devices. multipath provides fault tolerance, fail-over and enhanced performance for network storage devices attached to Red Hat Enterprise Linux systems.
multipath environment requires defined alias names for your multipath devices. Each storage device has a UUID which acts as a key for the aliased names. Identify a device's UUID using the scsi_id command.
# scsi_id -g -s /block/sdc
/dev/mpath directory. In the example below 4 devices are defined in /etc/multipath.conf:
multipaths {
multipath {
wwid 3600805f30015987000000000768a0019
alias oramp1
}
multipath {
wwid 3600805f30015987000000000d643001a
alias oramp2
}
mulitpath {
wwid 3600805f3001598700000000086fc001b
alias oramp3
}
mulitpath {
wwid 3600805f300159870000000000984001c
alias oramp4
}
}
/dev/mpath/oramp1, /dev/mpath/oramp2, /dev/mpath/oramp3 and /dev/mpath/oramp4. Once entered, the mapping of the devices' WWID to their new names are now persistent after rebooting.
virsh with the attach-disk parameter.
# virsh attach-disk [domain-id] [source] [target] --driver file --type cdrom --mode readonly
The source and target parameters are paths for the files and devices, on the host and guest respectively. The source parameter can be a path to an ISO file or the device from the /dev directory.
libvirt installation provides NAT based connectivity to virtual machines out of the box. This is the so called 'default virtual network'. Verify that it is available with the virsh net-list --all command.
# virsh net-list --all Name State Autostart ----------------------------------------- default active yes
# virsh net-define /usr/share/libvirt/networks/default.xml
/usr/share/libvirt/networks/default.xml
# virsh net-autostart default Network default marked as autostarted
# virsh net-start default Network default started
libvirt default network is running, you will see an isolated bridge device. This device does not have any physical interfaces added, since it uses NAT and IP forwarding to connect to outside world. Do not add new interfaces.
# brctl show bridge name bridge id STP enabled interfaces virbr0 8000.000000000000 yes
libvirt adds iptables rules which allow traffic to and from guests attached to the virbr0 device in the INPUT, FORWARD, OUTPUT and POSTROUTING chains. libvirt then attempts to enable the ip_forward parameter. Some other applications may disable ip_forward, so the best option is to add the following to /etc/sysctl.conf.
net.ipv4.ip_forward = 1
<interface type='network'> <source network='default'/> </interface>
<interface type='network'> <source network='default'/> <mac address='00:16:3e:1a:b3:4a'/> </interface>
/etc/xen/xend-config.sxp and changing the line:
(network-script network-bridge)
(network-script /bin/true)
/etc/sysconfig/network-scripts/ directory):
# chkconfig NetworkManager off # chkconfig network on # service NetworkManager stop # service network start
NM_CONTROLLED=no" to the ifcfg-* scripts used in the examples. If you do not either set this parameter or disable NetworkManager entirely, any bridge configuration will be overwritten and lost when NetworkManager next starts.
/etc/sysconfig/network-scripts directory:
# cd /etc/sysconfig/network-scripts
ifcfg-eth0 defines the physical network interface which is set as part of a bridge:
DEVICE=eth0 # change the hardware address to match the hardware address your NIC uses HWADDR=00:16:76:D6:C9:45 ONBOOT=yes BRIDGE=br0
MTU variable to the end of the configuration file.
MTU=9000
/etc/sysconfig/network-scripts directory called ifcfg-br0 or similar. The br0 is the name of the bridge, this can be anything as long as the name of the file is the same as the DEVICE parameter.
DEVICE=br0 TYPE=Bridge BOOTPROTO=dhcp ONBOOT=yes DELAY=0
ifcfg-br0 file). Network access will not function as expected if IP address details are configured on the physical interface that the bridge is connected to.
TYPE=Bridge, is case-sensitive. It must have uppercase 'B' and lower case 'ridge'.
# service network restart
iptables to allow all traffic to be forwarded across the bridge.
# iptables -I FORWARD -m physdev --physdev-is-bridged -j ACCEPT # service iptables save # service iptables restart
iptables rules. In /etc/sysctl.conf append the following lines:
net.bridge.bridge-nf-call-ip6tables = 0 net.bridge.bridge-nf-call-iptables = 0 net.bridge.bridge-nf-call-arptables = 0
sysctl.
# sysctl -p /etc/sysctl.conf
libvirt daemon.
# service libvirtd reload
# brctl show bridge name bridge id STP enabled interfaces virbr0 8000.000000000000 yes br0 8000.000e0cb30550 no eth0
virbr0 bridge. Do not attempt to attach a physical device to virbr0. The virbr0 bridge is only for Network Address Translation (NAT) connectivity.
virsh (Chapter 25, Managing guests with virsh) and virt-manager (Chapter 26, Managing guests with the Virtual Machine Manager (virt-manager)). Those chapters provide a detailed description of the networking configuration tasks using both tools.
system-config-network application. Alternatively, create a new configuration file named ifcfg-ethX in the /etc/sysconfig/network-scripts/ directory where X is any number not already in use. Below is an example configuration file for a second network interface called eth1
$ cat /etc/sysconfig/network-scripts/ifcfg-eth1 DEVICE=eth1 BOOTPROTO=static ONBOOT=yes USERCTL=no IPV6INIT=no PEERDNS=yes TYPE=Ethernet NETMASK=255.255.255.0 IPADDR=10.1.1.1 GATEWAY=10.1.1.254 ARP=yes
/etc/xen/scripts/network-bridge, to /etc/xen/scripts/network-bridge.xen.
/etc/xen/xend-config.sxp and add the line (network-xen-multi-bridge). A typical xend-config.sxp file should have the following line. Comment this line out. Use the # symbol to comment out lines.
network-script network-bridge
network-xen-multi-bridge parameter to enable multiple network bridges.
#network-script network-bridge network-script network-xen-multi-bridge
network-xen-multi-bridge.sh in the /etc/xen/scripts/ directory. A sample scripts is below, this example script will create two Xen network bridges (xenbr0 and xenbr1) one will be attached to eth1 and the other one to eth0. If you want to create additional bridges just follow the example in the script and copy nad paste the lines as required:
#!/bin/sh
# network-xen-multi-bridge
# Exit if anything goes wrong.
set -e
# First arg is the operation.
OP=$1
shift
script=/etc/xen/scripts/network-bridge.xen
case ${OP} in
start)
$script start vifnum=1 bridge=xenbr1 netdev=eth1
$script start vifnum=0 bridge=xenbr0 netdev=eth0
;;
stop)
$script stop vifnum=1 bridge=xenbr1 netdev=eth1
$script stop vifnum=0 bridge=xenbr0 netdev=eth0
;;
status)
$script status vifnum=1 bridge=xenbr1 netdev=eth1
$script status vifnum=0 bridge=xenbr0 netdev=eth0
;;
*)
echo 'Unknown command: ' ${OP}
echo 'Valid commands are: start, stop, status'
exit 1
esac
# chmod +x /etc/xen/scripts/network-xen-multi-bridge.sh
# service network restart
virt-manager. NetworkManager works with virtual network devices by default in Red Hat Enterprise Linux 5.1 and newer.
<interface type='network'> <mac address='AA:AA:AA:AA:AA:AA'/> <source network='default'/> <target dev='vnet0'/> <model type='virtio'/> </interface>
xm configuration files, virtual network devices are labeled "vif".
ifup or ifdown calls to the network interface it is using. In addition wireless network cards do not work well in a virtualization environment due to Xen's (default) bridged network usage.
dummy0. This will also allow you to use a hidden IP address space for your guests.
/etc/modprobe.conf
alias dummy0 dummy options dummy numdummies=1
/etc/sysconfig/network-scripts/ifcfg-dummy0:
DEVICE=dummy0 BOOTPROTO=none ONBOOT=yes USERCTL=no IPV6INIT=no PEERDNS=yes TYPE=Ethernet NETMASK=255.255.255.0 IPADDR=10.1.1.1 ARP=yes
xenbr0 to dummy0, so you can use networking even when not connected to a physical network. Edit /etc/xen/xend-config.sxp to include the netdev=dummy0 entry:
(network-script 'network-bridge bridge=xenbr0 netdev=dummy0')
/etc/sysconfig/network in the guest and modify the default gateway to point to dummy0. If you are using a static IP, set the guest's IP address to exist on the same subnet as dummy0.
NETWORKING=yes HOSTNAME=localhost.localdomain GATEWAY=10.1.1.1 IPADDR=10.1.1.10 NETMASK=255.255.255.0
/etc/init.d/xenLaptopNAT and create a soft link to /etc/rc3.d/S99xenLaptopNAT. this automatically starts NAT at boot time.
#!/bin/bash
PATH=/usr/bin:/sbin:/bin:/usr/sbin
export PATH
GATEWAYDEV=`ip route | grep default | awk {'print $5'}`
iptables -F
case "$1" in
start)
if test -z "$GATEWAYDEV"; then
echo "No gateway device found"
else
echo "Masquerading using $GATEWAYDEV"
/sbin/iptables -t nat -A POSTROUTING -o $GATEWAYDEV -j MASQUERADE
fi
echo "Enabling IP forwarding"
echo 1 > /proc/sys/net/ipv4/ip_forward
echo "IP forwarding set to `cat /proc/sys/net/ipv4/ip_forward`"
echo "done."
;;
*)
echo "Usage: $0 {start|restart|status}"
;;
esac
dnsmasq on a laptop running virtualization:
dnsmasq.tgz. The tar archive includes the following files:
nm-dnsmasq can be used as a dispatcher script for NetworkManager. It will be run every time NetworkManager detects a change in connectivity and force a restart/reload of dnsmasq. It should be copied to /etc/NetworkManager/dispatcher.d/nm-dnsmasq
xenDNSmasq can be used as the main start up or shut down script for /etc/init.d/xenDNSmasq
dnsmasq.conf is a sample configuration file for /etc/dnsmasq.conf
dnsmasq is the binary image for /usr/local/sbin/dnsmasq
/usr/local/sbin/dnsmasq) you need to edit your dnsmasq configuration file. The file is located in /etc/dnsmaqs.conf
interface parameter allows dnsmasq to listen for DHCP and DNS requests only on specified interfaces. This could be dummy interfaces but not your public interfaces as well as the local loopback interface. Add another interface line for more than one interface. interface=dummy0 is an example which listens on the dummy0 interface.
dhcp-range to enable the integrated DHCP server, you need to supply the range of addresses available for lease and optionally a lease time. If you have more than one network, you will need to repeat this for each network on which you want to supply DHCP service. An example would be (for network 10.1.1.* and a lease time of 12hrs): dhcp-range=10.1.1.10,10.1.1.50,255.255.255.0,12h
dhcp-option to override the default route supplied by dnsmasq, which assumes the router is the same machine as the one running dnsmasq. An example would be dhcp-option=3,10.1.1.1
xenDNSmasq to /etc/init.d
chkconfig --add xenDNSmasq
chkconfig --levels 345 xenDNSmasq on
dnsmasq to restart every time NetworkManager detects a change in connectivity you can use the supplied script nm-dnsmasq.
nm-dnsmasq script to /etc/NetworkManager/dispatcher.d/
dnsmasq will also detect changes in your /etc/resolv.conf and automatically reload them (that is, if you start up a VPN session for example).
nm-dnsmasq and xenDNSmasq script will also set up NAT if you have your virtualized guests on a hidden network to allow them access to the public network.
pv-on-hvm module, in the default kernel package. That means the para-virtualized drivers are available for Red Hat Enterprise Linux 4.7 and newer or 5.3 and newer guests.
kmod-xenpv.
kmod-xenpv,
modules-init-tools (for versions prior to Red Hat Enterprise Linux 4.6z you require modules-init-tools-3.1-0.pre5.3.4.el4_6.1 or greater), and
modversions.
kmod-xenpv.
/lib file system.
# rpm -q --queryformat '%{NAME}-%{VERSION}-%{RELEASE}.%{ARCH}\n' kernel
| Kernel Architecture | Red Hat Enterprise Linux 3 | Red Hat Enterprise Linux 4 | Red Hat Enterprise Linux 5 |
|---|---|---|---|
| athlon | Supported (AMD) | ||
| athlon-SMP | Supported (AMD) | ||
| i32e | Supported (Intel) | ||
| i686 | Supported (Intel) | Supported | Supported |
| i686-PAE | Supported | ||
| i686-SMP | Supported (Intel) | Supported | |
| i686-HUGEMEM | Supported (Intel) | Supported | |
| x86_64 | Supported (AMD) | Supported | Supported |
| x86_64-SMP | Supported (AMD) | Supported | |
| x86_64-LARGESMP | Supported | ||
| Itanium (IA64) | Supported |
# rpm -q --queryformat '%{NAME}-%{VERSION}-%{RELEASE}.%{ARCH}\n' kernel
kernel-PAE-2.6.18-53.1.4.el5.i686
GRUBGRUB), or a disk that contains the kernel initrd images. That is, any disk which contains the /boot directory or partition can not use the para-virtualized block device drivers.
xen-pci-platform module will fail.
| Guest kernel type | Required host kernel type |
|---|---|
| ia32e (UP and SMP) | x86_64 |
| i686 | i686 |
| i686-SMP | i686 |
| i686-HUGEMEM | i686 |
| Guest kernel type | Required host kernel type |
|---|---|
| athlon | i686 |
| athlon-SMP | i686 |
| x86_64 | x86_64 |
| x86_64-SMP | x86_64 |
MBR and the boot loader (GRUB), and for the /boot partition. This partition can be very small, as it only needs to have enough capacity to hold the /boot partition.
/, /usr) or logical volumes.
/boot partition will use the virtualized block device drivers.
rpm command or the yum command to install the packages. The rpm utility will install the following four new kernel modules into /lib/modules/[%kversion][%kvariant]/extra/xenpv/%release:
xen_platform_pci.ko,
xen_balloon.ko,
xen_vbd.ko,
xen_vnif.ko.
xen-vbd) disk driver.
<disk type='file' device='disk'> <driver name='file'/> <source file='/dev/hda6'/> <target dev='hda'/> </disk>
<disk type='file' device='disk'> <driver name='tap' type='aio'/> <source file='/dev/hda6'/> <target dev='xvda'/> </disk>
# xm start YourGuestNameYourGuestName is the name of the configuration file or the guest operating system's name as defined in its configuration file in the name = "os_name" parameter.
kernel-2.4.21-60.EL for all the required headers.
kmod-xenpv rpm for your hardware architecture and kernel variant to your guest operating system.
rpm utility to install the RPM packages. Ensure you have correctly identified which package you need for your guest operating system variant and architecture.
[root@rhel3]# rpm -ivh kmod-xenpv*
[root@rhel3]# mkdir -p /lib/modules/'uname -r'/extra/xenpv [root@rhel3]# cp -R /lib/modules/2.4.21-52.EL[%kvariant]/extra/xenpv/%release \ /lib/modules/'uname -r'/extra/xenpv [root@rhel3]# depmod -ae [root@rhel3]# modprobe xen-vbd [root@rhel3]# modprobe xen-vnif
insmod when installing the binary driver modules due to Red Hat Enterprise Linux 3 having MODVERSIONS enabled. These warnings can be ignored.
/etc/modules.conf and make sure you have an alias for eth0 like the one below. If you are planning to configure multiple interfaces add an additional line for each interface.
alias eth0 xen-vnif
/etc/rc.local and add the line:
insmod /lib/modules/'uname -r'/extra/xenpv/%release/xen-vbd.o
#shutdown -h now” inside the guest).
/etc/xen/YourGuestName with a text editor, performing the following changes:
type=ioemu” entry from the “vif=” entry.
xen-vbd) disk driver.
<disk type='file' device='disk'> <driver name='file'/> <source file='/dev/hda6'/> <target dev='hda'/> </disk>
<disk type='file' device='disk'> <driver name='tap' type='aio'/> <source file='/dev/hda6'/> <target dev='xvda'/> </disk>
# xm start YourGuestNameYourGuestName is the name of the configuration file or the guest operating system's name as defined in its configuration file in the name = "os_name" parameter.
weak-modules and modversions support is not provided in Red Hat Enterprise Linux 3. To insert the module execute the command below.
insmod xen_vbd.ko
xen-vbd. The steps below will cover how to create and register para-virtualized block devices.
#!/bin/sh
module="xvd"
mode="664"
major=`awk "\\$2==\"$module\" {print \\$1}" /proc/devices`
# < mknod for as many or few partitions on xvd disk attached to FV guest >
# change/add xvda to xvdb, xvbd, etc. for 2nd, 3rd, etc., disk added in
# in xen config file, respectively.
mknod /dev/xvdb b $major 16
mknod /dev/xvdb1 b $major 17
mknod /dev/xvdb2 b $major 18
chgrp disk /dev/xvd*
chmod 0660 /dev/xvd*
# mknod /dev/xvdc b $major 16 # mknod /dev/xvdc1 b $major 17
# mknod /dev/xvdd b $major 32 # mknod /dev/xvdd1 b $major 33
[root@rhel3]# cat /proc/partitions major minor #blocks name 3 0 10485760 hda 3 1 104391 hda1 3 2 10377990 hda2 202 16 64000 xvdb 202 17 32000 xvdb1 202 18 32000 xvdb2 253 0 8257536 dm-0 253 1 2031616 dm-1
xvdb” is available to the system.
[root@rhel3]# mkdir /mnt/pvdisk_p1 [root@rhel3]# mkdir /mnt/pvdisk_p2
[root@rhel3]# mount /dev/xvdb1 /mnt/pvdisk_p1 [root@rhel3]# mount /dev/xvdb2 /mnt/pvdisk_p2
[root@rhel3]# df /mnt/pvdisk_p1 Filesystem 1K-blocks Used Available Use% Mounted on /dev/xvdb1 32000 15 31985 1% /mnt/pvdisk_p1
/etc/fstab file inside the guest to mount the devices during the boot sequence. Add the following lines:
/dev/xvdb1 /mnt/pvdisk_p1 ext3 defaults 1 2 /dev/xvdb2 /mnt/pvdisk_p2 ext3 defaults 1 2
dom0), the "noapic" parameter should be added to the kernel boot line in your virtual guest's /boot/grub/grub.conf entry as seen below. Keep in mind your architecture and kernel version may be different.
kernel /vmlinuz-2.6.9-67.EL ro root=/dev/VolGroup00/rhel4_x86_64 rhgb noapic
kmod-xenpv, modules-init-tools and modversions RPMs for your hardware architecture and kernel variant to your guest operating system.
rpm utility to install the RPM packages. Make sure you have correctly identified which package you need for your guest operating system variant and architecture. An updated module-init-tools is required for this package, it is available with the Red Hat Enterprise Linux 4-6-z kernel or newer.
[root@rhel4]# rpm -ivh modversions [root@rhel4]# rpm -Uvh module-init-tools [root@rhel4]# rpm -ivh kmod-xenpv*
cat /etc/modprobe.conf to verify you have an alias for eth0 like the one below. If you are planning to configure multiple interfaces add an additional line for each interface. If it does not look like the entry below change it.
alias eth0 xen-vnif
#shutdown -h now” inside the guest).
/etc/xen/YourGuestsName in the following ways:
type=ioemu” entry from the “vif=” entry.
xen-vbd) disk driver.
disk=” section in the guest configuration file. The original “disk=” entry might also look like the entry below.
disk = [ "file:/var/lib/libvirt/images/rhel4_64_fv.dsk,hda,w"]
disk = [ "file:/var/lib/libvirt/images/rhel3_64_fv.dsk,hda,w", "tap:aio:/var/lib/libvirt/images/UserStorage.dsk,xvda,w" ]
tap:aio” for the para-virtualized device if a file-based image is used.
virsh command:
# virsh start YourGuestNamekudzu will ask you to "Keep or Delete the Realtek Network device" and "Configure the xen-bridge device". You should configure the xen-bridge and delete the Realtek network device.
dom0), the "noapic" parameter should be added to the kernel boot line in your virtual guest's /boot/grub/grub.conf entry as seen below. Keep in mind your architecture and kernel version may be different.
kernel /vmlinuz-2.6.9-67.EL ro root=/dev/VolGroup00/rhel4_x86_64 rhgb noapic
[root@rhel4]# cat /proc/partitions major minor #blocks name 3 0 10485760 hda 3 1 104391 hda1 3 2 10377990 hda2 202 0 64000 xvdb 202 1 32000 xvdb1 202 2 32000 xvdb2 253 0 8257536 dm-0 253 1 2031616 dm-1
xvdb” is available to the system.
[root@rhel4]# mkdir /mnt/pvdisk_p1 [root@rhel4]# mkdir /mnt/pvdisk_p2
[root@rhel4]# mount /dev/xvdb1 /mnt/pvdisk_p1 [root@rhel4]# mount /dev/xvdb2 /mnt/pvdisk_p2
[root@rhel4]# df /mnt/pvdisk_p1 Filesystem 1K-blocks Used Available Use% Mounted on /dev/xvdb1 32000 15 31985 1% /mnt/pvdisk_p1
/etc/fstab file inside the guest to mount the devices during the boot sequence. Add the following lines:
/dev/xvdb1 /mnt/pvdisk_p1 ext3 defaults 1 2 /dev/xvdb2 /mnt/pvdisk_p2 ext3 defaults 1 2
xen-vbd driver may not automatically load. Execute the following command on the guest, substituting %release with the correct release version for the para-virtualized drivers.
# insmod /lib/modules/'uname -r'/weak-updates/xenpv/%release/xen_vbd.ko
#shutdown -h now” inside the guest).
/etc/xen/<Your GuestsName> in the following ways:
type=ioemu” entry from the “vif=” entry.
xen-vbd) disk driver.
disk=” section in the guest configuration file. The original “disk=” entry might also look like the entry below.
disk = [ "file:/var/lib/libvirt/images/rhel4_64_fv.dsk,hda,w"]
disk = [ "file:/var/lib/libvirt/images/rhel3_64_fv.dsk,hda,w", "tap:aio:/var/lib/libvirt/images/UserStorage.dsk,xvda,w" ]
tap:aio” for the para-virtualized device if a file-based image is used.
virsh command:
# virsh start YourGuestName
[root@rhel5]# ifconfig eth0
[root@rhel5]# cat /proc/partitions major minor #blocks name 3 0 10485760 hda 3 1 104391 hda1 3 2 10377990 hda2 202 0 64000 xvdb 202 1 32000 xvdb1 202 2 32000 xvdb2 253 0 8257536 dm-0 253 1 2031616 dm-1
xvdb” is available to the system.
[root@rhel5]# mkdir /mnt/pvdisk_p1 [root@rhel5]# mkdir /mnt/pvdisk_p2
[root@rhel5]# mount /dev/xvdb1 /mnt/pvdisk_p1 [root@rhel5]# mount /dev/xvdb2 /mnt/pvdisk_p2
[root@rhel5]# df /mnt/pvdisk_p1 Filesystem 1K-blocks Used Available Use% Mounted on /dev/xvdb1 32000 15 31985 1% /mnt/pvdisk_p1
/etc/fstab file inside the guest to mount the devices during the boot sequence. Add the following lines:
/dev/xvdb1 /mnt/pvdisk_p1 ext3 defaults 1 2 /dev/xvdb2 /mnt/pvdisk_p2 ext3 defaults 1 2
dom0), the "noapic" parameter should be added to the kernel boot line in your virtual guest's /boot/grub/grub.conf entry as seen below. Keep in mind your architecture and kernel version may be different.
kernel /vmlinuz-2.6.9-67.EL ro root=/dev/VolGroup00/rhel4_x86_64 rhgb noapic
/etc/modprobe.d/blacklist file. Blacklist 8139cp and 8139too for the RealTek 8139 and e1000 for the virtualized Intel e1000 NIC.
8139cp8139tooe1000
/etc/sysconfig/network-scripts directory.
xen_emul_unplug=never
virt-manager open the console window for the guest and log in as root.
/etc/modprobe.conf contains the line “alias eth0 xen-vnif”.
# cat /etc/modprobe.conf alias eth0 xen-vnif
eth0 execute “# ifconfig eth0”. If you receive an error about the device not existing you should load the modules manually as outlined in Section 36.4, “Manually loading the para-virtualized drivers”.
ifconfig eth0
eth0 Link encap:Ethernet HWaddr 00:00:00:6A:27:3A
BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:630150 errors:0 dropped:0 overruns:0 frame:0
TX packets:9 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:109336431 (104.2 MiB) TX bytes:846 (846.0 b)
# system-config-network”. Click on the “” button to start the network card configuration.
ifconfig eth0
eth0 Link encap:Ethernet HWaddr 00:16:3E:49:E4:E0
inet addr:192.168.78.180 Bcast:192.168.79.255 Mask:255.255.252.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:630150 errors:0 dropped:0 overruns:0 frame:0
TX packets:501209 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:109336431 (104.2 MiB) TX bytes:46265452 (44.1 MiB)
/etc/xen/YourGuestName replacing YourGuestName with the name of your guest.
vif = [ "mac=00:16:3e:2e:c5:a9,bridge=xenbr0" ]
vif=” section of the configuration file similar to the one seen below.
vif = [ "mac=00:16:3e:2e:c5:a9,bridge=xenbr0",
"mac=00:16:3e:2f:d5:a9,bridge=xenbr0" ]
# echo 'import virtinst.util ; print virtinst.util.randomMAC()' | python
/etc/modules.conf in Red Hat Enterprise Linux 3 or /etc/modprobe.conf in Red Hat Enterprise Linux 4 and Red Hat Enterprise Linux 5. Add a new alias for each new interface you added.
alias eth1 xen-vnif
# ifconfig eth1
redhat-config-network on Red Hat Enterprise Linux 3 or system-config-network on Red Hat Enterprise Linux 4 and Red Hat Enterprise Linux 5.
/etc/xen/YourGuestName replacing YourGuestName with the name of your guest. The original entry may look like the one below.
disk = [ "file:/var/lib/libvirt/images/rhel5_64_fv.dsk,hda,w"]
disk=” parameter in the configuration file. Storage entities which use the para-virtualized driver resemble the entry below. The “tap:aio” parameter instructs the hypervisor to use the para-virtualized driver.
disk = [ "file:/var/lib/libvirt/images/rhel5_64_fv.dsk,hda,w",
"tap:aio:/var/lib/libvirt/images/UserStorage1.dsk,xvda,w" ]
disk=” section as a comma separated list.
xvd' device, that is for your second storage entity it would be 'xvdb' instead of 'xvda'.
disk = [ "file:/var/lib/libvirt/images/rhel5_64_fv.dsk,hda,w",
"tap:aio:/var/lib/libvirt/images/UserStorage1.dsk,xvda,w",
"tap:aio:/var/lib/libvirt/images/UserStorage2.dsk,xvdb,w" ]
# cat /proc/partitions major minor #blocks name 3 0 10485760 hda 3 1 104391 hda1 3 2 10377990 hda2 202 0 64000 xvda 202 1 64000 xvdb 253 0 8257536 dm-0 253 1 2031616 dm-1
xvdb” is available to the system.
/etc/fstab inside the guest to mount the devices and partitions at boot time.
# mkdir /mnt/pvdisk_xvda # mkdir /mnt/pvdisk_xvdb # mount /dev/xvda /mnt/pvdisk_xvda # mount /dev/xvdb /mnt/pvdisk_xvdb # df /mnt Filesystem 1K-blocks Used Available Use% Mounted on /dev/xvda 64000 15 63985 1% /mnt/pvdisk_xvda /dev/xvdb 64000 15 63985 1% /mnt/pvdisk_xvdb
Download the drivers
yum command.
# yum install virtio-win
virtio-win.iso, in the /usr/share/virtio-win/ directory.
Install the para-virtualized drivers
virt-manager to mount a CD-ROM image for a Windows guest” to add a CD-ROM image with virt-manager and then install the drivers.
virt-manager to mount a CD-ROM image for a Windows guestOpen virt-manager and the virtualized guest
virt-manager, select your virtualized guest from the list by double clicking the guest name.
Open the hardware tab
Select the device type
Select the ISO file
/usr/share/virtio-win/virtio-win.iso.
Disc assigned
Reboot
Open My Computer
Select the correct installation files
RHEV-Block.msi for 32-bit guests or RHEV-Block64.msi for 64-bit guests),
RHEV-Network.msi for 32-bit guests or RHEV-Block64.msi for 64-bit guests),
Install the block device driver
Start the block device driver installation
RHEV-Block.msi or RHEV-Block64.msi.
Confirm the exception
Finish
Install the network device driver
Start the network device driver installation
RHEV-Network.msi or RHEV-Network64.msi.
Performance setting
| Value | Action |
|---|---|
| 0 | Disable TCP timestamps and window scaling. |
| 1 | Enable TCP window scaling. |
| 2 | Enable TCP timestamps. |
| 3 | Enable TCP timestamps and window scaling. |
Confirm the exception
Finish
Reboot
viostor.vfd as a floppy
Windows Server 2003
Windows Server 2008
virtio driver instead of virtualized IDE driver. This example edits libvirt configuration files. Alternatively, virt-manager, virsh attach-disk or virsh attach-interface can add a new device using the para-virtualized drivers Section 13.4, “Using KVM para-virtualized drivers for new devices”.
<disk type='file' device='disk'> <source file='/var/lib/libvirt/images/disk1.img'/> <target dev='vda' bus='ide'/> </disk>
virtio.
<disk type='file' device='disk'>
<source file='/var/lib/libvirt/images/disk1.img'/>
<target dev='vda' bus='virtio'/>
</disk>
virt-manager.
virsh attach-disk or virsh attach-interface commands can be used to attach devices using the para-virtualized drivers.
virt-manager.
Enable the Intel VT-d extensions
Activate Intel VT-d in the kernel
intel_iommu=on parameter to the kernel line of the kernel line in the /boot/grub/grub.conf file.
grub.conf file with Intel VT-d activated.
default=0 timeout=5 splashimage=(hd0,0)/grub/splash.xpm.gz hiddenmenu title Red Hat Enterprise Linux Server (2.6.18-190.el5) root (hd0,0) kernel /vmlinuz-2.6.18-190.el5 ro root=/dev/VolGroup00/LogVol00 intel_iommu=on initrd /initrd-2.6.18-190.el5.img
Ready to use
Enable AMD IOMMU extensions
iommu=on parameter to the hypervisor command line. Modify the /boot/grub/grub.conf file as follows to enable PCI passthrough:
default=0 timeout=5 splashimage=(hd0,0)/grub/splash.xpm.gz hiddenmenu title Red Hat Enterprise Linux Server (2.6.18-192.el5) root (hd0,0) kernel /xen.gz-2.6.18-192.el5 iommu=on module /vmlinuz-2.6.18-192.el5xen ro root=/dev/VolGroup00/LogVol00 module /initrd-2.6.18-190.el5xen.img
pci_8086_3a6c, and a fully virtualized guest named win2k3.
Identify the device
virsh nodedev-list command lists all devices attached to the system. The --tree option is useful for identifying devices attached to the PCI device (for example, disk controllers and USB controllers).
# virsh nodedev-list --tree
# virsh nodedev-list | grep pci
**** is a four digit hexadecimal code):
pci_8086_****
lspci output to lspci -n (which turns off name resolution) output can assist in deriving which device has which device identifier code.
virsh nodedev-dumpxml command:
# virsh nodedev-dumpxml pci_8086_3a6c
<device>
<name>pci_8086_3a6c</name>
<parent>computer</parent>
<capability type='pci'>
<domain>0</domain>
<bus>0</bus>
<slot>26</slot>
<function>7</function>
<id='0x3a6c'>82801JD/DO (ICH10 Family) USB2 EHCI Controller #2</product>
<vendor id='0x8086'>Intel Corporation</vendor>
</capability>
</device># virsh nodedev-dettach pci_8086_3a6c Device pci_8086_3a6c dettached
$ printf %x 0 0 $ printf %x 26 1a $ printf %x 7 7
bus='0x00' slot='0x1a' function='0x7'
virsh edit (or virsh attach device) and add a device entry in the <devices> section to attach the PCI device to the guest. Only run this command on offline guests. Red Hat Enterprise Linux does not support hotplugging PCI devices at this time.
# virsh edit win2k3
<hostdev mode='subsystem' type='pci' managed='yes'>
<source>
<address domain='0x0000' bus='0x00' slot='0x1a' function='0x7'/>
</source>
</hostdev>ehci driver is loaded by default for the USB PCI controller.
$ readlink /sys/bus/pci/devices/0000\:00\:1d.7/driver ../../../bus/pci/drivers/ehci_hcd
$ virsh nodedev-dettach pci_8086_3a6c
$ readlink /sys/bus/pci/devices/0000\:00\:1d.7/driver ../../../bus/pci/drivers/pci-stub
# setsebool -P virt_use_sysfs 1
# virsh start win2k3
virt-manager tool. The following procedure adds a 2 port USB controller to a virtualized guest.
Identify the device
virsh nodedev-list command lists all devices attached to the system. The --tree option is useful for identifying devices attached to the PCI device (for example, disk controllers and USB controllers).
# virsh nodedev-list --tree
# virsh nodedev-list | grep pci
**** is a four digit hexadecimal code):
pci_8086_****
lspci output to lspci -n (which turns off name resolution) output can assist in deriving which device has which device identifier code.
Detach the PCI device
# virsh nodedev-dettach pci_8086_3a6c Device pci_8086_3a6c dettached
Power off the guest
Open the hardware settings
Add the new device
Select a PCI device
Confirm the new device
--host-device parameter.
Identify the PCI device
virsh nodedev-list command lists all devices attached to the system. The --tree option is useful for identifying devices attached to the PCI device (for example, disk controllers and USB controllers).
# virsh nodedev-list --tree
# virsh nodedev-list | grep pci
**** is a four digit hexadecimal code):
pci_8086_****
lspci output to lspci -n (which turns off name resolution) output can assist in deriving which device has which device identifier code.
Add the device
virsh nodedev command as the value for the --host-device parameter.
# virt-install \
-n hostdev-test -r 1024 --vcpus 2 \
--os-variant fedora11 -v --accelerate \
-l http://download.fedoraproject.org/pub/fedora/linux/development/x86_64/os \
-x 'console=ttyS0 vnc' --nonetworks --nographics \
--disk pool=default,size=8 \
--debug --host-device=pci_8086_10bd
Complete the installation
/etc/modprobe.conf hides the device from dom0. Either the bnx2 module must be reloaded or the host must be restarted.
install bnx2 /sbin/modprobe pciback; /sbin/modprobe --first-time --ignore-install bnx2 options pciback hide=(0000:09:00.0)
/etc/modprobe.conf to hide multiple devices.
options pciback hide=(0000:09:00.0)(0000:0a:04.1)
virsh (Section 14.1, “Adding a PCI device with virsh” - Step 5);
virt-manager (Section 14.2, “Adding a PCI device with virt-manager”); or
virt-install (Section 14.3, “PCI passthrough with virt-install”)
/boot/grub/grub.conf file to enable SR-IOV. To enable SR-IOV with Xen for Intel systems append the pci_pt_e820_access=on parameter to the kernel.
default=0 timeout=5 splashimage=(hd0,0)/grub/splash.xpm.gz hiddenmenu title Red Hat Enterprise Linux Server (2.6.18-192.el5xen) root (hd0,0) kernel /xen.gz-2.6.18-192.el5 iommu=1 module /vmlinuz-2.6.18-192.el5xen ro root=/dev/VolGroup00/LogVol00 pci_pt_e820_access=on module /initrd-2.6.18-192.el5xen.img
Enable Intel VT-d in BIOS and in the kernel
Verify support
lspci command to verify if the device was detected.
# lspci 03:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01) 03:00.1 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)
Start the SR-IOV kernel modules
modprobe command. The Intel 82576 network interface card uses the igb driver kernel module.
# modprobe igb [<option>=<VAL1>,<VAL2>,] # lsmod |grep igb igb 87592 0 dca 6708 1 igb
Activate Virtual Functions
max_vfs parameter of the igb module allocates the maximum number of Virtual Functions. The max_vfs parameter causes the driver to spawn, up to the value of the parameter in, Virtual Functions. For this particular card the valid range is 0 to 7.
# modprobe -r igb
max_vfs set to 1 or any number of Virtual Functions up to the maximum supported by your device.
# modprobe igb max_vfs=1
Inspect the new Virtual Functions
lspci command, list the newly added Virtual Functions attached to the Intel 82576 network device.
# lspci | grep 82576 03:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01) 03:00.1 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01) 03:10.0 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01) 03:10.1 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)
-n parameter of the lspci command.
# lspci -n | grep 03:00.0 03:00.0 0200: 8086:10c9 (rev 01) # lspci -n | grep 03:10.0 03:10.0 0200: 8086:10ca (rev 01)The Physical Function corresponds to
8086:10c9 and the Virtual Function to 8086:10ca.
Find the devices with virsh
virsh nodedev-list command to list available host devices.
# virsh nodedev-list | grep 8086
pci_8086_10c9
pci_8086_10c9_0
pci_8086_10ca
pci_8086_10ca_0
[output truncated]
Get advanced details
pci_8086_10c9 is one of the Physical Functions and pci_8086_10ca_0 is the first corresponding Virtual Function for that Physical Function. Use the virsh nodedev-dumpxml command to get advanced output for both devices.
# virsh nodedev-dumpxml pci_8086_10ca
# virsh nodedev-dumpxml pci_8086_10ca_0
<device>
<name>pci_8086_10ca_0</name>
<parent>pci_8086_3408</parent>
<driver>
<name>igbvf</name>
</driver>
<capability type='pci'>
<domain>0</domain>
<bus>3</bus>
<slot>16</slot>
<function>1</function>
<product id='0x10ca'>82576 Virtual Function</product>
<vendor id='0x8086'>Intel Corporation</vendor>
</capability>
</device>
pci_8086_10ca_0 to the guest in Step 8. Note the bus, slot and function parameters of the Virtual Function, these are required for adding the device.
Add the Virtual Function to the guest
virsh nodedev-dumpxml pci_8086_10ca_0 command to calculate the values for the configuration file. Convert slot and function values to hexadecimal values (from decimal) to get the PCI bus addresses. Append "0x" to the beginning of the output to tell the computer that the value is a hexadecimal number.
printf utility to convert decimal values to hexadecimal values.
$ printf %x 3 3 $ printf %x 16 10 $ printf %x 1 1
bus='0x03' slot='0x10' function='0x01'
virsh edit command. This example edits a guest named MyGuest.
# virsh edit MyGuest
devices section of the XML configuration file.
<hostdev mode='subsystem' type='pci' managed='yes'>
<source>
<address bus='0x03' slot='0x10' function='0x01'/>
</source>
</hostdev>
Restart
# virsh start MyGuest
# virsh start test error: Failed to start domain test error: internal error unable to start guest: char device redirected to /dev/pts/2 get_real_device: /sys/bus/pci/devices/0000:03:10.0/config: Permission denied init_assigned_device: Error: Couldn't get real device (03:10.0)! Failed to initialize assigned device host=03:10.0
ntpd service:
# service ntpd start
# chkconfig ntpd on
ntpd service should minimize the affects of clock skew in all cases.
constant_tsc flag is present. To determine if your CPU has the constant_tsc flag run the following command:
$ cat /proc/cpuinfo | grep constant_tsc
constant_tsc bit. If no output is given follow the instructions below.
constant_tsc bit, disable all power management features (BZ#513138). Each system has several timers it uses to keep time. The TSC is not stable on the host, which is sometimes caused by cpufreq changes, deep C state, or migration to a host with a faster TSC. Deep C sleep states can stop the TSC. To prevent the kernel using deep C states append "processor.max_cstate=1" to the kernel boot options in the grub.conf file on the host:
term Red Hat Enterprise Linux Server (2.6.18-159.el5)
root (hd0,0)
kernel /vmlinuz-2.6.18-159.el5 ro root=/dev/VolGroup00/LogVol00 rhgb quiet processor.max_cstate=1
cpufreq (only necessary on hosts without the constant_tsc) by editing the /etc/sysconfig/cpuspeed configuration file and change the MIN_SPEED and MAX_SPEED variables to the highest frequency available. Valid limits can be found in the /sys/devices/system/cpu/cpu*/cpufreq/scaling_available_frequencies files.
| Red Hat Enterprise Linux | Additional guest kernel parameters |
|---|---|
| 5.4 AMD64/Intel 64 with the para-virtualized clock | Additional parameters are not required |
| 5.4 AMD64/Intel 64 without the para-virtualized clock | divider=10 notsc lpj=n |
| 5.4 x86 with the para-virtualized clock | Additional parameters are not required |
| 5.4 x86 without the para-virtualized clock | divider=10 clocksource=acpi_pm lpj=n |
| 5.3 AMD64/Intel 64 | divider=10 notsc |
| 5.3 x86 | divider=10 clocksource=acpi_pm |
| 4.8 AMD64/Intel 64 | notsc divider=10 |
| 4.8 x86 | clock=pmtmr divider=10 |
| 3.9 AMD64/Intel 64 | Additional parameters are not required |
| 3.9 x86 | Additional parameters are not required |
boot.ini file:
/use pmtimer
boot.ini file is no longer used from Windows Vista and newer. Windows Vista, Windows Server 2008 and Windows 7 use the Boot Configuration Data Editor (bcdedit.exe) to modify the Windows boot parameters.
{default} in the example below) should be changed if the system UUID is different than the default boot device.
C:\Windows\system32>bcdedit /set {default} USEPLATFORMCLOCK on
The operation completed successfully# setenforce 1
AutoFS, NFS, FTP, HTTP, NIS, telnetd, sendmail and so on.
/var/lib/libvirt/images/. If you are using a different directory for your virtual machine images make sure you add the directory to your SELinux policy and relabel it before starting the installation.
vsftpd server.
fstab file, the initrd file or used by the kernel command line. If less privileged users, especially virtualized guests, have write access to whole partitions or LVM volumes.
/dev/sdb). Use partitions (for example, /dev/sdb1) or LVM volumes.
virt_image_t label applied to them. The /var/lib/libvirt/images directory has this label applied to it and its contents by default. This does not mean that images must be stored in this directory; images can be stored anywhere, provided they are labeled with virt_image_t.
NewVolumeName on the volume group named volumegroup.
# lvcreate -nNewVolumeName-L5G volumegroup
NewVolumeName logical volume with a file system that supports extended attributes, such as ext3.
# mke2fs -j /dev/volumegroup/NewVolumeName
/etc, /var, /sys) or in home directories (/home or /root). This example uses a directory called /virtstorage
# mkdir /virtstorage
# mount /dev/volumegroup/NewVolumeName /virtstoragesemanage fcontext -a -t xen_image_t "/virtstorage(/.*)?"
semanage fcontext -a -t virt_image_t "/virtstorage(/.*)?"
/etc/selinux/targeted/contexts/files/file_contexts.local file which makes the change persistent. The appended line may resemble this:
/virtstorage(/.*)? system_u:object_r:xen_image_t:s0
/dev/volumegroup/NewVolumeName with the correct label:
# semanage fcontext -a -t xen_image_t /dev/volumegroup/NewVolumeName # restorecon /dev/volumegroup/NewVolumeName
# semanage fcontext -a -t xen_image_t -f -b /dev/sda2 # restorecon /dev/sda2
xend_disable_t can set the xend to unconfined mode after restarting the daemon. It is better to disable protection for a single daemon than the whole system. It is advisable that you should not re-label directories as xen_image_t that you will use elsewhere.
| SELinux Boolean | Description |
|---|---|
| allow_unconfined_qemu_transition | Default: off. This boolean controls whether KVM guests can be transitioned to unconfined users. |
| qemu_full_network | Default: on. This boolean controls full network access to KVM guests. |
| qemu_use_cifs | Default: on. This boolean controls KVM's access to CIFS or Samba file systems. |
| qemu_use_comm | Default: off. This boolean controls whether KVM can access serial or parallel communications ports. |
| qemu_use_nfs | Default: on. This boolean controls KVM's access to NFS file systems. |
| qemu_use_usb | Default: on. This boolean allows KVM to access USB devices. |
net.ipv4.ip_forward = 1) is required for virtual bridge devices. Note that installing libvirt enables this variable so it will be enabled when the virtualization packages are installed unless it was manually disabled.
xend/etc/xen/xend-config.sxp. Here are the parameters you can enable or disable in the xend-config.sxp configuration file:
| Item | Description |
|---|---|
|
(console-limit)
|
Determines the console server's memory buffer limit and assigns that limit on a per domain basis.
|
|
(min-mem)
|
Determines the minimum number of megabytes that is reserved for domain0 (if you enter 0, the value does not change).
|
|
(dom0-cpus)
|
Determines the number of CPUs in use by domain0 (at least 1 CPU is assigned by default).
|
|
(enable-dump)
|
If this is enabled, when a crash occurs Xen creates a dump file (the default is 0).
|
|
(external-migration-tool)
|
Determines the script or application that handles external device migration. The scripts must reside in the
/etc/xen/scripts/external-device-migrate directory.
|
|
(logfile)
|
Determines the location of the log file (default is
/var/log/xend.log).
|
|
(loglevel)
|
Filters out the log mode values: DEBUG, INFO, WARNING, ERROR, or CRITICAL (default is DEBUG).
|
|
(network-script)
|
Determines the script that enables the networking environment. The scripts must reside in the
/etc/xen/scripts/ directory.
|
|
(xend-http-server)
|
Enables the http stream packet management server (the default is no).
|
|
(xend-unix-server)
|
Enables the UNIX domain socket server. The socket server is a communications endpoint that handles low level network connections and accepts or rejects incoming connections. The default value is set to yes.
|
|
(xend-relocation-server)
|
Enables the relocation server for cross-machine migrations (the default is no).
|
|
(xend-unix-path)
|
Determines the location where the
xend-unix-server command outputs data (default is /var/lib/xend/xend-socket)
|
|
(xend-port)
|
Determines the port that the http management server uses (the default is 8000).
|
|
(xend-relocation-port)
|
Determines the port that the relocation server uses (the default is 8002).
|
|
(xend-relocation-address)
|
Determines the host addresses allowed for migration. The default value is the value of
xend-address.
|
|
(xend-address)
|
Determines the address that the domain socket server binds to. The default value allows all connections.
|
service xend start
service xend stop
service xend restart
service xend status
xend at boot timechkconfig command to add the xend to the initscript.
chkconfig --level 345 xend
xend will now start at runlevels 3, 4 and 5.
virsh migrate command.
# virsh migrateGuestNamelibvirtURI
--live option for the virsh migrate command.
# virsh migrate--liveGuestNamelibvirtURI
/etc/xen/xend-config.sxp configuration file. By default, migration is disabled as migration can be a potential security hazard if incorrectly configured. Opening the migration port can allow an unauthorized host to initiate a migration or connect to the migration ports. Authentication and authorization are not configured for migration requests and the only control mechanism is based on hostnames and IP addresses. Special care should be taken to ensure the migration port is not accessible to unauthorized hosts.
/etc/xen/xend-config.sxp to enable migration. Modify the values, when necessary, and remove the comments (the # symbol) preceding the following parameters:
(xend-relocation-server yes)no. Change the value of xend-relocation-server to yes to enable migration.
(xend-relocation-port 8002)(xend-relocation-port), specifies the port xend should use for the relocation interface, if xend-relocation-server is set to yes
xend-relocation-port parameter must be open on both systems.
(xend-relocation-address '')(xend-relocation-address) is the address the xend listens for migration commands on the relocation-socket connection if xend-relocation-server is set.
(xend-relocation-address) parameter restricts the migration server to only listen to a specific interface. The default value in /etc/xen/xend-config.sxp is an empty string(''). This value should be replaced with a single, valid IP address. For example:
(xend-relocation-address '10.0.0.1')
(xend-relocation-hosts-allow '')(xend-relocation-hosts-allow 'hosts') parameter controls which hostnames can communicate on the relocation port.
xend-relocation-hosts-allow option to restrict access to the migration server.
xend-relocation-port and xend-relocation-address.
(xend-relocation-hosts-allow) parameter should be a sequence of regular expressions separated by spaces. Any host with a fully-qualified domain name or an IP address which matches one of these regular expressions will be accepted.
(xend-relocation-hosts-allow) attribute:
(xend-relocation-hosts-allow '^localhost$ ^localhost\\.localdomain$')
# service xend restart
et-virt07 and et-virt08), both of them are using eth1 as their default network interface hence they are using xenbr1 as their Xen networking bridge. We are using a locally attached SCSI disk (/dev/sdb) on et-virt07 for shared storage using NFS.
# mkdir /var/lib/libvirt/images # mount /dev/sdb /var/lib/libvirt/images
/var/lib/libvirt/images/ make sure you only export /var/lib/libvirt/images/ and not/var/lib/xen/ as this directory is used by the xend daemon and other tools. Sharing /var/lib/xen/ will cause unpredictable behavior.
# cat /etc/exports /var/lib/libvirt/images *(rw,async,no_root_squash)
# showmount -e et-virt07 Export list for et-virt07: /var/lib/libvirt/images *
# virt-install -p -f /var/lib/libvirt/images/testvm1.dsk -s 5 -n\testvm1--vnc -r 1024 -l http://example.com/RHEL5-tree\ Server/x86-64/os/ -b xenbr1
[et-virt08 ~]# brctl show bridge name bridge id STP enabled interfaces xenbr1 8000.feffffffffff no peth1 vif0.1
[et-virt07 ~]# brctl show bridge name bridge id STP enabled interfaces xenbr1 8000.feffffffffff no peth1 vif0.1
[et-virt07 ~]# grep xend-relocation /etc/xen/xend-config.sxp |grep -v '#' (xend-relocation-server yes) (xend-relocation-port 8002) (xend-relocation-address '') (xend-relocation-hosts-allow '')
[et-virt08 ~]# grep xend-relocation /etc/xen/xend-config.sxp |grep -v '#' (xend-relocation-server yes) (xend-relocation-port 8002) (xend-relocation-address '') (xend-relocation-hosts-allow '')
[et-virt07 ~]# lsof -i :8002 COMMAND PID USER FD TYPE DEVICE SIZE NODE NAME python 3445 root 14u IPv4 10223 TCP *:teradataordbms (LISTEN)
[et-virt08 ~]# lsof -i :8002 COMMAND PID USER FD TYPE DEVICE SIZE NODE NAME python 3252 root 14u IPv4 10901 TCP *:teradataordbms (LISTEN)
/var/lib/libvirt/images directory is available and mounted with networked storage on both hosts. Shared, networked storage is required for migrations.
[et-virt08 ~]# df /var/lib/libvirt/images Filesystem 1K-blocks Used Available Use% Mounted on et-virt07:/var/lib/libvirt/images 70562400 2379712 64598336 4% /var/lib/libvirt/images
[et-virt08 ~]# file /var/lib/libvirt/images/testvm1.dsk /var/lib/libvirt/images/testvm1.dsk: x86 boot sector; partition 1: ID=0x83, active, starthead 1, startsector 63, 208782 sectors; partition 2: ID=0x8e, starthead 0, startsector 208845, 10265535 sectors, code offset 0x48
[et-virt08 ~]# touch /var/lib/libvirt/images/foo [et-virt08 ~]# rm -f /var/lib/libvirt/images/foo
[et-virt07 ~]# virsh list Id Name State ---------------------------------- Domain-0 running
[et-virt07 ~]# virsh starttestvm1Domaintestvm1started
[et-virt07 ~]# virsh list
Id Name State
----------------------------------
Domain-0 running
testvm1 blocked
[et-virt07 images]# time virsh savetestvm1testvm1.sav real 0m15.744s user 0m0.188s sys 0m0.044s
[et-virt07 images]# ls -lrttestvm1.sav -rwxr-xr-x 1 root root 1075657716 Jan 12 06:46testvm1.sav
[et-virt07 images]# virsh list Id Name State ---------------------------------- Domain-0 running
[et-virt07 images]# virsh restore testvm1.sav
[et-virt07 images]# virsh list
Id Name State
----------------------------------
Domain-0 running
testvm1 blocked
domain-id from et-virt08 to et-virt07. The hostname you are migrating to and <domain-id> must be replaced with valid values. This example uses the et-virt08 host which must have SSH access to et-virt07
[et-virt08 ~]# xm migrate --live testvm1 et-virt07
et-virt08
[et-virt08 ~]# virsh list Id Name State ---------------------------------- Domain-0 running
et-virt07:
[et-virt07 ~]# virsh list
Id Name State
----------------------------------
Domain-0 running
testvm1 running
#!/bin/bash while true do touch /var/tmp/$$.log echo `hostname` >> /var/tmp/$$.log echo `date` >> /var/tmp/$$.log cat /var/tmp/$$.log df /var/tmp ls -l /var/tmp/$$.log sleep 3 done
et-virt07:
[et-virt08 ~]# virsh list
Id Name State
----------------------------------
Domain-0 running
testvm1 blocked
et-virt07. You can add the time command to see how long the migration takes:
[et-virt08 ~]# xm migrate --live testvm1 et-virt07
# ./doit dhcp78-218.lab.boston.redhat.com Fri Jan 12 02:26:27 EST 2007 Filesystem 1K-blocks Used Available Use% Mounted on /dev/mapper/VolGroup00-LogVol00 2983664 2043120 786536 73% / -rw-r--r-- 1 root root 62 Jan 12 02:26 /var/tmp/2279.log dhcp78-218.lab.boston.redhat.com Fri Jan 12 02:26:27 EST 2007 dhcp78-218.lab.boston.redhat.com Fri Jan 12 02:26:30 EST 2007 Filesystem 1K-blocks Used Available Use% Mounted on /dev/mapper/VolGroup00-LogVol00 2983664 2043120 786536 73% / -rw-r--r-- 1 root root 124 Jan 12 02:26 /var/tmp/2279.log dhcp78-218.lab.boston.redhat.com Fri Jan 12 02:26:27 EST 2007 dhcp78-218.lab.boston.redhat.com Fri Jan 12 02:26:30 EST 2007 dhcp78-218.lab.boston.redhat.com Fri Jan 12 02:26:33 EST 2007 Filesystem 1K-blocks Used Available Use% Mounted on /dev/mapper/VolGroup00-LogVol00 2983664 2043120 786536 73% / -rw-r--r-- 1 root root 186 Jan 12 02:26 /var/tmp/2279.log Fri Jan 12 02:26:45 EST 2007 dhcp78-218.lab.boston.redhat.com Fri Jan 12 02:26:48 EST 2007 dhcp78-218.lab.boston.redhat.com Fri Jan 12 02:26:51 EST 2007 dhcp78-218.lab.boston.redhat.com Fri Jan 12 06:54:57 EST 2007 dhcp78-218.lab.boston.redhat.com Fri Jan 12 06:55:00 EST 2007 dhcp78-218.lab.boston.redhat.com Fri Jan 12 06:55:03 EST 2007 Filesystem 1K-blocks Used Available Use% Mounted on /dev/mapper/VolGroup00-LogVol00 2983664 2043120 786536 73% / -rw-r--r-- 1 root root 744 Jan 12 06:55 /var/tmp/2279.log dhcp78-218.lab.boston.redhat.com Fri Jan 12 02:26:27 EST 2007
et-virt08:
[et-virt08 ~]# virsh list Id Name State ---------------------------------- Domain-0 running
et-virt07:
[et-virt07 images]# virsh list
Id Name State
----------------------------------
Domain-0 running
testvm1 blocked
et-virt07 to et-virt08. Initiate a migration from et-virt07 to et-virt08:
[et-virt07 images]# xm migrate --live testvm1 et-virt08
[et-virt07 images]# virsh list Id Name State ---------------------------------- Domain-0 running
# ./doit dhcp78-218.lab.boston.redhat.com Fri Jan 12 06:57:53 EST 2007 Filesystem 1K-blocks Used Available Use% Mounted on /dev/mapper/VolGroup00-LogVol00 2983664 2043120 786536 73% / -rw-r--r-- 1 root root 62 Jan 12 06:57 /var/tmp/2418.log dhcp78-218.lab.boston.redhat.com Fri Jan 12 06:57:53 EST 2007 dhcp78-218.lab.boston.redhat.com Fri Jan 12 06:57:56 EST 2007 Filesystem 1K-blocks Used Available Use% Mounted on /dev/mapper/VolGroup00-LogVol00 2983664 2043120 786536 73% / -rw-r--r-- 1 root root 124 Jan 12 06:57 /var/tmp/2418.log dhcp78-218.lab.boston.redhat.com Fri Jan 12 06:57:53 EST 2007 dhcp78-218.lab.boston.redhat.com Fri Jan 12 06:57:56 EST 2007 dhcp78-218.lab.boston.redhat.com Fri Jan 12 06:58:00 EST 2007 Filesystem 1K-blocks Used Available Use% Mounted on /dev/mapper/VolGroup00-LogVol00 2983664 2043120 786536 73% / -rw-r--r-- 1 root root 186 Jan 12 06:57 /var/tmp/2418.log dhcp78-218.lab.boston.redhat.com Fri Jan 12 06:57:53 EST 2007 dhcp78-218.lab.boston.redhat.com Fri Jan 12 06:57:56 EST 2007 dhcp78-218.lab.boston.redhat.com Fri Jan 12 06:58:00 EST 2007 dhcp78-218.lab.boston.redhat.com Fri Jan 12 02:30:00 EST 2007 Filesystem 1K-blocks Used Available Use% Mounted on /dev/mapper/VolGroup00-LogVol00 2983664 2043120 786536 73% / -rw-r--r-- 1 root root 248 Jan 12 02:30 /var/tmp/2418.log dhcp78-218.lab.boston.redhat.com Fri Jan 12 06:57:53 EST 2007 dhcp78-218.lab.boston.redhat.com Fri Jan 12 06:57:56 EST 2007 dhcp78-218.lab.boston.redhat.com Fri Jan 12 06:58:00 EST 2007 dhcp78-218.lab.boston.redhat.com Fri Jan 12 02:30:00 EST 2007 dhcp78-218.lab.boston.redhat.com Fri Jan 12 02:30:03 EST 2007 Filesystem 1K-blocks Used Available Use% Mounted on /dev/mapper/VolGroup00-LogVol00 2983664 2043120 786536 73% / -rw-r--r-- 1 root root 310 Jan 12 02:30 /var/tmp/2418.log dhcp78-218.lab.boston.redhat.com Fri Jan 12 06:57:53 EST 2007 dhcp78-218.lab.boston.redhat.com Fri Jan 12 06:57:56 EST 2007 dhcp78-218.lab.boston.redhat.com Fri Jan 12 06:58:00 EST 2007 dhcp78-218.lab.boston.redhat.com Fri Jan 12 02:30:00 EST 2007 dhcp78-218.lab.boston.redhat.com Fri Jan 12 02:30:03 EST 2007 dhcp78-218.lab.boston.redhat.com Fri Jan 12 02:30:06 EST 2007 Filesystem 1K-blocks Used Available Use% Mounted on /dev/mapper/VolGroup00-LogVol00 2983664 2043120 786536 73% / -rw-r--r-- 1 root root 372 Jan 12 02:30 /var/tmp/2418.log
et-virt07 verify on et-virt08 that the virtual machine has started:
[et-virt08 ~]# virsh list
Id Name State
----------------------------------
Domain-0 running
testvm1 blocked
[et-virt08 ~]# time virsh migrate --live testvm1 et-virt07
real 0m10.378s
user 0m0.068s
sys 0m0.052s
xm migrate command). Live migration can be done from the same command. However there are some additional modifications that you must do to the xend-config configuration file. This example identifies the entries that you must modify to ensure a successful migration:
(xend-relocation-server yes)(xend-relocation-port 8002)xend uses for migration. Use this value unless your network environment requires a custom value. Remove the comment symbol to enable it.
(xend-relocation-address )xend-relocation-server . The Xen hypervisor only listens for migration network traffic on the specified interface.
(xend-relocation-hosts-allow )(xend-relocation-hosts-allow- '^localhost\\.localdomain$' )>
virsh command. The migrate command accepts parameters in the following format:
# virsh migrate --live GuestName DestinationURLGuestName parameter represents the name of the guest which you want to migrate.
DestinationURL parameter is the URL or hostname of the destination system. The destination system must run the same version of Red Hat Enterprise Linux, be using the same hypervisor and have libvirt running.
test1.example.com to test2.example.com. Change the host names for your environment. This example migrates a virtual machine named RHEL4test.
Verify the guest is running
test1.example.com, verify RHEL4test is running:
[root@test1 ~]# virsh list Id Name State ---------------------------------- 10 RHEL4 running
Migrate the guest
test2.example.com. Append /system to the end of the destination URL to tell libvirt that you need full access.
# virsh migrate --live RHEL4test qemu+ssh://test2.example.com/systemWait
virsh only reports errors. The guest continues to run on the source host until fully migrated.
Verify the guest has arrived at the destination host
test2.example.com, verify RHEL4test is running:
[root@test2 ~]# virsh list Id Name State ---------------------------------- 10 RHEL4 running
virt-manager.
ssh or TLS and SSL.
libvirt management connection securely tunneled over an SSH connection to manage the remote machines. All the authentication is done using SSH public key cryptography and passwords or passphrases gathered by your local SSH agent. In addition the VNC console for each guest virtual machine is tunneled over SSH.
virt-manager virt-manager must run as the user who owns the keys to connect to the remote host. That means, if the remote systems are managed by a non-root user virt-manager must be run in unprivileged mode. If the remote systems are managed by the local root user then the SSH keys must be own and created by root.
virt-manager.
Optional: Changing user
$ su -
Generating the SSH key pair
virt-manager is used. This example uses the default key location, in the ~/.ssh/ directory.
$ ssh-keygen -t rsa
Coping the keys to the remote hosts
root@example.com).
# ssh-copy-id -i ~/.ssh/id_rsa.pub root@example.com root@example.com's password: Now try logging into the machine, with "ssh 'root@example.com'", and check in: .ssh/authorized_keys to make sure we haven't added extra keys that you weren't expecting
Optional: Add the passphrase to the ssh-agent
ssh-agent, if required. On the local host, use the following command to add the passphrase (if there was one) to enable password-less login.
# ssh-add ~/.ssh/id_rsa.pub
libvirt daemon (libvirtd)libvirt daemon provide an interface for managing virtual machines. You must have the libvirtd daemon installed and running on every remote host that needs managing.
$ ssh root@somehost
# chkconfig libvirtd on
# service libvirtd start
libvirtd and SSH are configured you should be able to remotely access and manage your virtual machines. You should also be able to access your guests with VNC at this point.
libvirt management connection opens a TCP port for incoming connections, which is securely encrypted and authenticated based on x509 certificates. In addition the VNC console for each guest virtual machine will be setup to use TLS with x509 certificate authentication.
libvirt server setup/etc/xen/xend-config.sxp. Remove the commenting on the (vnc-tls 1) configuration parameter in the configuration file.
/etc/xen/vnc directory needs the following 3 files:
ca-cert.pem - The CA certificate
server-cert.pem - The Server certificate signed by the CA
server-key.pem - The server private key
(vnc-x509-verify 1) parameter.
virt-manager and virsh client setuplibvirt management API over TLS, the CA and client certificates must be placed in /etc/pki. For details on this consult http://libvirt.org/remote.html
virt-manager user interface, use the '' transport mechanism option when connecting to a host.
virsh, the URI has the following format:
qemu://hostname.guestname/system for KVM.
xen://hostname.guestname/ for Xen.
$HOME/.pki, that is the following three files:
ca-cert.pem - The CA certificate.
libvirt-vnc or clientcert.pem - The client certificate signed by the CA.
libvirt-vnc or clientkey.pem - The client private key.
libvirt supports the following transport modes:
/var/run/libvirt/libvirt-sock and /var/run/libvirt/libvirt-sock-ro (for read-only connections).
libvirtd) must be running on the remote machine. Port 22 must be open for SSH access. You should use some sort of ssh key management (for example, the ssh-agent utility) or you will be prompted for a password.
ext parameter is used for any external program which can make a connection to the remote machine by means outside the scope of libvirt. This parameter is unsupported.
virsh and libvirt to connect to a remote host. URIs can also be used with the --connect parameter for the virsh command to execute single commands or migrations on remote hosts.
driver[+transport]://[username@][hostname][:port]/[path][?extraparameters]
towada, using SSH transport and the SSH username ccurran.
xen+ssh://ccurran@towada/
towada using TLS.
xen://towada/
towada using TLS. The no_verify=1 tells libvirt not to verify the server's certificate.
xen://towada/?no_verify=1
towada using SSH.
qemu+ssh://towada/system
qemu+unix:///system?socket=/opt/libvirt/run/libvirt/libvirt-sock
test+tcp://10.1.1.10:5000/default
| Name | Transport mode | Description | Example usage |
|---|---|---|---|
| name | all modes | The name passed to the remote virConnectOpen function. The name is normally formed by removing transport, hostname, port number, username and extra parameters from the remote URI, but in certain very complex cases it may be better to supply the name explicitly. | name=qemu:///system |
| command | ssh and ext | The external command. For ext transport this is required. For ssh the default is ssh. The PATH is searched for the command. | command=/opt/openssh/bin/ssh |
| socket | unix and ssh | The path to the UNIX domain socket, which overrides the default. For ssh transport, this is passed to the remote netcat command (see netcat). | socket=/opt/libvirt/run/libvirt/libvirt-sock |
| netcat | ssh |
The
netcat command can be used to connect to remote systems. The default netcat parameter uses the nc command. For SSH transport, libvirt constructs an SSH command using the form below:
The
port, username and hostname parameters can be specified as part of the remote URI. The command, netcat and socket come from other extra parameters.
| netcat=/opt/netcat/bin/nc |
| no_verify | tls | If set to a non-zero value, this disables client checks of the server's certificate. Note that to disable server checks of the client's certificate or IP address you must change the libvirtd configuration. | no_verify=1 |
| no_tty | ssh | If set to a non-zero value, this stops ssh from asking for a password if it cannot log in to the remote machine automatically (for using ssh-agent or similar). Use this when you do not have access to a terminal - for example in graphical programs which use libvirt. | no_tty=1 |
Table of Contents
vmstat
iostat
lsof
# lsof -i :5900 xen-vncfb 10635 root 5u IPv4 218738 TCP grumble.boston.redhat.com:5900 (LISTEN)
qemu-img
systemTap
crash
xen-gdbserver
sysrq
sysrq t
sysrq w
sysrq c
brtcl
# brctl show
bridge name bridge id STP enabled interfaces
xenbr0 8000.feffffffffff no vif13.0
pdummy0
vif0.0
# brctl showmacs xenbr0 port no mac addr is local? aging timer 1 fe:ff:ff:ff:ff:ff yes 0.00
# brctl showstp xenbr0 xenbr0 bridge id 8000.feffffffffff designated root 8000.feffffffffff root port 0 path cost 0 max age 20.00 bridge max age 20.00 hello time 2.00 bridge hello time 2.00 forward delay 0.00 bridge forward delay 0.00 aging time 300.01 hello timer 1.43 tcn timer 0.00 topology change timer 0.00 gc timer 0.02 flags vif13.0 (3) port id 8003 state forwarding designated root 8000.feffffffffff path cost 100 designated bridge 8000.feffffffffff message age timer 0.00 designated port 8003 forward delay timer 0.00 designated cost 0 hold timer 0.43 flags pdummy0 (2) port id 8002 state forwarding designated root 8000.feffffffffff path cost 100 designated bridge 8000.feffffffffff message age timer 0.00 designated port 8002 forward delay timer 0.00 designated cost 0 hold timer 0.43 flags vif0.0 (1) port id 8001 state forwarding designated root 8000.feffffffffff path cost 100 designated bridge 8000.feffffffffff message age timer 0.00 designated port 8001 forward delay timer 0.00 designated cost 0 hold timer 0.43 flags
ifconfig
tcpdump
ps
pstree
top
kvmtrace
kvm_stat
xentop
xm dmesg
xm log
virsh is a command line interface tool for managing guests and the hypervisor.
virsh tool is built on the libvirt management API and operates as an alternative to the xm command and the graphical guest Manager (virt-manager). virsh can be used in read-only mode by unprivileged users. You can use virsh to execute scripts for the guest machines.
| Command | Description |
|---|---|
help
| Prints basic help information. |
list
| Lists all guests. |
dumpxml
| Outputs the XML configuration file for the guest. |
create
| Creates a guest from an XML configuration file and starts the new guest. |
start
| Starts an inactive guest. |
destroy
| Forces a guest to stop. |
define
| Outputs an XML configuration file for a guest. |
domid
| Displays the guest's ID. |
domuuid
| Displays the guest's UUID. |
dominfo
| Displays guest information. |
domname
| Displays the guest's name. |
domstate
| Displays the state of a guest. |
quit
| Quits the interactive terminal. |
reboot
| Reboots a guest. |
restore
| Restores a previously saved guest stored in a file. |
resume
| Resumes a paused guest. |
save
| Save the present state of a guest to a file. |
shutdown
| Gracefully shuts down a guest. |
suspend
| Pauses a guest. |
undefine
| Deletes all files associated with a guest. |
migrate
| Migrates a guest to another host. |
virsh command options manage guest and hypervisor resources:
| Command | Description |
|---|---|
setmem
| Sets the allocated memory for a guest. |
setmaxmem
| Sets maximum memory limit for the hypervisor. |
setvcpus
| Changes number of virtual CPUs assigned to a guest. |
vcpuinfo
| Displays virtual CPU information about a guest. |
vcpupin
| Controls the virtual CPU affinity of a guest. |
domblkstat
| Displays block device statistics for a running guest. |
domifstat
| Displays network interface statistics for a running guest. |
attach-device
| Attach a device to a guest, using a device definition in an XML file. |
attach-disk
| Attaches a new disk device to a guest. |
attach-interface
| Attaches a new network interface to a guest. |
detach-device
|
Detach a device from a guest, takes the same kind of XML descriptions as command attach-device.
|
detach-disk
| Detach a disk device from a guest. |
detach-interface
| Detach a network interface from a guest. |
virsh options:
| Command | Description |
|---|---|
version
|
Displays the version of virsh
|
nodeinfo
| Outputs information about the hypervisor |
virsh:
# virsh connect {hostname OR URL}<name> is the machine name of the hypervisor. To initiate a read-only connection, append the above command with -readonly.
virsh:
# virsh dumpxml {domain-id, domain-name or domain-uuid}stdout). You can save the data by piping the output to a file. An example of piping the output to a file called guest.xml:
# virsh dumpxmlThis fileGuestID>guest.xml
guest.xml can recreate the guest (refer to Editing a guest's configuration file. You can edit this XML configuration file to configure additional devices or to deploy additional guests. Refer to Section 33.1, “Using XML configuration files with virsh” for more information on modifying files created with virsh dumpxml.
virsh dumpxml output:
# virsh dumpxml r5b2-mySQL01
<domain type='xen' id='13'>
<name>r5b2-mySQL01</name>
<uuid>4a4c59a7ee3fc78196e4288f2862f011</uuid>
<bootloader>/usr/bin/pygrub</bootloader>
<os>
<type>linux</type>
<kernel>/var/lib/libvirt/vmlinuz.2dgnU_</kernel>
<initrd>/var/lib/libvirt/initrd.UQafMw</initrd>
<cmdline>ro root=/dev/VolGroup00/LogVol00 rhgb quiet</cmdline>
</os>
<memory>512000</memory>
<vcpu>1</vcpu>
<on_poweroff>destroy</on_poweroff>
<on_reboot>restart</on_reboot>
<on_crash>restart</on_crash>
<devices>
<interface type='bridge'>
<source bridge='xenbr0'/>
<mac address='00:16:3e:49:1d:11'/>
<script path='vif-bridge'/>
</interface>
<graphics type='vnc' port='5900'/>
<console tty='/dev/pts/4'/>
</devices>
</domain>
dumpxml option (refer to Creating a virtual machine XML dump (configuration file)). To create a guest with virsh from an XML file:
# virsh create configuration_file.xmldumpxml option (refer to Creating a virtual machine XML dump (configuration file)) guests can be edited either while they run or while they are offline. The virsh edit command provides this functionality. For example, to edit the guest named softwaretesting:
# virsh edit softwaretesting$EDITOR shell parameter (set to vi by default).
virsh:
# virsh suspend {domain-id, domain-name or domain-uuid}resume (Resuming a guest) option.
virsh using the resume option:
# virsh resume {domain-id, domain-name or domain-uuid}suspend and resume operations.
virsh command:
# virsh save {domain-name, domain-id or domain-uuid} filenamerestore (Restore a guest) option. Save is similar to pause, instead of just pausing a guest the present state of the guest is saved.
virsh save command (Save a guest) using virsh:
# virsh restore filenamevirsh command:
# virsh shutdown {domain-id, domain-name or domain-uuid}on_shutdown parameter in the guest's configuration file.
virsh command:
#virsh reboot {domain-id, domain-name or domain-uuid}on_reboot element in the guest's configuration file.
virsh command:
# virsh destroy {domain-id, domain-name or domain-uuid}virsh destroy can corrupt guest file systems . Use the destroy option only when the guest is unresponsive. For para-virtualized guests, use the shutdown option(Shut down a guest) instead.
# virsh domid {domain-name or domain-uuid}# virsh domname {domain-id or domain-uuid}# virsh domuuid {domain-id or domain-name}virsh domuuid output:
# virsh domuuid r5b2-mySQL01 4a4c59a7-ee3f-c781-96e4-288f2862f011
virsh with the guest's domain ID, domain name or UUID you can display information on the specified guest:
# virsh dominfo {domain-id, domain-name or domain-uuid}virsh dominfo output:
# virsh dominfo r5b2-mySQL01 id: 13 name: r5b2-mysql01 uuid: 4a4c59a7-ee3f-c781-96e4-288f2862f011 os type: linux state: blocked cpu(s): 1 cpu time: 11.0s max memory: 512000 kb used memory: 512000 kb
# virsh nodeinfo
virsh nodeinfo output:
# virsh nodeinfo CPU model x86_64 CPU (s) 8 CPU frequency 2895 Mhz CPU socket(s) 2 Core(s) per socket 2 Threads per core: 2 Numa cell(s) 1 Memory size: 1046528 kb
virsh:
# virsh list
--inactive option to list inactive guests (that is, guests that have been defined but are not currently active), and
--all option lists all guests. For example:
# virsh list --all Id Name State ---------------------------------- 0 Domain-0 running 1 Domain202 paused 2 Domain010 inactive 3 Domain9600 crashed
virsh list is categorized as one of the six states (listed below).
running state refers to guests which are currently active on a CPU.
blocked are blocked, and are not running or runnable. This is caused by a guest waiting on I/O (a traditional wait state) or guests in a sleep mode.
paused state lists domains that are paused. This occurs if an administrator uses the pause button in virt-manager, xm pause or virsh suspend. When a guest is paused it consumes memory and other resources but it is ineligible for scheduling and CPU resources from the hypervisor.
shutdown state is for guests in the process of shutting down. The guest is sent a shutdown signal and should be in the process of stopping its operations gracefully. This may not work with all guest operating systems; some operating systems do not respond to these signals.
dying state are in is in process of dying, which is a state where the domain has not completely shut-down or crashed.
crashed guests have failed while running and are no longer running. This state can only occur if the guest has been configured not to restart on crash.
virsh:
# virsh vcpuinfo {domain-id, domain-name or domain-uuid}virsh vcpuinfo output:
# virsh vcpuinfo r5b2-mySQL01 VCPU: 0 CPU: 0 State: blocked CPU time: 0.0s CPU Affinity: yy
# virsh vcpupin domain-id vcpu cpulistdomain-id parameter is the guest's ID number or name.
vcpu parameter denotes the number of virtualized CPUs allocated to the guest.The vcpu parameter must be provided.
cpulist parameter is a list of physical CPU identifier numbers separated by commas. The cpulist parameter determines which physical CPUs the VCPUs can run on.
virsh:
# virsh setvcpus {domain-name, domain-id or domain-uuid} countcount value cannot exceed the count above the amount specified when the guest was created.
virsh :
# virsh setmem {domain-id or domain-name} count
virsh domblkstat to display block device statistics for a running guest.
# virsh domblkstat GuestName block-devicevirsh domifstat to display network interface statistics for a running guest.
# virsh domifstat GuestName interface-device virsh. Migrate domain to another host. Add --live for live migration. The migrate command accepts parameters in the following format:
# virsh migrate --live GuestName DestinationURL--live parameter is optional. Add the --live parameter for live migrations.
GuestName parameter represents the name of the guest which you want to migrate.
DestinationURL parameter is the URL or hostname of the destination system. The destination system requires:
libvirt service must be started.
virsh command. To list virtual networks:
# virsh net-list
# virsh net-list Name State Autostart ----------------------------------------- default active yes vnet1 active yes vnet2 active yes
# virsh net-dumpxml NetworkName# virsh net-dumpxml vnet1
<network>
<name>vnet1</name>
<uuid>98361b46-1581-acb7-1643-85a412626e70</uuid>
<forward dev='eth0'/>
<bridge name='vnet0' stp='on' forwardDelay='0' />
<ip address='192.168.100.1' netmask='255.255.255.0'>
<dhcp>
<range start='192.168.100.128' end='192.168.100.254' />
</dhcp>
</ip>
</network>
virsh commands used in managing virtual networks are:
virsh net-autostart network-name — Autostart a network specified as network-name.
virsh net-create XMLfile — generates and starts a new network using an existing XML file.
virsh net-define XMLfile — generates a new network device from an existing XML file without starting it.
virsh net-destroy network-name — destroy a network specified as network-name.
virsh net-name networkUUID — convert a specified networkUUID to a network name.
virsh net-uuid network-name — convert a specified network-name to a network UUID.
virsh net-start nameOfInactiveNetwork — starts an inactive network.
virsh net-undefine nameOfInactiveNetwork — removes the definition of an inactive network.
virt-manager) windows, dialog boxes, and various GUI controls.
virt-manager provides a graphical view of hypervisors and guest on your system and on remote machines. You can use virt-manager to define both para-virtualized and fully virtualized guests. virt-manager can perform virtualization management tasks, including:
virt-manager. The UUID field displays the globally unique identifier for the virtual machines.
dom0)'s loopback address (127.0.0.1). This ensures only those with shell privileges on the host can access virt-manager and the virtual machine through VNC.
virt-manager sticky key capability to send these sequences. To use this capability, you must press any modifier key (Ctrl or Alt) 3 times and the key you specify gets treated as active until the next non-modifier key is pressed. You can then send Ctrl-Alt-F11 to the guest by entering the key sequence 'Ctrl Ctrl Ctrl Alt+F1'.
virt-manager, it is not possible to use this sticky key feature to send a Sysrq key combination to a guest.
virt-manager session open the menu, then the menu and select (virt-manager).
virt-manager main window appears.
virt-managervirt-manager can be started remotely using ssh as demonstrated in the following command:
ssh -X host's address[remotehost]# virt-manager
Using ssh to manage virtual machines and hosts is discussed further in Section 22.1, “Remote management with SSH”.
virt-manager's preferences window.
DHCP range
xm command can manage the Xen hypervisor. Most operations can be performed with the libvirt tools, virt-manager application or the virsh command. The xm command does not have the error checking capacity of the libvirt tools and should not be used for tasks the libvirt tools support.
xm command do not work in Red Hat Enterprise Linux 5. The list below provides an overview of command options available and unavailable.
virsh or virt-manager instead of xm. The xm command does not handle error checking or configuration file errors very well and mistakes can lead to system instability or errors in virtual machines. Editing Xen configuration files manually is dangerous and should be avoided. Use this chapter at your own risk.
xm commands:
xm help [--long]: view available options and help text.
xm list command to list active domains:
$ xm list Name ID Mem(MiB) VCPUs State Time(s) Domain-0 0 520 2 r----- 1275.5 r5b2-mySQL01 13 500 1 -b---- 16.1
xm create [-c] DomainName/ID: start a virtual machine. If the -c option is used, the start up process will attach to the guest's console.
xm console DomainName/ID: attach to a virtual machine's console.
xm destroy DomainName/ID: terminates a virtual machine , similar to a power off.
xm reboot DomainName/ID: reboot a virtual machine, runs through the normal system shut down and start up process.
xm shutdown DomainName/ID: shut down a virtual machine, runs a normal system shut down procedure.
xm pause
xm unpause
xm save
xm restore
xm migrate
xm commands to manage resources:
xm mem-set
xm vcpu-list to list virtualized CPU affinities:
$ xm vcpu-list Name ID VCPUs CPU State Time(s) CPU Affinity Domain-0 0 0 0 r-- 708.9 any cpu Domain-0 0 1 1 -b- 572.1 any cpu r5b2-mySQL01 13 0 1 -b- 16.1 any cpu
xm vcpu-pin
xm vcpu-set
xm sched-credit command to display scheduler parameters for a given domain:
$ xm sched-credit -d 0
{'cap': 0, 'weight': 256}
$ xm sched-credit -d 13
{'cap': 25, 'weight': 256}
xm commands for monitoring and troubleshooting:
xm top
xm dmesg
xm info
xm log
xm uptime to display the uptime of guests and hosts:
$ xm uptime Name ID Uptime Domain-0 0 3:42:18 r5b2-mySQL01 13 0:06:27
xm sysrq
xm dump-core
xm rename
xm domid
xm domname
xm vnet-list is currently unsupported.
/boot/grub/grub.conf) creates the list of operating systems the GRUB boot menu interface. When you install the kernel-xen RPM, a script adds the kernel-xen entry to the GRUB configuration file which boots kernel-xen by default. Edit the grub.conf file to modify the default kernel or to add additional kernel parameters.
title Red Hat Enterprise Linux Server (2.6.18-3.el5xen) root (hd0,0) kernel /xen.gz.-2.6.18-3.el5 module /vmlinuz-2.6..18-3.el5xen ro root=/dev/VolGroup00/LogVol00 rhgb quiet module /initrd-2.6.18-3. el5xenxen.img
initrd image, and Linux kernel. Since the kernel entry is on top of the other entries, the kernel loads into memory first. The boot loader sends, and receives, command line arguments to and from the hypervisor and Linux kernel. This example entry shows how you would restrict the Dom0 linux kernel memory to 800 MB.
title Red Hat Enterprise Linux Server (2.6.18-3.el5xen) root (hd0,0) kernel /xen.gz.-2.6.18-3.el5 dom0_mem=800M module /vmlinuz-2.6..18-3.el5xen ro root=/dev/VolGroup00/LogVol00 rhgb quiet module /initrd-2.6.18-3. el5xenxen.img
mem
com1=115200, 8n1
dom0_mem
dom0_max_vcpus
acpi
/* **** Linux config options: propagated to domain0 ****/ /* "acpi=off": Disables both ACPI table parsing and interpreter. */ /* "acpi=force": Overrides the disable blacklist. */ /* "acpi=strict": Disables out-of-spec workarounds. */ /* "acpi=ht": Limits ACPI from boot-time to enable HT. */ /* "acpi=noirq": Disables ACPI interrupt routing. */
noacpi
/etc/elilo.conf, contains a list of global options and image stanzas. When you install the kernel-xen RPM, a post install script adds the appropriate image stanza to the elilo.conf.
image=vmlinuz-2.6.18-92.el5xen
vmm=xen.gz-2.6.18-92.el5
label=linux
initrd=initrd-2.6.18-92.el5xen.img
read-only
root=/dev/VolGroup00/rhel5_2
append="-- rhgb quiet"
image parameter indicates the following lines apply to a single boot selection. This stanza defines a hypervisor (vmm), initrd, and command line arguments (read-only, root and append) to the hypervisor and kernel. When ELILO is loaded during the boot sequence, the image is labeled linux.
read-only to the kernel command line option ro which causes the root file system to be mounted read-only until the initscripts mount the root drive as read-write. ELILO copies the "root" line to the kernel command line. These are merged with the "append" line to build a complete command line:
"-- root=/dev/VolGroup00/rhel5_2 ro rhgb quiet"
-- symbols delimit hypervisor and kernel arguments. The hypervisor arguments come first, then the -- delimiter, followed by the kernel arguments. The hypervisor does not usually have any arguments.
--. An example of the hypervisor memory (mem) parameter and the quiet parameter for the kernel:
append="dom0_mem=2G -- quiet"
| Parameter | Description |
|---|---|
mem= | The mem parameter defines the hypervisor maximum RAM usage. Any additional RAM in the system is ignored. The parameter may be specified with a B, K, M or G suffix; representing bytes, kilobytes, megabytes and gigabytes respectively. If no suffix is specified the default unit is kilobytes. |
dom0_mem= | dom0_mem= sets the amount of RAM to allocate to dom0. The same suffixes are respected as for the mem parameter above. The default in Red Hat Enterprise Linux 5.2 on Itanium® is 4G. |
dom0_max_vcpus= | dom0_max_vcpus= sets the number of CPUs to allocate to the hypervisor. The default in Red Hat Enterprise Linux 5.2 on Itanium® is 4. |
com1= | com1= sets the parameters for the first serial line. For example, com1=9600,8n1,0x408,5. The io_base and irq options can be omitted to leave them as the standard defaults. The baud parameter can be set as auto to indicate the boot loader setting should be preserved. The com1 parameter can be omitted if serial parameters are set as global options in ELILO or in the EFI configuration. |
com2= | Set the parameters for the second serial line. Refer the description of the com1 parameter above. |
console= | The console is a comma delimited preference list for the console options. Options include vga, com1 and com2. This setting should be omitted because the hypervisor attempts to inherit EFI console settings. |
image=vmlinuz-2.6.18-92.el5xen
vmm=xen.gz-2.6.18-92.el5
label=linux
initrd=initrd-2.6.18-92.el5xen.img
read-only
root=/dev/VolGroup00/rhel5_2
append="dom0_mem=2G dom0_max_vcpus=2 --"
rhgb quiet" so that kernel and initscript output are generated on the console. Note the double-dash remains so that the append line is correctly interpreted as hypervisor arguments.
| Item | Description |
|---|---|
pae
|
Specifies the physical address extension configuration data.
|
apic
|
Specifies the advanced programmable interrupt controller configuration data.
|
memory
|
Specifies the memory size in megabytes.
|
vcpus
|
Specifies the numbers of virtual CPUs.
|
console
|
Specifies the port numbers to export the domain consoles to.
|
nic
|
Specifies the number of virtual network interfaces.
|
vif
|
Lists the randomly-assigned MAC addresses and bridges assigned to use for the domain's network addresses.
|
disk
|
Lists the block devices to export to the domain and exports physical devices to domain with read only access.
|
dhcp
|
Enables networking using DHCP.
|
netmask
|
Specifies the configured IP netmasks.
|
gateway
|
Specifies the configured IP gateways.
|
acpi
|
Specifies the advanced configuration power interface configuration data.
|
libvirt configuration files for most tasks. Some users may need Xen configuration files which contain the following standard variables. Configuration items within these files must be enclosed in single quotes('). These configuration files reside in the /etc/xen directory.
xm create --help_config.
|
Parameter
|
Description
|
|---|---|
vncpasswd=NAME
| Password for VNC console on HVM domain. |
vncviewer=no | yes
|
Spawn a vncviewer listening for a vnc server in the domain. The address of the vncviewer is passed to the domain on the kernel command line using VNC_SERVER=<host>:<port>. The port used by vnc is 5500 + DISPLAY. A display value with a free port is chosen if possible. Only valid when vnc=1.
|
vncconsole=no | yes
| Spawn a vncviewer process for the domain's graphical console. Only valid when vnc=1. |
name=NAME
| Domain name. Must be unique. |
bootloader=FILE
| Path to boot loader. |
bootargs=NAME
| Arguments to pass to boot loader. |
bootentry=NAME
|
DEPRECATED. Entry to boot via boot loader. Use bootargs.
|
kernel=FILE
| Path to kernel image. |
ramdisk=FILE
| Path to ramdisk. |
features=FEATURES
| Features to enable in guest kernel |
builder=FUNCTION
| Function to use to build the domain. |
memory=MEMORY
| Domain memory in MB. |
maxmem=MEMORY
| Maximum domain memory in MB. |
shadow_memory=MEMORY
| Domain shadow memory in MB. |
cpu=CPU
| CPU which hosts VCPU0. |
cpus=CPUS
| CPUS to run the domain on. |
pae=PAE
| Disable or enable PAE of HVM domain. |
acpi=ACPI
| Disable or enable ACPI of HVM domain. |
apic=APIC
| Disable or enable APIC of HVM domain. |
vcpus=VCPUs
| The number of Virtual CPUS in domain. |
cpu_weight=WEIGHT
|
Set the new domain's cpu weight. WEIGHT is a float that controls the domain's share of the cpu.
|
restart=onreboot | always | never
|
Deprecated. Use on_poweroff, on_reboot, and on_crash instead. Whether the domain should be restarted on exit. - onreboot: restart on exit with shutdown code reboot - always: always restart on exit, ignore exit code - never: never restart on exit, ignore exit code
|
on_poweroff=destroy | restart | preserve | destroy
|
Behavior when a domain exits with reason 'poweroff'. - destroy: the domain is cleaned up as normal; - restart: a new domain is started in place of the old one; - preserve: no clean-up is done until the domain is manually destroyed (using xm destroy, for example); - rename-restart: the old domain is not cleaned up, but is renamed and a new domain started in its place.
|
on_reboot=destroy | restart | preserve | destroy
| Behavior when a domain exits with reason 'reboot'. - destroy: the domain is cleaned up as normal; - restart: a new domain is started in place of the old one; - preserve: no clean-up is done until the domain is manually destroyed (using xm destroy, for example); - rename-restart: the old domain is not cleaned up, but is renamed and a new domain started in its place. |
on_crash=destroy | restart | preserve | destroy
| Behavior when a domain exits with reason 'crash'. - destroy: the domain is cleaned up as normal; - restart: a new domain is started in place of the old one; - preserve: no clean-up is done until the domain is manually destroyed (using xm destroy, for example); - rename-restart: the old domain is not cleaned up, but is renamed and a new domain started in its place. |
blkif=no | yes
| Make the domain a block device backend. |
netif=no | yes
| Make the domain a network interface backend. |
tpmif=no | yes
| Make the domain a TPM interface backend. |
disk=phy:DEV,VDEV,MODE[,DOM]
|
Add a disk device to a domain. The physical device is DEV, which is exported to the domain as VDEV. The disk is read-only if MODE is r, read-write if MODE is w. If DOM is specified it defines the backend driver domain to use for the disk. The option may be repeated to add more than one disk.
|
pci=BUS:DEV.FUNC
|
Add a PCI device to a domain, using given parameters (in hex). For example pci=c0:02.1a. The option may be repeated to add more than one pci device.
|
ioports=FROM[-TO]
|
Add a legacy I/O range to a domain, using given params (in hex). For example ioports=02f8-02ff. The option may be repeated to add more than one i/o range.
|
irq=IRQ
|
Add an IRQ (interrupt line) to a domain. For example irq=7. This option may be repeated to add more than one IRQ.
|
usbport=PATH
| Add a physical USB port to a domain, as specified by the path to that port. This option may be repeated to add more than one port. |
vfb=type=
vnclisten=
xauthority=
keymap=KEYMAP
|
Make the domain a framebuffer backend. The backend type should be either sdl or vnc. For type=vnc, connect an external vncviewer. The server will listen on ADDR (default 127.0.0.1) on port N+5900. N defaults to the domain id. If vncunused=1, the server will try to find an arbitrary unused port above 5900. For type=sdl, a viewer will be started automatically using the given DISPLAY and XAUTHORITY, which default to the current user's ones.
|
vif=type=
script==NAME
|
Add a network interface with the given MAC address and bridge. The vif is configured by calling the given configuration script. If type is not specified, default is netfront not ioemu device. If mac is not specified a random MAC address is used. If not specified then the network backend chooses its own MAC address. If bridge is not specified the first bridge found is used. If script is not specified the default script is used. If backend is not specified the default backend driver domain is used. If vifname is not specified the backend virtual interface will have name vifD.N where D is the domain id and N is the interface id. This option may be repeated to add more than one vif. Specifying vifs will increase the number of interfaces as needed.
|
vtpm=instance=INSTANCE,backend=DOM
|
Add a TPM interface. On the backend side use the given instance as virtual TPM instance. The given number is merely the preferred instance number. The hotplug script will determine which instance number will actually be assigned to the domain. The association between virtual machine and the TPM instance number can be found in /etc/xen/vtpm.db. Use the backend in the given domain.
|
access_control=policy=POLICY,label=LABEL
| Add a security label and the security policy reference that defines it. The local ssid reference is calculated when starting or resuming the domain. At this time, the policy is checked against the active policy as well. This way, migrating through the save or restore functions are covered and local labels are automatically created correctly on the system where a domain is started or resumed. |
nics=NUM
| DEPRECATED. Use empty vif entries instead. Set the number of network interfaces. Use the vif option to define interface parameters, otherwise defaults are used. Specifying vifs will increase the number of interfaces as needed. |
root=DEVICE
|
Set the root= parameter on the kernel command line. Use a device, e.g. /dev/sda1, or /dev/nfs for NFS root.
|
extra=ARGS
| Set extra arguments to append to the kernel command line. |
ip=IPADDR
| Set the kernel IP interface address. |
gateway=IPADDR
| Set the kernel IP gateway. |
netmask=MASK
| Set the kernel IP netmask. |
hostname=NAME
| Set the kernel IP hostname. |
interface=INTF
| Set the kernel IP interface name. |
dhcp=off|dhcp
| Set the kernel dhcp option. |
nfs_server=IPADDR
| Set the address of the NFS server for NFS root. |
nfs_root=PATH
| Set the path of the root NFS directory. |
device_model=FILE
| Path to device model program. |
fda=FILE
| Path to fda |
fdb=FILE
| Path to fdb |
serial=FILE
| Path to serial or pty or vc |
localtime=no | yes
| Is RTC set to localtime |
keymap=FILE
| Set keyboard layout used |
usb=no | yes
| Emulate USB devices |
usbdevice=NAME
|
Name of a USB device to add
|
stdvga=no | yes
|
Use std vga or Cirrus Logic
graphics
|
isa=no | yes
|
Simulate an ISA only system
|
boot=a|b|c|d
| Default boot device |
nographic=no | yes
| Should device models use graphics? |
soundhw=audiodev
| Should device models enable audio device? |
vnc
| Should the device model use VNC? |
vncdisplay
| VNC display to use |
vnclisten
| Address for VNC server to listen on. |
vncunused
| Try to find an unused port for the VNC server. Only valid when vnc=1. |
sdl
| Should the device model use SDL? |
display=DISPLAY
| X11 display to use |
xauthority=XAUTHORITY
| X11 Authority to use |
uuid
| xenstore UUID (universally unique identifier) to use. One will be randomly generated if this option is not set, just like MAC addresses for virtual network interfaces. This must be a unique value across the entire cluster. |
| Parser function | Valid arguments |
|---|---|
set_bool
|
Accepted values:
|
set_float
|
Accepts a floating point number with Python's float(). For example:
|
set_int
|
Accepts an integer with Python's int().
|
set_value
|
accepts any Python value.
|
append_value
|
accepts any Python value, and appends it to the previous value which is stored in an array.
|
| Parameter | Parser function | Default value |
|---|---|---|
name
|
setter
|
default value
|
vncpasswd
|
set_value
|
None
|
vncviewer
|
set_bool
|
None
|
vncconsole
|
set_bool
|
None
|
name
|
set_value
|
None
|
bootloader
|
set_value
|
None
|
bootargs
|
set_value
|
None
|
bootentry
|
set_value
|
None
|
kernel
|
set_value
|
None
|
ramdisk
|
set_value
|
''
|
features
|
set_value
|
''
|
builder
|
set_value
|
'linux'
|
memory
|
set_int
|
128
|
maxmem
|
set_int
|
None
|
shadow_memory
|
set_int
|
0
|
cpu
|
set_int
|
None
|
cpus
|
set_value
|
None
|
pae
|
set_int
|
0
|
acpi
|
set_int
|
0
|
apic
|
set_int
|
0
|
vcpus
|
set_int
|
1
|
cpu_weight
|
set_float
|
None
|
restart
|
set_value
|
None
|
on_poweroff
|
set_value
|
None
|
on_reboot
|
set_value
|
None
|
on_crash
|
set_value
|
None
|
blkif
|
set_bool
|
0
|
netif
|
set_bool
|
0
|
tpmif
|
append_value
|
0
|
disk
|
append_value
|
[]
|
pci
|
append_value
|
[]
|
ioports
|
append_value
|
[]
|
irq
|
append_value
|
[]
|
usbport
|
append_value
|
[]
|
vfb
|
append_value
|
[]
|
vif
|
append_value
|
[]
|
vtpm
|
append_value
|
[]
|
access_control
|
append_value
|
[]
|
nics
|
set_int
|
-1
|
root
|
set_value
|
''
|
extra
|
set_value
|
''
|
ip
|
set_value
|
''
|
gateway
|
set_value
|
''
|
netmask
|
set_value
|
''
|
hostname
|
set_value
|
''
|
interface
|
set_value
|
"eth0"
|
dhcp
|
set_value
|
'off'
|
nfs_server
|
set_value
|
None
|
nfs_root
|
set_value
|
None
|
device_model
|
set_value
|
''
|
fda
|
set_value
|
''
|
fdb
|
set_value
|
''
|
serial
|
set_value
|
''
|
localtime
|
set_bool
|
0
|
keymap
|
set_value
|
''
|
usb
|
set_bool
|
0
|
usbdevice
|
set_value
|
''
|
stdvga
|
set_bool
|
0
|
isa
|
set_bool
|
0
|
boot
|
set_value
|
'c'
|
nographic
|
set_bool
|
0
|
soundhw
|
set_value
|
''
|
vnc
|
set_value
|
None
|
vncdisplay
|
set_value
|
None
|
vnclisten
|
set_value
|
None
|
vncunused
|
set_bool
|
1
|
sdl
|
set_value
|
None
|
display
|
set_value
|
None
|
xauthority
|
set_value
|
None
|
uuid
|
set_value
|
None
|
Table of Contents
ftpd ftpd virsh to set a guest, TestServer, to automatically start when the host boots.
# virsh autostart TestServer
Domain TestServer marked as autostarted
--disable parameter
# virsh autostart --disable TestServer
Domain TestServer unmarked as autostarted
Install the KVM package
# yum install kvm
Verify which kernel is in use
uname command to determine which kernel is running:
$ uname -r 2.6.18-159.el5xen
2.6.18-159.el5xen", is running on the system. If the default kernel, "2.6.18-159.el5", is running you can skip the substep.
Changing the Xen kernel to the default kernel
grub.conf file determines which kernel is booted. To change the default kernel edit the /boot/grub/grub.conf file as shown below.
default=1
timeout=5
splashimage=(hd0,0)/grub/splash.xpm.gz
hiddenmenu
title Red Hat Enterprise Linux Server (2.6.18-159.el5)
root (hd0,0)
kernel /vmlinuz-2.6.18-159.el5 ro root=/dev/VolGroup00/LogVol00 rhgb quiet
initrd /initrd-2.6.18-159.el5.img
title Red Hat Enterprise Linux Server (2.6.18-159.el5xen)
root (hd0,0)
kernel /xen.gz-2.6.18-159.el5
module /vmlinuz-2.6.18-159.el5xen ro root=/dev/VolGroup00/LogVol00 rhgb quiet
module /initrd-2.6.18-159.el5xen.img
0 (or the number for the default kernel):
default=0
timeout=5
splashimage=(hd0,0)/grub/splash.xpm.gz
hiddenmenu
title Red Hat Enterprise Linux Server (2.6.18-159.el5)
root (hd0,0)
kernel /vmlinuz-2.6.18-159.el5 ro root=/dev/VolGroup00/LogVol00 rhgb quiet
initrd /initrd-2.6.18-159.el5.img
title Red Hat Enterprise Linux Server (2.6.18-159.el5xen)
root (hd0,0)
kernel /xen.gz-2.6.18-159.el5
module /vmlinuz-2.6.18-159.el5xen ro root=/dev/VolGroup00/LogVol00 rhgb quiet
module /initrd-2.6.18-159.el5xen.img
Reboot to load the new kernel
$ lsmod | grep kvm kvm_intel 85992 1 kvm 222368 2 ksm,kvm_intel
kvm module and either the kvm_intel module or the kvm_amd module are present if everything worked.
Install the Xen packages
# yum install kernel-xen xen
Verify which kernel is in use
uname command to determine which kernel is running.
$ uname -r 2.6.18-159.el5
2.6.18-159.el5", is running on the system. This is the default kernel. If the kernel has xen on the end (for example, 2.6.18-159.el5xen) then the Xen kernel is running and you can skip the substep.
Changing the default kernel to the Xen kernel
grub.conf file determines which kernel is booted. To change the default kernel edit the /boot/grub/grub.conf file as shown below.
default=0
timeout=5
splashimage=(hd0,0)/grub/splash.xpm.gz
hiddenmenu
title Red Hat Enterprise Linux Server (2.6.18-159.el5)
root (hd0,0)
kernel /vmlinuz-2.6.18-159.el5 ro root=/dev/VolGroup00/LogVol00 rhgb quiet
initrd /initrd-2.6.18-159.el5.img
title Red Hat Enterprise Linux Server (2.6.18-159.el5xen)
root (hd0,0)
kernel /xen.gz-2.6.18-159.el5
module /vmlinuz-2.6.18-159.el5xen ro root=/dev/VolGroup00/LogVol00 rhgb quiet
module /initrd-2.6.18-159.el5xen.img
1 (or the number for the Xen kernel):
default=1
timeout=5
splashimage=(hd0,0)/grub/splash.xpm.gz
hiddenmenu
title Red Hat Enterprise Linux Server (2.6.18-159.el5)
root (hd0,0)
kernel /vmlinuz-2.6.18-159.el5 ro root=/dev/VolGroup00/LogVol00 rhgb quiet
initrd /initrd-2.6.18-159.el5.img
title Red Hat Enterprise Linux Server (2.6.18-159.el5xen)
root (hd0,0)
kernel /xen.gz-2.6.18-159.el5
module /vmlinuz-2.6.18-159.el5xen ro root=/dev/VolGroup00/LogVol00 rhgb quiet
module /initrd-2.6.18-159.el5xen.img
Reboot to load the new kernel
uname command:
$ uname -r 2.6.18-159.el5xen
xen on the end the Xen kernel is running.
qemu-img command line tool is used for formatting various file systems used by Xen and KVM. qemu-img should be used for formatting virtualized guest images, additional storage devices and network storage. qemu-img options and usages are listed below.
size and format format.
# qemu-img create [-6] [-e] [-b base_image] [-f format] filename [size]
convert option is used for converting a recognized format to another image format.
# qemu-img convert [-c] [-e] [-f format]filename[-Ooutput_format]output_filename
filename to disk image output_filename using format output_format. The disk image can be optionally encrypted with the -e option or compressed with the -c option.
qcow" supports encryption or compression. the compression is read-only. it means that if a compressed sector is rewritten, then it is rewritten as uncompressed data.
qcow or cow. The empty sectors are detected and suppressed from the destination image.
info parameter displays information about a disk image. the format for the info option is as follows:
# qemu-img info [-f format] filename
raw qemu-img info to know the real size used by the image or ls -ls on Unix/Linux.
qcow2 qcow cow cow format is included only for compatibility with previous versions. It does not work with Windows.
vmdk cloop (0.5 * RAM) + (overcommit ratio * RAM) = Recommended swap size
/etc/grub.conf file to use the virtualization kernel. You must use the xen kernel to use the Xen hypervisor. Copy your existing xen kernel entry make sure you copy all of the important lines or your system will panic upon boot (initrd will have a length of '0'). If you require xen hypervisor specific values you must append them to the xen line of your grub entry.
grub.conf entry from a system running the kernel-xen package. The grub.conf on your system may vary. The important part in the example below is the section from the title line to the next new line.
#boot=/dev/sda default=0 timeout=15 #splashimage=(hd0,0)/grub/splash.xpm.gz hiddenmenu serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1 terminal --timeout=10 serial console title Red Hat Enterprise Linux Server (2.6.17-1.2519.4.21.el5xen) root (hd0,0) kernel /xen.gz-2.6.17-1.2519.4.21.el5 com1=115200,8n1 module /vmlinuz-2.6.17-1.2519.4.21.el5xen ro root=/dev/VolGroup00/LogVol00 module /initrd-2.6.17-1.2519.4.21.el5xen.img
grub.conf...grub.conf could look very different if it has been manually edited before or copied from an example. Read Chapter 28, Configuring the Xen kernel boot parameters for more information on using virtualization and grub.
dom0_mem=256M to the xen line in your grub.conf. A modified version of the grub configuration file in the previous example:
#boot=/dev/sda default=0 timeout=15 #splashimage=(hd0,0)/grub/splash.xpm.gz hiddenmenu serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1 terminal --timeout=10 serial console title Red Hat Enterprise Linux Server (2.6.17-1.2519.4.21.el5xen) root (hd0,0) kernel /xen.gz-2.6.17-1.2519.4.21.el5 com1=115200,8n1 dom0_mem=256MB module /vmlinuz-2.6.17-1.2519.4.21.el5xen ro root=/dev/VolGroup00/LogVol00 module /initrd-2.6.17-1.2519.4.21.el5xen.img
$ grep -E 'svm|vmx' /proc/cpuinfo
vmx entry indicating an Intel processor with the Intel VT extensions:
flags : fpu tsc msr pae mce cx8 apic mtrr mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall lm constant_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr lahf_lm
svm entry indicating an AMD processor with the AMD-V extensions:
flags : fpu tsc msr pae mce cx8 apic mtrr mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt lm 3dnowext 3dnow pni cx16 lahf_lm cmp_legacy svm cr8legacy ts fid vid ttp tm stc
flags:" content may appear multiple times for each hyperthread, core or CPU on the system.
For users of the KVM hypervisor
kvm modules are loaded in the kernel:
# lsmod | grep kvm
kvm_intel or kvm_amd then the kvm hardware virtualization modules are loaded and your system meets requirements. sudo
virsh command can output a full list of virtualization system capabilities. Run virsh capabilities as root to receive the complete list.
kpartx tool, covered by this section, to mount the guest file system as a loop device which can then be accessed.
kpartx command creates device maps from partition tables. Each guest storage image has a partition table embedded in the file.
# yum install kpartx
guest1.img.
# kpartx -l /var/lib/libvirt/images/guest1.img
loop0p1 : 0 409600 /dev/loop0 63
loop0p2 : 0 10064717 /dev/loop0 409663guest1 is a Linux guest. The first partition is the boot partition and the second partition is an EXT3 containing the root partition.
/dev/mapper/.
# kpartx -a /var/lib/libvirt/images/guest1.img
# ls /dev/mapper/ loop0p1 loop0p2
loopXpY.
/mnt/guest1 for mounting the partition.
# mkdir /mnt/guest1
# mount /dev/mapper/loop0p1 /mnt/guest1 -o loop,ro/mnt/guest1 directory. Read or copy the files.
# umount /mnt/tmp
# kpartx -d /var/lib/libvirt/images/guest1.img
guest1.img to the recognized devices in the /dev/mapper/ directory.
# kpartx -a /var/lib/libvirt/images/guest1.img
vgscan command to find the new volume groups.
# vgscan Reading all physical volumes . This may take a while... Found volume group "VolGroup00" using metadata type lvm2
VolGroup00 by default) with the vgchange -ay command.
# vgchange -ay VolGroup00 2 logical volumes in volume group VolGroup00 now active.
lvs command to display information about the new volumes. The volume names (the LV column) are required to mount the volumes.
# lvs LV VG Attr Lsize Origin Snap% Move Log Copy% LogVol00 VolGroup00 -wi-a- 5.06G LogVol01 VolGroup00 -wi-a- 800.00M
/dev/VolGroup00/LogVol00 in the /mnt/guestboot/ directory.
# mount /dev/VolGroup00/LogVol00 /mnt/guestboot
/mnt/guestboot directory. Read or copy the files.
# umount /mnt/
VolGroup00
# vgchange -an VolGroup00
# kpartx -d /var/lib/libvirt/images/guest1.img
virsh nodeinfo command provides information about how many sockets, cores and hyperthreads there are attached a host.
# virsh nodeinfo CPU model: x86_64 CPU(s): 8 CPU frequency: 1000 MHz CPU socket(s): 2 Core(s) per socket: 4 Thread(s) per core: 1 NUMA cell(s): 1 Memory size: 8179176 kB
virsh capabilities to get additional output data on the CPU configuration.
# virsh capabilities
<capabilities>
<host>
<cpu>
<arch>x86_64</arch>
</cpu>
<migration_features>
<live/>
<uri_transports>
<uri_transport>tcp</uri_transport>
</uri_transports>
</migration_features>
<topology>
<cells num='2'>
<cell id='0'>
<cpus num='4'>
<cpu id='0'/>
<cpu id='1'/>
<cpu id='2'/>
<cpu id='3'/>
</cpus>
</cell>
<cell id='1'>
<cpus num='4'>
<cpu id='4'/>
<cpu id='5'/>
<cpu id='6'/>
<cpu id='7'/>
</cpus>
</cell>
</cells>
</topology>
<secmodel>
<model>selinux</model>
<doi>0</doi>
</secmodel>
</host>
[ Additional XML removed ]
</capabilities>virsh freecell command to display the free memory on all NUMA nodes.
# virsh freecell 0: 2203620 kB 1: 3354784 kB
virsh capabilities command) about NUMA topology.
virsh capabilities output.
<topology>
<cells num='2'>
<cell id='0'>
<cpus num='4'>
<cpu id='0'/>
<cpu id='1'/>
<cpu id='2'/>
<cpu id='3'/>
</cpus>
</cell>
<cell id='1'>
<cpus num='4'>
<cpu id='4'/>
<cpu id='5'/>
<cpu id='6'/>
<cpu id='7'/>
</cpus>
</cell>
</cells>
</topology><cell id='1'>, has physical CPUs 4 to 7.
cpuset attribute to the configuration file.
virsh edit.
vcpus element.
<vcpus>4</vcpus>
cpuset attribute with the CPU numbers for the relevant NUMA cell.
<vcpus cpuset='4-7'>4</vcpus>
virt-install provisioning tool provides a simple way to automatically apply a 'best fit' NUMA policy when guests are created.
cpuset option for virt-install can use a CPU set of processors or the parameter auto. The auto parameter automatically determines the optimal CPU locking using the available NUMA data.
--cpuset=auto with the virt-install command when creating new guests.
virsh vcpuinfo and virsh vcpupin commands can perform CPU affinity changes on running guests.
virsh vcpuinfo command gives up to date information about where each virtual CPU is running.
guest1 is a guest with four virtual CPUs is running on a KVM host.
# virsh vcpuinfo guest1
VCPU: 0
CPU: 3
State: running
CPU time: 0.5s
CPU Affinity: yyyyyyyy
VCPU: 1
CPU: 1
State: running
CPU Affinity: yyyyyyyy
VCPU: 2
CPU: 1
State: running
CPU Affinity: yyyyyyyy
VCPU: 3
CPU: 2
State: running
CPU Affinity: yyyyyyyyvirsh vcpuinfo output (the yyyyyyyy value of CPU Affinity) shows that the guest can presently run on any CPU.
# virsh vcpupinguest10 4 # virsh vcpupinguest11 5 # virsh vcpupinguest12 6 # virsh vcpupinguest13 7
virsh vcpuinfo command confirms the change in affinity.
# virsh vcpuinfo guest1
VCPU: 0
CPU: 4
State: running
CPU time: 32.2s
CPU Affinity: ----y---
VCPU: 1
CPU: 5
State: running
CPU time: 16.9s
CPU Affinity: -----y--
VCPU: 2
CPU: 6
State: running
CPU time: 11.9s
CPU Affinity: ------y-
VCPU: 3
CPU: 7
State: running
CPU time: 14.6s
CPU Affinity: -------ymacgen.py. Now from that directory you can run the script using ./macgen.py and it will generate a new MAC address. A sample output would look like the following:
$ ./macgen.py 00:16:3e:20:b0:11 #!/usr/bin/python # macgen.py script to generate a MAC address for virtualized guests on Xen # import random # def randomMAC(): mac = [ 0x00, 0x16, 0x3e, random.randint(0x00, 0x7f), random.randint(0x00, 0xff), random.randint(0x00, 0xff) ] return ':'.join(map(lambda x: "%02x" % x, mac)) # print randomMAC()
python-virtinst to generate a new MAC address and UUID for use in a guest configuration file:
# echo 'import virtinst.util ; print\ virtinst.util.uuidToString(virtinst.util.randomUUID())' | python # echo 'import virtinst.util ; print virtinst.util.randomMAC()' | python
#!/usr/bin/env python # -*- mode: python; -*- print "" print "New UUID:" import virtinst.util ; print virtinst.util.uuidToString(virtinst.util.randomUUID()) print "New MAC:" import virtinst.util ; print virtinst.util.randomMAC() print ""
rate” parameter part of the VIF entries can throttle virtualized guests.
rate rate= option can be added to the VIF= entry in a virtual machine configuration file to limit a virtual machine's network bandwidth or specify a specific time interval for a time window.
rate= option:
rate parameter values and uses.
rate=10Mb/s rate=250KB/s rate=10MB/s@50ms VIF entry would look like the following:
vif = [ 'rate=10MB/s , mac=00:16:3e:7a:55:1c, bridge=xenbr1']
rate entry would limit the virtual machine's interface to 10MB/s for outgoing traffic
virsh or virt-manager:
virsh refer to Configuring virtual CPU affinity for more information.
virt-manager refer to Section 26.11, “Displaying virtual CPUs ” for more information.
/boot/grub/grub.conf. Managing several or more hosts configuration files quickly becomes difficult. System administrators often prefer to use the 'cut and paste' method for editing multiple grub.conf files. If you do this, ensure you include all five lines in the Virtualization entry (or this will create system errors). Hypervisor specific values are all found on the 'xen' line. This example represents a correct grub.conf virtualization entry:
# boot=/dev/sda/
default=0
timeout=15
#splashimage=(hd0, 0)/grub/splash.xpm.gz
hiddenmenu
serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1
terminal --timeout=10 serial console
title Red Hat Enterprise Linux Server (2.6.17-1.2519.4.21. el5xen)
root (hd0, 0)
kernel /xen.gz-2.6.17-1.2519.4.21.el5 com1=115200,8n1
module /vmlinuz-2.6.17-1.2519.4.21el5xen ro root=/dev/VolGroup00/LogVol00
module /initrd-2.6.17-1.2519.4.21.el5xen.img
'dom0_mem=256M'. This example is the grub.conf with the hypervisor's memory entry modified.
# boot=/dev/sda
default=0
timeout=15
#splashimage=(hd0,0)/grubs/splash.xpm.gz
hiddenmenu
serial --unit=0 --speed =115200 --word=8 --parity=no --stop=1
terminal --timeout=10 serial console
title Red Hat Enterprise Linux Server (2.6.17-1.2519.4.21. el5xen)
root (hd0,0)
kernel /xen.gz-2.6.17-1.2519.4.21.el5 com1=115200,8n1 dom0_mem=256MB
module /vmlinuz-2.6.17-1.2519.4.21.el5xen ro root=/dev/VolGroup00/LogVol00
module /initrd-2.6.17-1.2519.4.21.el5xen.img
ftpd vsftpd can provide access to installation trees for para-virtualized guests (for example, the Red Hat Enterprise Linux 5 repositories) or other data. If you have not installed vsftpd during the server installation you can grab the RPM package from your Server directory of your installation media and install it using the rpm -ivh vsftpd*.rpm (note that the RPM package must be in your current directory).
vsftpd, edit /etc/passwd using vipw and change the ftp user's home directory to the directory where you are going to keep the installation trees for your para-virtualized guests. An example entry for the FTP user would look like the following:
ftp:x:14:50:FTP User:/xen/pub:/sbin/nologin
vsftpd is not enabled using the chkconfig --list vsftpd:
$ chkconfig --list vsftpd vsftpd 0:off 1:off 2:off 3:off 4:off 5:off 6:off
chkconfig --levels 345 vsftpd on to start vsftpd automatically for run levels 3, 4 and 5.
chkconfig --list vsftpd command to verify the vsftpd daemon is enabled to start during system boot:
$ chkconfig --list vsftpd vsftpd 0:off 1:off 2:off 3:on 4:on 5:on 6:off
service vsftpd start vsftpd to start the vsftpd service:
$service vsftpd start vsftpd Starting vsftpd for vsftpd: [ OK ]
udev to implement LUN persistence. Before implementing LUN persistence in your system, ensure that you acquire the proper UUIDs. Once you acquire these, you can configure LUN persistence by editing the scsi_id file that resides in the /etc directory. Once you have this file open in a text editor, you must comment out this line:
# options=-b
# options=-g
scsi_id command:
# scsi_id -g -s /block/sdc *3600a0b80001327510000015427b625e*
20-names.rules file in the /etc/udev/rules.d directory. The device naming rules follow this format:
# KERNEL="sd*", BUS="scsi", PROGRAM="sbin/scsi_id", RESULT="UUID", NAME="devicename"
UUID and devicename with the above UUID retrieved entry. The rule should resemble the following:
KERNEL="sd*", BUS="scsi", PROGRAM="sbin/scsi_id", RESULT="3600a0b80001327510000015427b625e", NAME="mydevicename"
/dev/sd* pattern to inspect the given UUID. When it finds a matching device, it creates a device node called /dev/devicename. For this example, the device node is /dev/mydevice . Finally, append the /etc/rc.local file with this line:
/sbin/start_udev
multipath.conf file that resides in the /etc/ directory:
multipath {
wwid 3600a0b80001327510000015427b625e
alias oramp1
}
multipath {
wwid 3600a0b80001327510000015427b6
alias oramp2
}
multipath {
wwid 3600a0b80001327510000015427b625e
alias oramp3
}
multipath {
wwid 3600a0b80001327510000015427b625e
alias oramp4
}
/dev/mpath/oramp1, /dev/mpath/oramp2, /dev/mpath/oramp3, and dev/mpath/oramp4. The devices will reside in the /dev/mpath directory. These LUN names are persistent after reboots as it creates aliased names on the wwid for each of the LUNs.
/sbin/service smartd stop /sbin/chkconfig --del smartd
/var/lib/xen, the usually named vmlinuz.****** and initrd.******. These files are the initrd and vmlinuz files from virtual machines which either failed to boot or failed for some other reason. These files are temporary files extracted from virtual machine's boot disk during the start up sequence. These files should be automatically removed after the virtual machine is shut down cleanly. Then you can safely delete old and stale copies from this directory.
vino-preferences command.
~/.vnc/xstartup file to start a GNOME session whenever vncserver is started. The first time you run the vncserver script it will ask you for a password you want to use for your VNC session.
xstartup file:
#!/bin/sh # Uncomment the following two lines for normal desktop: # unset SESSION_MANAGER # exec /etc/X11/xinit/xinitrc [ -x /etc/vnc/xstartup ] && exec /etc/vnc/xstartup [ -r $HOME/.Xresources ] && xrdb $HOME/.Xresources #xsetroot -solid grey #vncconfig -iconic & #xterm -geometry 80x24+10+10 -ls -title "$VNCDESKTOP Desktop" & #twm & if test -z "$DBUS_SESSION_BUS_ADDRESS" ; then eval `dbus-launch --sh-syntax –exit-with-session` echo "D-BUS per-session daemon address is: \ $DBUS_SESSION_BUS_ADDRESS" fi exec gnome-session
uuidgen command. Then for the vif entries you must define a unique MAC address for each guest (if you are copying a guest configuration from an existing guest, you can create a script to handle it). For the xen bridge information, if you move an existing guest configuration file to a new host, you must update the xenbr entry to match your local networking configuration. For the Device entries, you must modify the entries in the 'disk=' section to point to the correct guest image.
/etc/sysconfig/network file to match the new guest's hostname.
HWADDR address of the /etc/sysconfig/network-scripts/ifcfg-eth0 file to match the output from ifconfig eth0 file and if you use static IP addresses, you must modify the IPADDR entry.
name uuid uuidgen command. A sample UUID output:
$ uuidgen a984a14f-4191-4d14-868e-329906b211e5
vif xenbr entry to correspond with your local networking configuration (you can obtain the bridge information using the command brctl show command).
disk= section to point to the correct guest image.
/etc/sysconfig/network HOSTNAME entry to the guest's new hostname.
/etc/sysconfig/network-scripts/ifcfg-eth0 HWADDR address to the output from ifconfig eth0
IPADDR entry if a static IP address is used.
libvirt.
libvirt.
virsh can handle XML configuration files. You may want to use this to your advantage for scripting large deployments with special options. You can add devices defined in an XML file to a running para-virtualized guest. For example, to add a ISO file as hdc to a running guest create an XML file:
# cat satelliteiso.xml <disk type="file" device="disk"> <driver name="file"/> <source file="/var/lib/libvirt/images/rhn-satellite-5.0.1-11-redhat-linux-as-i386-4-embedded-oracle.iso"/> <target dev="hdc"/> <readonly/> </disk>Run
virsh attach-device to attach the ISO as hdc to a guest called "satellite" :
# virsh attach-device satellite satelliteiso.xml
Table of Contents
xentop xentop displays real-time information about a host system and the guest domains.
xm dmesg and log
vmstat
iostat
lsof
iostat, mpstat and sar commands are all provided by the sysstat package.
XenOprofile
systemtap
crash
sysrq
sysrq t
sysrq w
ifconfig
tcpdump
tcpdump command 'sniffs' network packets. tcpdump is useful for finding network abnormalities and problems with network authentication. There is a graphical version of tcpdump named wireshark.
brctl
brctl is a networking tool that inspects and configures the Ethernet bridge configuration in the Virtualization linux kernel. You must have root access before performing these example commands:
# brctl show bridge-name bridge-id STP enabled interfaces ----------------------------------------------------------------------------- xenbr0 8000.feffffff no vif13.0 xenbr1 8000.ffffefff yes pddummy0 xenbr2 8000.ffffffef no vif0.0 # brctl showmacs xenbr0 port-no mac-addr local? aging timer 1 fe:ff:ff:ff:ff: yes 0.00 2 fe:ff:ff:fe:ff: yes 0.00 # brctl showstp xenbr0 xenbr0 bridge-id 8000.fefffffffff designated-root 8000.fefffffffff root-port 0 path-cost 0 max-age 20.00 bridge-max-age 20.00 hello-time 2.00 bridge-hello-time 2.00 forward-delay 0.00 bridge-forward-delay 0.00 aging-time 300.01 hello-timer 1.43 tcn-timer 0.00 topology-change-timer 0.00 gc-timer 0.02
Server repositories Red Hat Enterprise Linux 5.
yum install vnc command.
yum install vnc-server command.
/etc/xen/. This directory contains the xend daemon and other virtual machine configuration files. The networking script files are found in the scripts directory.
/var/log/xen directory.
/var/lib/libvirt/images directory.
/proc/xen/ directory.
xend daemon and qemu-dm process, two utilities that write the multiple log files to the /var/log/xen/ directory:
xend.log is the log file that contains all the data collected by the xend daemon, whether it is a normal system event, or an operator initiated action. All virtual machine operations (such as create, shutdown, destroy and so on) appear in this log. The xend.log is usually the first place to look when you track down event or performance problems. It contains detailed entries and conditions of the error messages.
xend-debug.log is the log file that contains records of event errors from xend and the Virtualization subsystems (such as framebuffer, Python scripts, and so on).
xen-hotplug-log is the log file that contains data from hotplug events. If a device or a network script does not come online, the event appears here.
qemu-dm.[PID].log is the log file created by the qemu-dm process for each fully virtualized guest. When using this log file, you must retrieve the given qemu-dm process PID, by using the ps command to examine process arguments to isolate the qemu-dm process on the virtual machine. Note that you must replace the [PID] symbol with the actual PID qemu-dm process.
virt-manager.log file that resides in the /.virt-manager directory. Note that every time you start the Virtual Machine Manager, it overwrites the existing log file contents. Make sure to backup the virt-manager.log file, before you restart the Virtual Machine manager after a system error.
/var/lib/libvirt/images directory.
xend daemon, it updates the xend-database that resides in the /var/lib/xen/xend-db directory.
xm dump-core command) resides in the /var/lib/xen/dumps directory.
/etc/xen directory contains the configuration files that you use to manage system resources. The xend daemon configuration file is /etc/xen/xend-config.sxp. This file can be edited to implement system-wide changes and configure the networking. However, manually editing files in the /etc/xen/ folder is not advised.
proc folders are another resource that allows you to gather system information. These proc entries reside in the /proc/xen directory:
/proc/xen/capabilities
/proc/xen/balloon
/proc/xen/xenbus/
xend.log file contains the same basic information as when you run the xm log command. This log is found in the /var/log/ directory. Here is an example log entry for when you create a domain running a kernel:
[2006-12-27 02:23:02 xend] ERROR (SrvBase: 163) op=create: Error creating domain: (0, 'Error')
Traceback (most recent call list)
File "/usr/lib/python2.4/site-packages/xen/xend/server/SrvBase.py" line 107 in_perform val = op_method (op,req)
File
"/usr/lib/python2.4/site-packages/xen/xend/server/SrvDomainDir.py line 71 in op_create
raise XendError ("Error creating domain: " + str(ex))
XendError: Error creating domain: (0, 'Error')
xend-debug.log, is very useful to system administrators since it contains even more detailed information than xend.log . Here is the same error data for the same kernel domain creation problem:
ERROR: Will only load images built for Xen v3.0 ERROR: Actually saw: GUEST_OS=netbsd, GUEST_VER=2.0, XEN_VER=2.0; LOADER=generic, BSD_SYMTAB' ERROR: Error constructing guest OS
grub.conf file to enable a 38400-bps serial console on com1 /dev/ttyS0:
title Red Hat Enterprise Linux (2.6.18-8.2080_xen0) root (hd0,2) kernel /xen.gz-2.6.18-8.el5 com1=38400,8n1 module /vmlinuz-2.618-8.el5xen ro root=LABEL=/rhgb quiet console=xvc console=tty xencons=xvc module /initrd-2.6.18-8.el5xen.img
sync_console can help determine a problem that causes hangs with asynchronous hypervisor console output, and the "pnpacpi=off" works around a problem that breaks input on the serial console. The parameters "console=ttyS0" and "console=tty" means that kernel errors get logged with on both the normal VGA console and on the serial console. Then you can install and set up ttywatch to capture the data on a remote host connected by a standard null-modem cable. For example, on the remote host you could type:
ttywatch --name myhost --port /dev/ttyS0
/dev/ttyS0 into the file /var/log/ttywatch/myhost.log .
# virsh console [guest name, ID or UUID]
virt-manager to display the virtual text console. In the guest console window, select Serial Console from the View menu.
grub.conf file, and include the 'console =ttyS0 console=tty0' parameter. This ensures that the kernel messages are sent to the virtual serial console (and the normal graphical console). To use the guest's serial console, you must edit the libvirt configuration file configuration file. On the host, access the serial console with the following command:
# virsh console
virt-manager to display the virtual text console. In the guest console window, select Serial Console from the View menu.
xend service, nothing happens. Type virsh list and receive the following:
Error: Error connecting to xend: Connection refused. Is xend running?
xend start manually and receive more errors:
Error: Could not obtain handle on privileged command interfaces (2 = No such file or directory) Traceback (most recent call last:) File "/usr/sbin/xend/", line 33 in ? from xen.xend.server. import SrvDaemon File "/usr/lib/python2.4/site-packages/xen/xend/server/SrvDaemon.py" , line 26 in ? from xen.xend import XendDomain File "/usr//lib/python2.4/site-packages/xen/xend/XendDomain.py" , line 33, in ? from xen.xend import XendDomainInfo File "/usr/lib/python2.4/site-packages/xen/xend/image.py" , line37, in ? import images File "/usr/lib/python2.4/site-packages/xen/xend/image.py" , line30, in ? xc = xen.lowlevel.xc.xc () RuntimeError: (2, 'No such file or directory' )
kernel-xen kernel. To correct this, you must select the kernel-xen kernel at boot time (or set the kernel-xen kernel to the default in the grub.conf file).
"Invalid argument" error message. This usually means that the kernel image you are trying to boot is incompatible with the hypervisor. An example of this would be if you were attempting to run a non-PAE FC5 kernel on a PAE only FC6 hypervisor.
grub.conf default kernel switches right back to a bare-metal kernel instead of the Virtualization kernel.
/etc/sysconfig/kernel/ directory. You must ensure that kernel-xen parameter is set as the default option in your grub.conf file.
/boot/grub/grub.conf file by setting the appropriate serial device parameters.
com1, modify /boot/grub/grub.conf by inserting the lines com1=115200,8n1, console=tty0 and console=ttyS0,115200 where shown.
title Red Hat Enterprise Linux 5 i386 Xen (2.6.18-92.el5xen) root (hd0, 8) kernel /boot/xen.gz-2.6.18-92.el5com1=115200,8n1module /boot/vmlinuz-2.6.18-92.el5xen ro root=LABEL=VG_i386console=tty0console=ttyS0,115200module /boot/initrd-2.6.18-92.el5xen.img
com2, modify /boot/grub/grub.conf by inserting the lines com2=115200,8n1 console=com2L, console=tty0 and console=ttyS0,115200 where shown.
title Red Hat Enterprise Linux 5 i386 Xen (2.6.18-92.el5xen) root (hd0, 8) kernel /boot/xen.gz-2.6.18-92.el5 com2=115200,8n1 console=com2L module /boot/vmlinuz-2.6.18-92.el5xen ro root=LABEL=VG_i386 console=tty0 console=ttyS0,115200 module /boot/initrd-2.6.18-92.el5xen.img
com1, com2 and so on) you selected in the previous step.
com2 port, the parameter console=ttyS0 on the vmlinuz line us used. The behavior of every port being used as console=ttyS0 is not standard Linux behavior and is specific to the Xen environment.
virsh console" or in the "Serial Console" window of virt-manager. Set up the virtual serial console using this procedure:
/boot/grub/grub.conf as follows:
Red Hat Enterprise Linux 5 i386 Xen (2.6.18-92.el5xen) root (hd0, 0) kernel /boot/vmlinuz-2.6.18-92.el5xen ro root=LABEL=VG_i386 console=xvc0 initrd /boot/initrd-2.6.18-92.el5xen.img
xend) can be configured to log the output from serial consoles of para-virtualized guests.
xend edit /etc/sysconfig/xend. Change the entry:
# Log all guest console output (cf xm console) #XENCONSOLED_LOG_GUESTS=no
# Log all guest console output (cf xm console) XENCONSOLED_LOG_GUESTS=yes
/var/log/xen/console file.
virsh console" command.
ttyS0 on Linux or COM1 on Windows.
/boot/grub/grub.conf file by inserting the line "console=tty0 console=ttys0,115200".
title Red Hat Enterprise Linux Server (2.6.18-92.el5)
root (hd0,0)
kernel /vmlinuz-2.6.18-92.el5 ro root=/dev/volgroup00/logvol00
console=tty0 console=ttys0,115200
initrd /initrd-2.6.18-92.el5.img
virsh console" command.
/var/log/xen/console as they are for para-virtualized guests.
virt-manager or virt-install tools on Red Hat Enterprise Linux 5, the guests configuration files are created automatically in the /etc/xen directory.
name = "rhel5vm01" memory = "2048" disk = ['tap:aio:/var/lib/libvirt/images/rhel5vm01.dsk,xvda,w',] vif = ["type=ieomu, mac=00:16:3e:09:f0:12 bridge=xenbr0', "type=ieomu, mac=00:16:3e:09:f0:13 ] vnc = 1 vncunused = 1 uuid = "302bd9ce-4f60-fc67-9e40-7a77d9b4e1ed" bootloader = "/usr/bin/pygrub" vcpus=2 on_reboot = "restart" on_crash = "restart"
serial="pty" is the default for the configuration file. This configuration file example is for a fully-virtualized guest:
name = "rhel5u5-86_64" builder = "hvm" memory = 500 disk = ['/var/lib/libvirt/images/rhel5u5-x86_64.dsk.hda,w'] vif = [ 'type=ioemu, mac=00:16:3e:09:f0:12, bridge=xenbr0', 'type=ieomu, mac=00:16:3e:09:f0:13, bridge=xenbr1'] uuid = "b10372f9-91d7-ao5f-12ff-372100c99af5' device_model = "/usr/lib64/xen/bin/qemu-dm" kernel = "/usr/lib/xen/boot/hvmloader/" vnc = 1 vncunused = 1 apic = 1 acpi = 1 pae = 1 vcpus =1 serial ="pty" # enable serial console on_boot = 'restart'
virsh dumpxml and virsh create (or virsh edit) to edit the libvirt configuration files (xml based) which have error checking and safety checks.
failed domain creation due to memory shortage, unable to balloon domain0
xend.log file for this error:
[2006-12-21] 20:33:31 xend 3198] DEBUG (balloon:133) Balloon: 558432 Kib free; 0 to scrub; need 1048576; retries: 20 [2006-12-21] 20:33:31 xend. XendDomainInfo 3198] ERROR (XendDomainInfo: 202 Domain construction failed
xm list domain0 command. If dom0 is not ballooned down, you can use the command virsh setmem dom0 NewMemSize to check memory.
wrong kernel image: non-PAE kernel on a PAE
# xm create -c va-base Using config file "va-base" Error: (22, 'invalid argument') [2006-12-14 14:55:46 xend.XendDomainInfo 3874] ERRORs (XendDomainInfo:202) Domain construction failed Traceback (most recent call last) File "/usr/lib/python2.4/site-packages/xen/xend/XendDomainInfo.py", line 195 in create vm.initDomain() File " /usr/lib/python2.4/site-packages/xen/xend/XendDomainInfo.py", line 1363 in initDomain raise VmError(str(exn)) VmError: (22, 'Invalid argument') [2006-12-14 14:55:46 xend.XendDomainInfo 3874] DEBUG (XenDomainInfo: 1449] XendDlomainInfo.destroy: domain=1 [2006-12-14 14:55:46 xend.XendDomainInfo 3874] DEBUG (XenDomainInfo: 1457] XendDlomainInfo.destroy:Domain(1)
Unable to open a connection to the Xen hypervisor or daemon
/etc/hosts configuration file. Check the file and verify if the localhost entry is enabled. Here is an example of an incorrect localhost entry:
# Do not remove the following line, or various programs # that require network functionality will fail. localhost.localdomain localhost
# Do not remove the following line, or various programs # that require network functionality will fail. 127.0.0.1 localhost.localdomain localhost localhost.localdomain. localhost
xen-xend.logfile ):
Bridge xenbr1 does not exist!
# xm create mySQL01 Using config file " mySQL01" Going to boot Red Hat Enterprise Linux Server (2.6.18.-1.2747 .el5xen) kernel: /vmlinuz-2.6.18-12747.el5xen initrd: /initrd-2.6.18-1.2747.el5xen.img Error: Device 0 (vif) could not be connected. Hotplug scripts not working.
xend.log displays the following errors:
[2006-11-14 15:07:08 xend 3875] DEBUG (DevController:143) Waiting for devices vif [2006-11-14 15:07:08 xend 3875] DEBUG (DevController:149) Waiting for 0 [2006-11-14 15:07:08 xend 3875] DEBUG (DevController:464) hotplugStatusCallback /local/domain/0/backend/vif/2/0/hotplug-status [2006-11-14 15:08:09 xend.XendDomainInfo 3875] DEBUG (XendDomainInfo:1449) XendDomainInfo.destroy: domid=2 [2006-11-14 15:08:09 xend.XendDomainInfo 3875] DEBUG (XendDomainInfo:1457) XendDomainInfo.destroyDomain(2) [2006-11-14 15:07:08 xend 3875] DEBUG (DevController:464) hotplugStatusCallback /local/domain/0/backend/vif/2/0/hotplug-status
/etc/xen directory. For example, editing the guest mySQL01
# vim /etc/xen/mySQL01
Locate the vif entry. Assuming you are using xenbr0 as the default bridge, the proper entry should resemble the following:
# vif = ['mac=00:16:3e:49:1d:11, bridge=xenbr0',]
# xm shutdown win2k3xen12 # xm create win2k3xen12 Using config file "win2k3xen12". /usr/lib64/python2.4/site-packages/xenxm/opts.py:520: Deprecation Warning: Non ASCII character '\xc0' in file win2k3xen12 on line 1, but no encoding declared; see http://www.python.org/peps/pep-0263.html for details execfile (defconfig, globs, locs,) Error: invalid syntax 9win2k3xen12, line1)
/etc/xen/ directory containsxend daemon.
scripts directory which contains the scripts for Virtualization networking.
/var/log/xen/ /var/lib/libvirt/images/ /proc/xen/ cat /proc/partitions" as seen below.
# cat /proc/partitions major minor #blocks name 202 16 104857600 xvdb 3 0 8175688 hda
/etc/inittab file:
1:12345:respawn:/sbin/mingetty xvc0
Xen Virtual Ethernet networking card inside the guest operation system. Verify this by executing the following (for Red Hat Enterprise Linux 4 and Red Hat Enterprise Linux 5):
cat /etc/modprobe.conf
cat /etc/modules.conf
alias eth0 xen-vnif
alias eth0 xen-vnif) for every para-virtualized interface for the guest.
/etc/modprobe.conf. Edit /etc/modprobe.conf and add the following line to it:
options loop max_loop=64
phy: block device or tap:aio commands. To employ loop device backed guests for a full virtualized system, use the phy: device or file: file commands.
dom0 has not ballooned down enough to provide space for a recently created or started guest. In your /var/log/xen/xend.log, an example error message indicating this has occurred:
[2006-11-21 20:33:31 xend 3198] DEBUG (balloon:133) Balloon: 558432 KiB free; 0 to scrub; need 1048576; retries: 20. [2006-11-21 20:33:52 xend.XendDomainInfo 3198] ERROR (XendDomainInfo:202) Domain construction failedYou can verify the amount of memory currently used by
dom0 with the command “xm list Domain-0”. If dom0 is not ballooned down you can use the command “xm mem-set Domain-0 NewMemSize” where NewMemSize should be a smaller value.
# xm create testVM Using config file "./testVM". Going to boot Red Hat Enterprise Linux Server (2.6.18-1.2839.el5) kernel: /vmlinuz-2.6.18-1.2839.el5 initrd: /initrd-2.6.18-1.2839.el5.img Error: (22, 'Invalid argument')In the above error you can see that the kernel line shows that the system is trying to boot a non kernel-xen kernel. The correct entry in the example is ”
kernel: /vmlinuz-2.6.18-1.2839.el5xen”.
/etc/grub.conf configuration file.
kernel-xen installed in your guest you can start your guest:
xm create -c GuestName
GuestName is the name of the guest. The previous command will present you with the GRUB boot loader screen and allow you to select the kernel to boot. You will have to choose the kernel-xen kernel to boot. Once the guest has completed the boot process you can log into the guest and edit /etc/grub.conf to change the default boot kernel to your kernel-xen. Simply change the line “default=X” (where X is a number starting at '0') to correspond to the entry with your kernel-xen line. The numbering starts at '0' so if your kernel-xen entry is the second entry you would enter '1' as the default,for example “default=1”.
# xm create -c va-base Using config file "va-base". Error: (22, 'Invalid argument') [2006-12-14 14:55:46 xend.XendDomainInfo 3874] ERROR (XendDomainInfo:202) Domain construction failed Traceback (most recent call last): File "/usr/lib/python2.4/site-packages/xen/xend/XendDomainInfo.py", line 195, in create vm.initDomain() File "/usr/lib/python2.4/site-packages/xen/xend/XendDomainInfo.py", line 1363, in initDomain raise VmError(str(exn)) VmError: (22, 'Invalid argument') [2006-12-14 14:55:46 xend.XendDomainInfo 3874] DEBUG (XendDomainInfo:1449) XendDomainInfo.destroy: domid=1 [2006-12-14 14:55:46 xend.XendDomainInfo 3874] DEBUG (XendDomainInfo:1457) XendDomainInfo.destroyDomain(1)If you need to run a 32 bit or non-PAE kernel you will need to run your guest as a fully-virtualized virtual machine. The rules for hypervisor compatibility are:
Your CPU does not support long mode. Use a 32 bit distribution. This problem is caused by a missing or incorrect pae setting. Ensure you have an entry “pae=1” in your guest's configuration file.
virt-manager application may fail to launch and display an error such as “Unable to open a connection to the Xen hypervisor/daemon”. This is usually caused by a missing localhost entry in the /etc/hosts file. Verify that you indeed have a localhost entry and if it is missing from /etc/hosts and insert a new entry for localhost if it is not present. An incorrect /etc/hosts may resemble the following:
# Do not remove the following line, or various programs # that require network functionality will fail. localhost.localdomain localhost
# Do not remove the following line, or various programs # that require network functionality will fail. 127.0.0.1 localhost.localdomain localhost localhost.localdomain localhost
Applying Intel CPU microcode update: FATAL: Module microcode not found. ERROR: Module microcode does not exist in /proc/modulesAs the virtual machine is running on virtual CPUs there is no point updating the microcode. Disabling the microcode update for your virtual machines will stop this error:
/sbin/service microcode_ctl stop /sbin/chkconfig --del microcode_ctl
xm create” will look in the current directory for a configuration file and then in /etc/xen
# xm shutdown win2k3xen12 # xm create win2k3xen12 Using config file "win2k3xen12". /usr/lib64/python2.4/site-packages/xen/xm/opts.py:520: DeprecationWarning: Non-ASCII character '\xc0' in file win2k3xen12 on line 1, but no encoding declared; see http://www.python.org/peps/pep-0263.html for details execfile(defconfig, globs, locs) Error: invalid syntax (win2k3xen12, line 1)
Enabling the virtualization extensions in BIOS
cat /proc/cpuinfo | grep vmx svm. If the command outputs, the virtualization extensions are now enabled. If there is no output your system may not have the virtualization extensions or the correct BIOS setting enabled.
virsh command (where GUEST is the guest's name):
# virsh edit GUESTvirsh edit command uses the $EDITOR shell variable to determine which editor to use.
<interface type='network'>
[output truncated]
<model type='rtl8139' />
</interface>
'rtl8139' to 'e1000'. This will change the driver from the rtl8139 driver to the e1000 driver.
<interface type='network'>
[output truncated]
<model type='e1000' />
</interface>
# virsh dumpxml GUEST > /tmp/guest.xml
# cp /tmp/guest.xml /tmp/new-guest.xml # vi /tmp/new-guest.xml
<interface type='network'>
[output truncated]
<model type='e1000' />
</interface>
# virsh define /tmp/new-guest.xml # virsh start new-guest
/var/log/xen/xend daemon and qemu-dm process.
xend.logxend-debug.logxen-hotplug.logqemu-dm.PID.logqemu-dm process which is started for each fully virtualized guest.
PID is replaced with the PID of the process of the related qemu-dm process
PID for a given qemu-dm process using the ps command and in looking at the process arguments you can identify the virtual machine the qemu-dm process belongs to.
.virt-manager in the user's home directory whom ran virt-manager. This directory will usually be ~/.virt-manager/virt-manager.
/var/lib/libvirt/images//var/lib/xen/xend-db//etc/xen//etc/xen/xend-config.sxp is the main configuration for the xend daemon. The xend-config.sxp file enables or disables migration and other features not configured by libvirt. Use the libvirt tools for all other features.
/var/lib/xen/dump/xm dump-core command.
/proc/xen/xen-kernel information in the following files:
/proc/xen/capabilities
/proc/xen/privcmd
/proc/xen/balloon
/proc/xen/xenbus
/proc/xen/xsd_port
/proc/xen/xsd_kva
# insmod xen-platform-pci.o Warning: kernel-module version mismatch xen-platform-pci.o was compiled for kernel version 2.4.21-52.EL while this kernel is version 2.4.21-50.EL xen-platform-pci.o: unresolved symbol __ioremap_R9eac042a xen-platform-pci.o: unresolved symbol flush_signals_R50973be2 xen-platform-pci.o: unresolved symbol pci_read_config_byte_R0e425a9e xen-platform-pci.o: unresolved symbol __get_free_pages_R9016dd82 [...]The solution is to use the correct RPM package for your hardware architecture for the para-virtualized drivers.
Warning: kernel-module version mismatch xen-platform-pci.o was compiled for kernel version 2.4.21-52.EL while this kernel is version 2.4.21-50.EL Warning: loading xen-platform-pci.o will taint the kernel: forced load See http://www.tux.org/lkml/#export-tainted for information about tainted modules Module xen-platform-pci loaded, with warnings
# cd /lib/modules/`uname -r`/ # find . -name 'xen_*.ko' -print
# insmod \
/lib/modules/'uname -r'/{LocationofPV-drivers}/xen_platform_pci.ko
# insmod /lib/modules/'uname -r'/{LocationofPV-drivers}/xen_balloon.ko
# insmod /lib/modules/'uname -r'/{LocationofPV-drivers}/xen_vnif.ko
# insmod /lib/modules/'uname -r'/{LocationofPV-drivers}/xen_vbd.ko
/var/log/messages
# grep -E "vif|vbd|xen" /var/log/messages
xen_mem: Initialising balloon driver
vif vif-0: 2 parsing device/vif/0/mac
vbd vbd-768: 19 xlvbd_add at /local/domain/0/backend/vbd/21/76
vbd vbd-768: 19 xlvbd_add at /local/domain/0/backend/vbd/21/76
xen-vbd: registered block device major 202
You can also use the lsmod command to list the loaded para-virtualized drivers. It should output a list containing the xen_vnif, xen_vbd, xen_platform_pci and xen_balloon modules.
# lsmod|grep xen xen_vbd 19168 1 xen_vnif 28416 0 xen_balloon 15256 1 xen_vnif xen_platform_pci 98520 3 xen_vbd,xen_vnif,xen_balloon,[permanent]
type=ioemu' part of 'vif=' line in your guest's configuration file.
domU refers to the guest operating systems which run on the host system (the dom0 domain).
/dev/VolGroup00/LogVol02),
/dev/sda5), and
/dev/sda).
tap:aiotap:aio parameter sets the Xen hypervisor to use an advanced access mode designed for safety and performance. File-based, are accessed using a kernel thread and a user-space process. The tap:aio method respects guest flush requests which makes it safer than the file driver. The virtualization tools use tap:aio by default for accessing file-based guest disks on the Xen Hypervisor.
ext2 and ext3 file system identifiers, RAID device identifiers, iSCSI and LUN device identifiers, MAC addresses and virtual machine identifiers.
libvirt virtualization API.
/usr/share/doc/xen-<version-number>/ is the directory which contains information about the Xen para-virtualization hypervisor and associated management tools, including various example configurations, hardware-specific information, and the current Xen upstream user documentation.
man virsh and /usr/share/doc/libvirt-<version-number> — Contains sub commands and options for the virsh virtual machine management utility as well as comprehensive information about the libvirt virtualization library API.
/usr/share/doc/gnome-applet-vm-<version-number> — Documentation for the GNOME graphical panel applet that monitors and manages locally-running virtual machines.
/usr/share/doc/libvirt-python-<version-number> — Provides details on the Python bindings for the libvirt library. The libvirt-python package allows python developers to create programs that interface with the libvirt virtualization management library.
/usr/share/doc/python-virtinst-<version-number> — Provides documentation on the virt-install command that helps in starting installations of Fedora and Red Hat Enterprise Linux related distributions inside of virtual machines.
/usr/share/doc/virt-manager-<version-number> — Provides documentation on the Virtual Machine Manager, which provides a graphical tool for administering virtual machines.