The Book of Xen Part 17

All in all, Xen is quite a good platform for Windows. Not perfect, but certainly usable.

Chapter14.TIPS

By now you're some kind of Xen expert, we imagine.[81] As such, now we'd like to devote a chapter to the more esoteric aspects of working with Xen. Here are some things that didn't seem to fit anywhere else-stuff like the framebuffer, forwarding PCI devices, or building added functionality into the XenStore. Tips, in other words. As such, now we'd like to devote a chapter to the more esoteric aspects of working with Xen. Here are some things that didn't seem to fit anywhere else-stuff like the framebuffer, forwarding PCI devices, or building added functionality into the XenStore. Tips, in other words.

A number of the topics in Chapter15 Chapter15 might also come in handy when you work through our examples here. Some of the software discussed here is even more bleeding edge than the rest of Xen, which is itself some kind of might also come in handy when you work through our examples here. Some of the software discussed here is even more bleeding edge than the rest of Xen, which is itself some kind of heavenly sword heavenly sword, ravening and incarnadine. What we're trying to get at is that the material in this chapter might not work straight off.

Compiling Xen Although we've relied, for the most part, on Xen packages provided by distro maintainers, we think it's generally worthwhile to compile Xen from scratch. This will allow you to use a much more up-to-date version of Xen than versions that come with the distros. It also lets you enable options that the distro maintainers may have disabled.

If you're feeling adventurous, it's also nice to be able to play with the code-change it around a bit, perhaps, or add some printk printk messages to help with debugging. messages to help with debugging.

The easiest way to compile is to check out the latest source from the Mercurial repository. Start by making sure you have Mercurial and a bunch of build dependencies. On CentOS 5, we installed these packages with yum yum:[82]

mercurial zlib-devel gcc libX11-devel openssl-devel ncurses-devel bridge-utils python-devel git dev86 glibc-devel If you want the docs to build successfully, you should also install the following packages: texinfo tetex-latex pstoedit transfig Given the large amount of Xen doc.u.mentation available online and from other sources, the included docs are fairly optional.

When these packages are installed, clone the development repository. We're using xen-unstable xen-unstable here, but if you'd like to use a less unstable repo, you might want to try something like here, but if you'd like to use a less unstable repo, you might want to try something like xen-3.3-testing.hg xen-3.3-testing.hg. As of early 2009, prgmr.com runs xen-3.3-testing.hg xen-3.3-testing.hg. It has been pretty stable.

#hgclonehttp://xenbits.xen.org/xen-unstable.hgRPMFORGE REPOSITORYIf you want to install all the previously listed packages using yum yum, you must use the RPMForge repo. To do so safely, install yum-priorities install yum-priorities. On CentOS 5:#yuminstallyum-prioritiesEdit each of the files under /etc/yum.repos.d/* /etc/yum.repos.d/* and add the line and add the line priority=N priority=N,where N N is a number from 1 to 99 (lower number is greater priority). You want is a number from 1 to 99 (lower number is greater priority). You want base base, addons addons, updates updates, and extras extras to be priority to be priority 1 1; centosplus centosplus and and contrib contrib to be priority to be priority 2 2; and everything else to be priority 10 10 or higher. or higher.Now, install RPMforge:wget http://apt.sw.be/redhat/el5/en/i386/RPMS.dag/rpmforge-release-0.3.6-1 .el5.rf.i386.rpmOf course, if your machine is x86_64, subst.i.tute the appropriate architecture:rpm--importhttp://dag.wieers.com/rpm/packages/RPM-GPG-KEY.dag.txtthen install rpmforge rpmforge with: with:rpm-Krpmforge-release-0.3.6-1.el5.rf.*.rpmFinally, edit /etc/yum.repos.d/rpmforge.repo /etc/yum.repos.d/rpmforge.repo and add and add priority=10 priority=10.

This will download the repo to a local directory (xen-unstable.hg in this case). Next, in this case). Next, cd cd into that directory and run into that directory and run make world make world: #cdxen-unstable.hg #makeworld&&makeinstall This will build and install the Xen hypervisor, its supporting tools, and a Linux kernel for dom0. DomUs can use it as well. Often, this will be all you need. However, if you want to change your kernel configuration, you can. To configure the Linux kernel, run: #makelinux-2.6-xen-configconfigmode=MENUCONFIG This will open the standard Linux kernel configurator. Configure the kernel as usual.

NoteYou probably want to leave the dom0 8250 serial driver disabled because it conflicts with the Xen serial console. As usual, don't forget the drivers for your boot device.

Then run: #makelinux-2.6-build #makelinux-2.6-install This builds and installs the kernel. Now, if you are on CentOS, you probably want to make an initrd: #mkinitrd/boot/initrd-2.6.18.8-xen.img2.6.18.8-xenNoteThere is a bug in early releases of RHEL 5.3 that causes problems with this. See https://bugzilla.redhat.com/show_bug.cgi?id=488991 for details. The solution is to add for details. The solution is to add --allow-missing --allow-missing to the to the mkinitrd mkinitrd command line, thus command line, thus: # mkinitrd /boot/initrd-2.6.18.8-xen.img 2.6.18.8-xen --allow-missing # mkinitrd /boot/initrd-2.6.18.8-xen.img 2.6.18.8-xen --allow-missing.

Now, you need to fix /boot/grub/menu.lst /boot/grub/menu.lst. Add a stanza like this, but remember to use appropriate devices, paths, and possibly filenames: t.i.tleXen.org2.6.18.8-xen-3.3 root(hd0,0) kernel/boot/xen-3.3.gz module/boot/vmlinuz-2.6.18.8-xenroroot=/dev/md0 module/boot/initrd-2.6.18.8-xen.img Reboot and enjoy your new Xen installation.

[81] Or at least, anyone who hasn't thrown this book out the window must be extremely good at filling in vague directions. Or at least, anyone who hasn't thrown this book out the window must be extremely good at filling in vague directions.

[82] We've also included a more Debian-centric set of compilation instructions in We've also included a more Debian-centric set of compilation instructions in paravirt_ops Dom0 paravirt_ops Dom0 on on Alternate Kernels (Dom0 and DomU) Alternate Kernels (Dom0 and DomU) and and paravirt_ops DomU paravirt_ops DomU on on paravirt_ops DomU paravirt_ops DomU.

Compile-Time Tuning That's the quick and easy way to build Xen, but the basic compilation with make world make world is just the beginning. Compilation represents the first opportunity we have to configure Xen, and there's a lot more that we can do with it now that we've had some practice. is just the beginning. Compilation represents the first opportunity we have to configure Xen, and there's a lot more that we can do with it now that we've had some practice.

Most of the compile-time tuning can be done by twiddling variables in Config.mk Config.mk, at the top level of the Xen source tree. This file is fairly extensively commented and amenable to editing-take a look. You'll find that there's a brief section where you can decide which optional Xen bits to build.

We usually turn on all of the optional components except for the Virtual Trusted Platform Module (VTPM) tools, leading to a section like this: XENSTAT_XENTOP?=y VTPM_TOOLS?=n LIBXENAPI_BINDINGS?=y XENFB_TOOLS?=y PYTHON_TOOLS?=yNoteXen's VTPM tools are interesting. They've been a subject of heavy development, they have some interesting implications for signed code, and there's the looming specter of DRM, but we just haven't gotten into them. If you decide to build them, you can add virtual TPMs to domains via the vtpm= vtpm= option in the domain configuration option in the domain configuration.

If you're having trouble (trust us, you probably will at some point), it would be a good idea to make a debug build. To do that, set the DEBUG DEBUG variable at the top of the file: variable at the top of the file: DEBUG?=y Don't worry: Xen will not run in debug mode unless you specifically instruct it to do so at runtime.

These optional Xen components have a bunch of undoc.u.mented dependencies, some of which aren't checked for by the Makefiles. In particular, the LIBXENAPI_BINDINGS demand libxml2 and curl or the -devel versions of these packages, if you're using a Red Hat derivative.

Also, if something doesn't work when you build the tools, it would probably be a good idea to avoid running make world make world again because that takes a while. Most likely, you can get by with just again because that takes a while. Most likely, you can get by with just make tools make tools.

Alternate Kernels (Dom0 and DomU) The default Xen Makefile will build a single kernel that can be used in both the dom0 and domU. If saving memory is a high priority, you can build a separate kernel for each. These kernels will each have a reasonable set of configuration options: minimal for the domU, modular for the dom0. Specify the KERNELS KERNELS variable on your variable on your make make command line: command line: #makeKERNELS="linux-2.6-dom0linux-2.6-domU"

The primary reason to do this, of course, is so that you can strip all the non-Xen device drivers out of the domU kernel. This saves memory and-if you happen to be testing a lot of kernels-compile time.

paravirt_ops Dom0 To understand why paravirt_ops gets treated as a separate piece, we have to recall that a lot of the early Xen development took place before virtualization went mainstream. The Xen developers, to paravirtualize the Linux kernel, made sweeping changes that proved to be difficult to merge with mainline kernel development.

paravirt_ops is a generic solution to this problem. It's a kernel-level framework for adding code to enable Linux to run under various hypervisors, including Xen. The idea is that, by making these interfaces part of the official kernel, we can make Xen less invasive and easier to maintain.

Xen has supported paravirt_ops domUs since version 3.1, and the official Linux kernel has had domU support since version 2.6.23 for i386 and since version 2.6.26 for x86_64. Unfortunately, the kernel.org kernel, as of this writing, only has guest support.

But there is light at the end of the tunnel. With the latest patches from Jeremy Fitzhardinge's paravirt_ops dom0 work and a Xen 3.4 hypervisor, it is, in fact, possible to run a paravirt_ops dom0 based on Linux kernel version 2.6.30.

These directions represent a snapshot from a very long development process. They work for us today. URLs may change. The status of the software certainly will. With that in mind, though, here's how we set up a functioning paravirt_ops dom0.

First, you're going to need some development packages. This time we're using the Debian package names: mercurial build-essential libncurses5-dev gawk openssl xorg-dev gettext python-dev gitk libcurl4-openssl-dev bcc libz-dev libxml2-dev Next, check out Xen-unstable with Mercurial. We warned you that this stuff is still in development.

#hgclonehttp://xenbits.xensource.com/xen-unstable.hg #cdxen-unstable.hg #makexen #makeinstall-xen #maketools #makeinstall-tools Then check out the current Linux patches from Jeremy Fitzhardinge's git repo: #gitclonegit://git.kernel.org/pub/scm/linux/kernel/git/jeremy/xen.gitlinux-2.6-xen #cdlinux-2.6-xen #gitcheckoutorigin/push2/xen/dom0/master-bpush2/xen/dom0/master Configure the kernel. We copied the Ubuntu configuration to .config .config and used that as a base. and used that as a base.

#cp/boot/config-2.6.26-11-server.config #makemenuconfig Since we're building a paravirt_ops dom0, make sure to turn on the appropriate support: Processortypeandfeatures ->Paravirtualizedguestsupport ........->EnableXenprivilegeddomainsupport Be sure to turn on the Xen block device frontend support, under: DeviceDrivers ->Blockdevices Next, build the kernel.

#make #makemodules_installinstall #depmod2.6.30-tip #mkinitramfs...o...b..ot/initrd-2.6.30-tip.img2.6.30-tip Add /proc/xen /proc/xen to to fstab fstab and mount it so that tools like and mount it so that tools like xend xend will be able to communicate with the hypervisor: will be able to communicate with the hypervisor: none/proc/xenxenfsdefaults00 Create a GRUB entry to boot your new Xen paravirt_ops dom0: t.i.tleXen3.4/Ubuntu8.10,kernel2.6.30-tip kernel/boot/xen-3.4.gz module/boot/vmlinuz-2.6.30-tiproot=/dev/sdb2roconsole=tty0 module/boot/initrd-2.6.30-tip.img Make sure that these are appropriate values for your setup, of course. That's all there is to it.

paravirt_ops DomU "But what," you ask, "is all this about using a kernel.org kernel in a domU?" If you just want to make your own domU kernel, this is a much less involved process, supported without out-of-tree patches since version 2.6.23. All of these directions are presented from within a domU that boots using PV-GRUB or PyGRUB-no intervention from the dom0 administrator should be necessary.

First, download the kernel source you prefer: #wgethttp://kernel.org/pub/linux/kernel/v2.6/linux-2.6.29.3.tar.bz2 Next, install the packages normally required to build the kernel. This example is for Debian, but it should be easy enough to find out what packages your favorite distro needs to build the kernel.

#apt-getinstallbuild-essentiallibncurses5-dev Untar and configure the kernel. Personally, we like menuconfig menuconfig, but that's just a matter of taste: #tar-jxflinux-2.6.29.3.tar.bz2 #cdlinux-2.6.29.3 #makemenuconfig Don't forget to enable Xen support: ->Processortypeandfeatures ->Paravirtualizedguestsupport ->XEN Don't forget your network driver: ->DeviceDrivers ->Networkdevicesupport ->XEN_NETDEV_FRONTEND or your disk driver: ->DeviceDrivers ->Blockdevices ->XEN_BLKDEV_FRONTEND Xenfs, which allows you to access the XenBus, is sometimes useful: ->DeviceDrivers ->XENFS Then customize to your heart's content. Remember, you can remove support for just about all hardware now. We also leave out the balloon driver. RAM is cheap, and we like having definite memory allocations.

Now, make the kernel as usual: make-j4;makeinstallmodules_install Make your initrd as per the usual kernel build. Since we're using Debian for this example, that means using mkinitramfs mkinitramfs. If you compiled xenblk xenblk as a module, make sure to include it. as a module, make sure to include it.

mkinitramfs...o...b..ot/initrd-2.6.29.3.img2.6.29.3 Set up GRUB as you normally would: t.i.tlekernel.orgparavirtDomU root(hd0,0) kernel/boot/vmlinuz-2.6.29.3root=LABEL=DISK1ro initrd/boot/initrd-2.6.29.3.img One last thing before you reboot: Note that the device name for your console will be hvc0, for hypervisor console hypervisor console. This takes the place of the Xen-specific xvc0. If your distro doesn't do so already, you probably want to set up the domain to start a getty getty on hvc0. Now, simply restart your domain (halt it and start it up if you are using PyGRUB) and enjoy your modern kernel. on hvc0. Now, simply restart your domain (halt it and start it up if you are using PyGRUB) and enjoy your modern kernel.

#uname-a Linuxsebastian.xen.prgmr.com2.6.29.3#1SMPTueMay1206:32:52UTC2009x86_64GNU/Linux2009

The Xen API: The Way of the Future The Xen API is an XML-RPC interface to Xen that replaces the old interface used for communication with the hypervisor. It promises to provide a standard, stable interface so that people can build Xen frontends without worrying about the interface changing out from under them. It also extends the previous Xen command set so that more of Xen's functionality can be harnessed in a standardized tool.

In current versions of Xen, the API is an optional component, but that shouldn't deter you from using it; the most recent Citrix Xen Server product, for example, relies on the API exclusively for communication between the administration frontend and the virtualization host.

The Xen API is enabled by setting the LIBXENAPI_BINDINGS flag at the top of Config.mk Config.mk: LIBXENAPI_BINDINGS?=y When you've built Xen with support for the Xen API, the use of the API is controlled by the (xen-api-server) (xen-api-server) directive in directive in /etc/xen/xend-config.sxp /etc/xen/xend-config.sxp.

(xen-api-server((9363none)(unixnone))) This directive turns on the API server and specifies how to connect to it. Each list in parentheses is a connection method. In this case, we're using TCP port 9363 and a local Unix socket, each with no authentication whatsoever.

To specify that we want to authenticate using PAM, we can modify this configuration a bit: (xen-api-server((9363pam'192.0.2.*')) Ordinarily, even when developing a Xen client, you won't need to interact with the Xen API at a low level. Bindings exist for most of the popular languages, including C and, of course, Python. The Xen.org API doc.u.mentation, accessible from http://wiki.xensource.com/xenwiki/XenApi/, is the last word on the subject.

Managing Memory with the Balloon Driver Moving on from compile time and installation issues to the serious day-to-day business of running Xen, we encounter the problem of memory. As we've mentioned, most Xen installations are limited in practice by physical memory.

Xen expends a great deal of effort on virtualizing memory; its approach is one of the defining features of paravirtualization, and it usually "just works," on a level low enough to ignore completely. However, it sometimes can benefit from a bit of attention by the administrator.

We've been dancing around the subject of memory oversubscription for a long time, and we'd better come clean: It is possible to a.s.sign a dynamic amount of memory to a domU, but we don't do it because it's not suitable for our virtual private server virtual private server model. Also, the developers have historically been leery about recommending it for production. However, it does exist, and there are some good reasons to use it, which we're sure you can imagine. model. Also, the developers have historically been leery about recommending it for production. However, it does exist, and there are some good reasons to use it, which we're sure you can imagine.

Xen's control over reallocating memory to virtual machines is somewhat indirect. First, the value in the domU config file for memory is a maximum maximum-conceptually, the amount of memory that's physically accessible to the virtual machine. The amount of memory in the config file is what the kernel sees at boot. Adding more would require a reboot. This is being worked on, mostly from the direction of Linux's memory hotplug. We confidently expect someone to provide a patch soon.

Within this inflexible maximum, Xen can reduce this memory using the balloon driver balloon driver, which is nothing but a module that sits in the domU and inflates inflates to consume memory, which it then hands back to the hypervisor. to consume memory, which it then hands back to the hypervisor.

Because the dom0 is also a Xen domain, it also can have a memory balloon, which Xen uses to reclaim dom0 memory for allocation to domUs.

NoteWe favor setting the dom0 memory directly with the dom0_mem dom0_mem boot option, which actively hides memory from the dom0. By setting boot option, which actively hides memory from the dom0. By setting (dom0-min-mem 0) (dom0-min-mem 0) and and (enable-dom0-ballooning no) (enable-dom0-ballooning no) in in xend-config.sxp, xend-config.sxp, we can ensure that dom0 doesn't balloon out, and thus has a consistent memory reservation we can ensure that dom0 doesn't balloon out, and thus has a consistent memory reservation.[83]

You can use xm xm to manually adjust the amount of memory used by the balloon: to manually adjust the amount of memory used by the balloon: #xmmem-setsebastian112 The domain treats this as a target, giving memory to the balloon as it becomes free.[84] Thus, it's possible that a heavily loaded domain will take a while to give up memory to the balloon. Thus, it's possible that a heavily loaded domain will take a while to give up memory to the balloon.

You can see the effect of the balloon in the list of VMs: #xmlistsebastian NameIDMem(MiB)VCPUsStateTime(s) sebastian711111-b----38.4 You can also see balloon-related information from within the domU via /proc/xen/balloon /proc/xen/balloon: #cat/proc/xen/balloon Currentallocation:114688kB Requestedtarget:114688kB Low-memballoon:24576kB High-memballoon:0kB Driverpages:136kB Xenhardlimit:???kBNoteThe balloon is pretty aggressive; it can cause an out-of-memory condition in the domU. Use it with caution.

[83] In newer versions of Xen, you really only need In newer versions of Xen, you really only need (enable-dom0-ballooning no) (enable-dom0-ballooning no), but that has no effect on older versions. Before the enable-dom0-ballooning enable-dom0-ballooning option was enabled, setting option was enabled, setting dom0-min-mem dom0-min-mem to 0 would disable ballooning. Really, you could get away with just setting to 0 would disable ballooning. Really, you could get away with just setting dom0-min-mem dom0-min-mem to 0; I tested it after embarra.s.sing myself on the xen-devel list, and it works, but to 0; I tested it after embarra.s.sing myself on the xen-devel list, and it works, but (enable-dom0-ballooning no) (enable-dom0-ballooning no) is nice and clear, and this sort of thing is important enough to specify twice. is nice and clear, and this sort of thing is important enough to specify twice.

[84] Although Linux does its best to keep memory in use at all times, it'll give memory to the balloon rather than using it for buffers or cache. Although Linux does its best to keep memory in use at all times, it'll give memory to the balloon rather than using it for buffers or cache.

PCI Forwarding You can allow a domU to access arbitrary PCI devices and use them with full privileges. Of course, there's no such thing as a free lunch; Xen can't miraculously duplicate PCI hardware. For a domU to use a PCI device, it has to be hidden from the dom0 and not forwarded to any other domUs.

Figure14-1.Xen PCI device forwarding As Figure14-1 Figure14-1 shows, PCI forwarding uses a client/server model in which the shows, PCI forwarding uses a client/server model in which the pcifront driver pcifront driver runs in the domU and communicates directly with the runs in the domU and communicates directly with the pciback driver pciback driver, which binds to the PCI device and hides it from the dom0.

First, consider the device that you want to forward to the domU. The test machine that we're sitting in front of appears to have seven (!) USB controllers, so we'll just take a couple of those.

Use lspci lspci to determine bus IDs: to determine bus IDs: #lspci

00:1a.0USBController:IntelCorporation82801H(ICH8Family)USB UHCI#4(rev02) 00:1a.1USBController:IntelCorporation82801H(ICH8Family)USB UHCI#5(rev02) 00:1a.7USBController:IntelCorporation82801H(ICH8Family)USB2 EHCI#2(rev02)

(&c.) We'll forward 00:1a.1 00:1a.1 and and 00:1a.7 00:1a.7, which are the second of the USB1 controllers listed and the USB2 controller. Your device names will most likely vary from those in this example.

If pciback is compiled into the kernel, you can boot the dom0 with a pciback.hide pciback.hide option on the kernel command line. For these two controllers, the option would look like this: option on the kernel command line. For these two controllers, the option would look like this: pciback.hide=(00:1a.1)(00:1a.7) If pciback is a module, it's a little more difficult. We need to detach the PCI device from its driver and attach it to the pciback pa.s.sthrough.

#insmodpcibackhide=(00:1a.1)(00:1a.7) #echo-n00:1a.1>/sys/bus/pci/drivers/uhci_hcd/unbind #echo-n00:1a.1>/sys/bus/pci/drivers/pciback/new_slot #echo-n00:1a.1>/sys/bus/pci/drivers/pciback/bind Now put these devices into the domU config file: pci=['00:1a.1','00:1a.7']

On the next domU boot, these USB controllers should appear and be available to the native drivers in the domU. Hardware devices on platforms without an IOMMU can DMA to arbitrary memory regions. This can be a security problem if you're giving PCI access to arbitrary domains. The moral is to treat all domains with access to the PCI bus as privileged. Make sure you can trust them.

GRUB Configuration Of course, we've dealt with GRUB in pa.s.sing because it's one of the basic prerequisites for Xen. However, there are a few more aspects of GRUB that are worth mentioning in depth. A fair number of Xen's behavior k.n.o.bs can be tweaked in GRUB at boot time by adjusting the command-line parameters pa.s.sed to the hypervisor.

For example, the already-mentioned dom0_mem dom0_mem parameter adjusts the amount of memory that Xen allows the dom0 to see: parameter adjusts the amount of memory that Xen allows the dom0 to see: kernel/boot/xen.gzdom0_mem=131072 To keep the system from rebooting if you have a kernel panic, which happens more often than we would like, especially when trying to get machines initially set up, add noreboot noreboot to the to the kernel kernel line: line: kernel/boot/xen.gzdom0_mem=131072noreboot as well as panic=0 panic=0 to the Linux to the Linux module module line: line: module/boot/vmlinuz-2.6.18-53.1.21.el5xenpanic=0 This is, of course, in addition to the plethora of options supported by the Linux kernel, which you can then add to vmlinuz vmlinuz's module module line as you see fit. line as you see fit.

The Serial Console One other important GRUB-related task is setting up your serial console. As mentioned, we consider the serial console to be the gold standard for console access to any sort of server. It's much simpler than any sort of graphical interface, easy to access with a variety of devices, and is the output most likely to provide useful information when the machine is cras.h.i.+ng. Furthermore, because of the client/server architecture inherent in the system, anything that a cras.h.i.+ng machine manages to print goes to another, physically separate machine, where it can be a.n.a.lyzed at leisure.

Xen comes with miniterm, a minimal serial client for this sort of thing, in case you don't have access to a serial client. This is unlikely, but the client is tiny, so why not?

Miniterm is in the tools/misc/miniterm tools/misc/miniterm subdirectory of the Xen source tree. If you've built all the tools with Xen, it'll already be built and possibly even installed; if not, you can simply type subdirectory of the Xen source tree. If you've built all the tools with Xen, it'll already be built and possibly even installed; if not, you can simply type make make in that directory and run the resulting executable. in that directory and run the resulting executable.

Enabling Serial Output There are four components that need to have their output redirected to the serial port: GRUB, Xen, the Linux kernel, and Linux's userland. Each of the first three is a simple matter of adding a directive to GRUB's menu.lst menu.lst.

First, near the top of the file, add these lines: serial--unit=0--speed=115200--word=8--parity=no--stop=1 terminal--timeout10serialconsole Edit the Xen kernel line to tell the hypervisor to use the first serial port for output: kernel/boot/xen.gz-2.6.18-53.1.21.el5console=com1com1=115200,8n1 Tell the Linux kernel to print its messages on ttyS0 ttyS0: module/boot/vmlinuz-2.6.18-53.1.21.el5xenroroot=/dev/md0 console=ttyS0,115200n8 Finally, edit /etc/inittab /etc/inittab and add a line like the following: and add a line like the following: 7:2345:resp.a.w.n:/sbin/agetty115200ttyS0 You may also want to add ttyS0 ttyS0 to to /etc/securetty /etc/securetty so that root will be able to log in, after the manner of a traditional console. so that root will be able to log in, after the manner of a traditional console.

The Xen Hypervisor Console Xen adds another layer to the serial console by using it to access extra hypervisor features. First, break into the hypervisor console by pressing CTRL-A three times on the serial console. This won't work on the VGA console. You'll get a (XEN) prompt.

When you're in the hypervisor console, there are several useful (or at least interesting) commands you can give Xen. Try typing h h for help or one of the informational commands, like for help or one of the informational commands, like m m. You can also crash the machine, reboot it, or dump various pieces of information. Poke around and try it.

To exit the hypervisor console, type CTRL-A three more times.

Xen and LILO This section only applies to the real dinosaurs out there, but we sympathize. In keeping with the feeling that you have vanished into the mysterious past, we will present this example using Xen 3.0.

If you're dead set on using LILO, rather than GRUB, you will be pleased to learn that it is possible. Although it's generally thought that LILO's lack of an equivalent to GRUB's module module directive makes it impossible for it to boot Xen, it's possible to get around that by combining the hypervisor, dom0 kernel, and initrd into one file using directive makes it impossible for it to boot Xen, it's possible to get around that by combining the hypervisor, dom0 kernel, and initrd into one file using mbootpack mbootpack.

Consider the following entry in grub.conf grub.conf: t.i.tleslack-xen root(hd0,0) kernel/boot/xen.gz module/vmlinuz-2.6-xenroroot=/dev/hda1ro module/initrd-2.6.18-xen.gz It loads the hypervisor, xen-3.0.gz xen-3.0.gz, as the kernel then unpacks vmlinuz-2.6-xen vmlinuz-2.6-xen and and initrd.gz initrd.gz into memory. To combine these files, first decompress: into memory. To combine these files, first decompress: #cd/boot #gzcatxen-3.0.gz>xen-3.0 #gzcatvmlinuz-2.6-xen0>vmlinux-2.6-xen0 #gzcatinitrd.gz>initrd.img Note the change from vmlinuz vmlinuz to to vmlinux vmlinux. It's not important except that it keeps you from overwriting the kernel at the beginning of the gzcat gzcat process. process.

Then combine the three files using mbootpack mbootpack: #mbootpack-ovmlinux-2.6-xen.mpack-mvmlinux-2.6-xen0-minitrd.gz -minitrd.imgxen3.0 The grub.conf grub.conf entry then becomes a entry then becomes a lilo.conf lilo.conf entry: entry: image=/boot/vmlinux-2.6-xen.mpack label=xen root=/dev/ram0 Finally, run the lilo lilo command. command.

#/sbin/lilo

The Virtual Framebuffer For as much as purists would like to claim that all administration should be done via serial port, there's something to be said for all this newfangled graphical technology that we've been using for, oh, around the last 25 years. Xen makes a concession to these forward-thinking beliefs by including a facility for a virtual framebuffer virtual framebuffer.

You will need to edit your Config.mk Config.mk file to build the VFB: file to build the VFB: XENFB_TOOLS?=y At this point you'll also need libvncserver and libsdl-dev. Install them in your chosen way. We installed CentOS's SDL-devel package and installed libvncserver from source. Then we built Xen and installed it in the usual way.