My server runs CentOs 5.x, and uses the Gitco repo to provide later versions of Xen. I’ve previously sorted out the issues (at least in earlier Xen and CentOS versions) around bridging guest vifs to a bonded interface (I replaced Xen entwork scripts with stock CentOS ones, which needed patching to bring bonds up before bridges). Everything’s been fine with one small caveat – any time yum updates a kernel, before rebooting, I need to go check grub.conf to make sure that it’s using the xen version from gitco not the one matching the centOS kernel version.

So, with the Xen kernel 3.4.1 from Gitco, I did a yum update which included kernel 2.6.18-164.11.1.el5xen and somehow forgot to check grub.conf before rebooting. I waited and waited, but got no response from my server.

Luckily, I have a remote console on my physical server, so I was able to investigate. I found that the dom0 startup was hanging at “Bringing up interface bond0″. I hit the virtual reset button and dropped into the grub menu. I edited the kernel line to use xen.gz-3.4.1 and thought that would be it.

No, same problem. Same thing booting kernel 2.6.18-164.10.1.el5xen as well, either on its matching Xen kernel version, or 3.4.1 from Gitco. However, I could boot dom0 into 2.6.18-164.9.1.el5xen against Xen kernel 3.4.1, albeit with terribly slow network performance and “xen_net: Memory squeeze in netback driver” logged regularly in /var/log/messages, and network access to my guests was so slow that nearly everything timed out.

A bit of research on Google suggested that adding “dom0_mem=” to the kernel line in grub.conf might help.

Sure enough, it did – it cured the “memory squeeze” messages, and it’s let me boot dom0 into kernel 2.6.18-164.11.1.el5xen on top of Xen kernel 3.4.1 as well.

My guess is that kernels after 2.6.18-164.9.1.el5xen have some change that means without a restriction on dom0 memory, something somewhere breaks, and one of the side effects is terribly slow networking, possibly only when bonded?