Support for hugetlbfs was added with the release of kvm-62 and it can give your kvm guest machine a performance boost anywhere up to 10%. KVM provides you with an option of using hugetlbfs by using the –mem-path option when starting your kvm guest machine. Just as a sidenote, the mem-path option also supports tmpfs filesystems.
It is common nowadays for architectures to support more than one page size. As an example, x86 allows you to choose 4kb or 2MB pages. In the past, Linux traditionally used large pages for supporting the actual kernel image only but in kernel 2.6 linux started allowing processes to use huge pages. The whole purpose for huge page support is performance and you can get up to a 10% performance boost based on the application you’re running. Huge pages tend to give a bigger performance boost to applications that are memory and compute intensive. In the following sections, I’ll walk through the procedure for backing your kvm guest with huge pages.
First thing you need to do is check that your kernel has support for huge pages. If you’re running a 2.6 kernel then you most likely have huge page support already. You can confirm by running the following command.
# grep HUGETLB /boot/config-`uname -r` CONFIG_HUGETLBFS=y CONFIG_HUGETLB_PAGE=y
I ran this command on a Fedora Linux host but it should be similar for your distribution. Your output should look similar if your kernel has hugetlbfs support enabled.
On x86, large pages are 2MB in size so you need to find out how many 2MB pages you need to back your kvm guest. Let’s say you want to use 512 MB for your guest machine then divide 512 by 2 to get 256 pages. Add a few extra pages for additional memory requirements, lets say 40 pages to give a total of 296 pages. Record this value as you’ll need it later on.
Now you need to setup your hugetlbfs filesystem on your host. As root, create a directory called hugepages like so.
Next add an entry to your /etc/fstab so that when you reboot your computer, your hugetlbfs will mount automatically. Your entry should look like the following.
hugetlbfs /hugepages hugetlbfs defaults 0 0
Because it is difficult to reserve large pages on a busy system, it’s always best to reserve your huge pages shortly after restart.
As mentioned in the previous section, reserving huge pages is best done right after a reboot as it is difficult to reserve large pages on a busy system. Right after your computer has rebooted and using the value you calculated above for the number of huge pages you want to reserve, execute the following command.
echo 296 > /proc/sys/vm/nr_hugepages
If you want this to be a permanent setup , you can also add it to your /etc/rc.local script so that it is always reserved on startup. To verify that you have large pages reserved inspect the contents of the /proc/meminfo file as follows.
# tail -n 5 /proc/meminfo HugePages_Total: 296 HugePages_Free: 296 HugePages_Rsvd: 0 HugePages_Surp: 0 Hugepagesize: 2048 kB
Now you are ready to start backing your virtual machine with huge pages.
You have verified everything to this point and ready to back your virtual machine with large pages. Start your virtual machine with using the –mem-path option as follows:
qemu-system-x86_64 –hda windows.img –m 512 –mem-path /hugepages
You will probably notice a performance increase in your virtual machine. You can check that you’re actually using huge pages by inspecting the /proc/meminfo file again. For example, this is what mine looked like while running a kvm guest backed with the values in this post.
# tail -n 5 /proc/meminfo HugePages_Total: 296 HugePages_Free: 39 HugePages_Rsvd: 0 HugePages_Surp: 0 Hugepagesize: 2048 kB
That concludes the procedure for backing your guest with huge pages. It doesn't take that long to setup so give it a try and see if you get a performance boost.