kvm performance tuning.

Set swappiness to 10, swap space will be used at or 90% usage of system ram, generally on linux systems, this value is set to 60.

Checking the current swappiness value:

cat /proc/sys/vm/swappiness
60

To change the value at system run time use:

sysctl -w vm.swappiness=10

To keep the changes permanent we can add these lines to /etc/sysctl.conf:

kernel.sysrq = 1
vm.swappiness = 10
vm.vfs_cache_pressure = 50

vm.vfs_cache_pressure tells the reclaim value of memory used for directory and inode caches. By default this value is 100, on my host, it is set to 50.

Transparent Huge Pages (thp) a feature enabled by default in rhel7 systems, provides using memory pages greater than the default 4kB pages from memory, they can be 2MB or 1GB pages called as huge pages. KVM guests provisioned to use thp can get improved performance by increasing CPU cache hits against the Transaction Lookaside Buffer.

It is recommended to disable thp. Check if the thp pages are enabled:

cat /sys/kernel/mm/transparent_hugepage/enabled
cat /sys/kernel/mm/transparent_hugepage/defrag

To disable at system run time use:

echo 'never' > /sys/kernel/mm/transparent_hugepage/enabled
echo 'never' > /sys/kernel/mm/transparent_hugepage/defrag

We can create a systemd service to disable thp:

cat /etc/systemd/system/disable-thp.service
[Unit]
Description=Disable Transparent Huge Pages (THP)

[Service]
Type=simple
ExecStart=/bin/sh -c "echo 'never' > /sys/kernel/mm/transparent_hugepage/enabled && echo 'never' > /sys/kernel/mm/transparent_hugepage/defrag"

[Install]
WantedBy=multi-user.target

Another way of disabling this would be adding a grub line parameter:

GRUB_CMDLINE_LINUX="rhgb quiet video=SVIDEO-1:d elevator=noop console=tty0 console=ttyS0,115200n8 elevator=deadline transparent_hugepage=never"

Kernel same-page merging (ksm) combines identical memory pages from multiple running processes into one memory region. Because KVM guests run as processes under Linux, KSM provides the memory overcommit feature to hypervisors for more efficient use of memory.

Checking if ksm is enabled:

grep KSM /boot/config-`uname -r`
CONFIG_KSM=y

To turn it on:

echo 1 > /sys/kernel/mm/ksm/run

Non-Uniform Memory Access (NUMA) is enabled by default on redhat7.

systemctl start numad
systemctl enable numad

References: