Flashcache on CloudLinux and CentOS and OpenVZ or SolusVM

Major issue on a hosting server which serve hundreds of website is remain the same: I/O bottleneck. My curiosity began when I received a newsletter from a VPS provider in US, they said they have added SSD as filesystem cache to their server fleet.

Flashcache is a kernel module released by Facebook engineer back in April 2010 to speed up database I/O on a standard rotational drive with the speed of SSD. Since it works by caching filesystem blocks, so it can speed anything on the slow rotational disk.
We’ve already use non-raid with HDD disk on all of our hosting servers, but still I/O bottleneck often happens.

Server Configuration
We have 3 slot of 3.5″ drive bays on our dell C6100 server and load it up with :

1. Disk 0 : 250GB HDD (/dev/sdb)
2. Disk 1 : 120GB Intel 520 SSD (/dev/sdc)
3. Disk 2 : 1 x 1TB HDD on Non-Raid (/dev/sda)

OS : CloudLinux 6.4
Partitioning :
/dev/sdb3 mounted as /
/dev/sdb1 mounted as /boot
/dev/sda1 (RAID 10) mounted as /mysql
/dev/sda2 (RAID 10) mounted as /home
/dev/sdc1 (SSD) partitioned and unmount (do unmount first if it already mounted, don’t forget to edit /etc/fstab after unmount)

Flashcache Installation Goals
I decide to try Flashcache to speed up /dev/sda(1-2) as this partition stored user’s file and database.

 

Flashcache Installation

1. Update the whole system to get all packages up to date :

   yum -y update;

2. Install required packages :

   yum -y install kernel-devel git make gcc-c++ device-mapper dkms make yum-utils;

3. Reboot if you got new kernel version installed during update
4. For OpenVZ folks want DKMS Support use my script. (SolusVM)
   (Please make sure you have OpenVZ kernel installed and active SolusVM)
   (DKMS help when you update kernel so FlashCache will stay)
   

   curl -sL https://dl.dropboxusercontent.com/u/32483919/openvz_flashcache_dkms.sh | bash

   CentOS 6.x x64 with Support DKMS Flashcache installation
   

   cd /usr/src;
   wget http://download.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm;
   rpm -ivh epel-release-6-8.noarch.rpm;
   yum install dkms -y;
   rpm -e epel-release-6-8.noarch;
   git clone git://github.com/facebook/flashcache.git; 
   cd flashcache;
   make -f Makefile.dkms;
   make install;

   Normal (Without DKMS) install is below

   cd /usr/src; 
   git clone git://github.com/facebook/flashcache.git; 
   cd flashcache; 
   make && make install;

5. Test with:

   modprobe flashcache;
   dmesg | tail
   

6. Fix it(google it), if you have any error.
   Don’t forget to unmount your existing mount point first, if you want to mount somewhere else.
   For example:
   i want to do flashcache for /mysql, if you didn’t mount your SSD yet, you just comment out the /mysql in fstab, otherwise you need to comment out both SSD and your destination point,
   In this case /mysql is the destination.
   Tip: Don’t forget to unmount wherever your SSD mounted to,
   because you going to need to create flashcache by command line before reboot and use init.d to auto remount.
   

   umount /mysql;
   

   and very important

   Edit /etc/fstab and give a comment out the /mysql

   

7. If you haven’t touch the SSD, you’ll need to do partitioning and format this “fast and furious SSD disk”. I spare 20GB of the disk in order to prevent SSD performance degradation and divide the 100GB disk in equal ratio with my /dev/sda1 and /dev/sda2 size : /dev/sdc1 80GB (for /dev/sda2 cache)
8. after doing partitioned your SSD, Unmount destination point and Unmount your SSD cache device
   Then you going to create your first flashcache…wheee!!!!For write back use this

   flashcache_create -p back -v -b 64k <cachename> <SSD Dev> <DISK DEV>

   Example (write back)

   flashcache_create -p back -v -b 64k mysql_cache /dev/sdc1 /dev/mapper/vhdd-mysql

   For write through use this
   

   flashcache_create -p thru -v -b 64k <cachename> <SSD Dev> <DISK DEV>

   Example (write through)

   flashcache_create -p thru -v -b 64k mysql_cache /dev/sdc1 /dev/mapper/vhdd-mysql

9. I’m going to make a writethrough cache, but https://github.com/facebook/flashcache/blob/master/utils/flashcache is only for writeback. So I’ve made a custom script, download here! (download and change it to fit your needs). Skip this to step 10
10. Let’s download auto loader and edit to whatever you figured it

    cd /etc/init.d/;
    wget -O flashcache https://dl.dropboxusercontent.com/u/32483919/flashcache;
    chmod +x /etc/init.d/flashcache; 
    chkconfig flashcache on;

11. Let’s edit /etc/init.d/flashcache one last time before we reboot.
    

Done! Just reboot, everything should be working as you hoping.

Let’s test………!

cd /your_ssd_point;
dd bs=4K count=1024400 if=/dev/zero of=test conv=fdatasync;

Should see something like this (mine is SATA2 SSD so hardly ever go beyond 200MB/Sec but hey with single HDD normally it 100-120MB/Sec

With FlashCache SSD + 1TB HDD

Without FlashCache Just Single 1TB HDD 🙁  

Leave me a question, if you have any.

and Free Assistant Setup is also available.