Last year, I wrote about speeding up django unit tests. With this method, I’ve been able to significantly reduce the time it takes to run my unit tests.
But I still have a real short attention span. If anything, it’s shorter than it was last year. I also have an additional problem: I tend to create databases that have lots of preloaded data. This means that when I run tests or when I’m doing model development, I have to wait for all that data to load. Waiting sucks. Plus, in order to be able to test with the SQLite database, I have to keep my SQL compatible with both the MySQL and sqlite engines (usually not a problem, but sometimes it can be).
Solution? I put my database on a RAM disk. I now have full speed unit tests. If I make a model change, I can toss the whole database and reload everything instantly. When I’m running unit tests, I can run them against the same database engine that I use in production.
Current Method (still a bit rough)
Load up your favorite RAM disk software. I don’t have a favorite yet, but so far I’m having good luck with the free one from http://www.romexsoftware.com.
Create yourself a RAM disk, put it on your favorite drive letter (I use R:\). The disk doesn’t have to be large, since there probably won’t be tons of data on it. I only have 2GB of memory on my laptop, so I made a dainty 64MB RAM disk. (Note: if you are running 32-bit windows and have more than 3GB of RAM, you might be able to make your unusable memory into a RAM disk .. there are some details on the site above).
Make yourself a “Create Ramdisk MySQL.cmd” batch file (see below). Edit the necessary variables so they’ll work on your system, then run the file. Since the contents of ram disks are frequently lost (like when you reboot), the setup here needs to be automated. The batch file does 3 things. It creates the needed folders on the ram disk. It copies over your “mysql” database (so your passwords will still work). And it installs a Windows service for the ramdisk mysql (called MySQLR).
Make a copy of your my.ini file called my_ramdisk.ini (or whatever you like). Edit this new file and change the appropriate data directories to R:\mysql_volatile\data and R:\mysql_volatile\innodb.
If you need to have your normal mysql server running, then alter the my_ramdisk.ini so so the server runs on a different port (you will also have to alter the port number in your settings.py). I only use the mysql server on my computer for development, so can just turn it off before starting the ramdisk mysql.
Start up the ram disk mysql. After running the batch file, you’ll have a new service called MySQLR. Use the service control manager gui, or, since you’re going to be on the command line to run your unit tests anyway, just type
sc start mysqlr.
And finally, connect to your new mysql server and create a database for your project.
Listing: “Create Ramdisk MySQL.cmd”
REM path to the ramdisk mysql config file
REM ramdisk drive letter
REM datadir is your normal mysql data folder
REM path to your the mysqld-nt binary
rem copy mysql database to retain permissions
xcopy /ei %datadir%\data\mysql %ramdisk%\mysql_volatile\data\mysql
rem install a service for the ramdisk mysql
%mysqld% --install MySQLR --defaults-file=%config%
Remove the MySQLR service by running
mysqld-nt.exe --remove MySQLR.
You might want to alter the batch file to copy the django application’s database to the ramdisk as well (saves you having to run that step manually).
Unit tests with the RAM disk MySQL is around 30 times faster than normal MySQL, and only about a second and a half slower than using the SQLite backend.