Apache mpm_worker and mpm_prefork with Python/Django

Although the Django documentation clearly states that mpm_prefork is the way to go we decided to give mpm_workers with mod_wsgi a try. Django 1.1.1 is thread safe so why not? Also there is some good reasons why people wants to use mpm_worker instead of mpm_prefork.

Our website get about 100 requests by second spread on 3 frontend servers. That's really not much. First it went well. Then, when traffic increased a bit, everything started to slow down. It was slowing so much that the site was almost unusable. A little bit of profiling showed that a lot of CPU time was spent into some Django's template filters. We tried to use a Django template caching technics but it didn't helped much.

At the time the mpm_worker config was something like that:

ServerLimit 16
StartServers 2
MaxClients 150
MinSpareThreads 25
MaxSpareThreads 75
ThreadsPerChild 25

Python is known to not do well with threading because of the GIL so I try to experiment a bit with these values. I try to reduce the ThreadsPerChild value and it's what I found:

Value of ThreadsPerChildAverage response time in millisecondes

The performance improvement is clear. With less threads, the CPU usage was going down and the server started to feel a lot more snappy. These numbers don't even reflect how bad the situation was with more traffic. A single server was chocking with less than 100 requests by second.


If you really have to use mpm_worker with python code, make sure to test everything under high concurrent load. If the performances are bad, reduce the value of ThreadsPerChild. I would recommend 2-5 threads maximum.