Using Memcache server as Apache content cache

With memcached and mod_memcache_cache we can use the memcached server as a content cache. This allows us share content of cache between different servers.

Into Apache is available the mod_cache module, that allows to realice this task throw different storage modules. Apache has 2 built in:

  • mod_disk_cache: stores and fetchs cached content in hard disk
  • mod_mem_cache: cache’s data in the running httpd process memory

But at this article I want to introduce mod_memcache_cache, that uses an memcached server to store data, so the available memory is the sum of all the nodes this daemon has. Thus, by increasing the memory size of Memcached server, Apache will has more memory to cache contents. The main topic is that this Memcache server can be placed on other machine with a lot of memory and serving contents to more than 1 server, acting as a central repository of cached content listening to clients that want to store something in key-value structure.

I assume you have an memcached server listening on port 11211 on the same machine so let’s install all the stack to test. First of all we need to have installed Apache 2.2 with mod_cache installed. So download the apache 2.2 source code and compile it with the next steps:

# ./configure --prefix=/usr/local/apache22 \
              --exec-prefix=/usr/local/apache22 \
              --enable-so --enable-rewrite --enable-cache
# make && make install

mod_memcached_cache uses apr_memcache routines for interacting with memcached servers. apr_memcache is included in newer development branches of APR-Util, but is not available in the current released versions. So let me install the apr_memcache library from source code:

# wget http://www.outoforder.cc/downloads/apr_memcache/apr_memcache-0.7.0.tar.bz2
# tar xvjf apr_memcache-0.7.0.tar.bz2 && cd  apr_memcache-0.7.0/
# ./configure --with-apr=/usr/local/apache22/ \
             --with-apr-util=/usr/local/apache22/ \
             --prefix=/usr/local/ \
             --exec-prefix=/usr/local/
# make && make install

We can now install the memcache module so:

# wget http://modmemcachecache.googlecode.com/files/mod_memcached_cache-0.1.0.tar.bz2
# tar xvzf mod_memcached_cache-0.1.0.tar.bz2 && cd mod_memcached_cache-0.1.0
# CFLAGS="-I/usr/local/src/httpd-2.2.9/modules/cache/" \
./configure --with-apxs=/usr/local/apache22/bin/apxs \
             --with-apr-memcache=/usr/local/
# make && make install

Finally we must include into our Virtualhost the next configurations to use the apache’s memcache module

LoadModule memcached_cache_module modules/mod_memcached_cache.so
<IfModule mod_memcached_cache.c>
        CacheEnable memcached /
        MemcachedCacheServer localhost:11211
        MemcachedMaxServers 10
        MemcachedMinConnections 10
        MemcachedSMaxConnections 10
        MemcachedMaxConnections 10
        MemcachedConnectionTTL 10
        MemcachedCacheMinFileSize 1
        MemcachedCacheMaxFileSize 2097152

        CacheDisable /admin/
</IfModule>

5 Comments

  1. Oscar

    Despois de descargalos paquetes con wget fáltache destarealos. Xa sei que todo o mundo o debería saber, pero como o pos de corrido algún copia e pega e non se da conta

  2. admin

    Thanks Oscar for the feedback!

    I’ll add and fix what you are saying. But I have to say that this server config is not trivial thus “only” sysadmins should do this.

    Bye

  3. opyate

    Hi Fran,

    I’ve had a look at mod_memcache_cache’s sources and googlecode pages, and can’t find where to configure the strategy for creating the memcache key.

    I left a comment on the googlecode page (http://code.google.com/p/modmemcachecache/wiki/Configure) which I’ll re-iterate here:

    -START-
    Please make it a function of the request URI and the Accepts header (perhaps also per-method, e.g. only cache a response when GET is used).

    So,

    memcached key = md5(URI + Accepts) memcached value = response entity body (XML or JSON representation, for example)

    This function will generate different keys for different response types (e.g. XML or JSON).
    -END-

    Perhaps you found the solution? I guess I have to install it and play around to deduce the answer.

    Thanks,
    Juan

  4. admin

    Hi opyate,

    I’ll study the memcache apache module this weekend but the previous week I taken a look and It’s just a line on the code, very clean indeed.

    Thanks for the comment

  5. grimfromghent

    Hi,

    I’m not sure why, but there seems to be something wrong with the way this mod stores/reads its values from memcached servers. As long as the time between a read and a store is long enough, everything goes fine.

    That’s why I made a script to download the images (jpeg) at fast speed (over localhost on the server). What did I see?

    iterate over a list of DIFFERENT images: everything fine
    iterate over a list of SAME images: MIME-type of the images not correct
    (after the second iteration, the MIME-type was wrong)

    I wrote the value to a logfile, and then changed the extension to see if my
    explorer could open the image. But NO. Looking in this so called image,
    the data seemed to be incorrect (no JFIF header for example). At other moments, the header was correct, but only a part of the image was stored.

    Does this have something to do with the maximum file size of every object?
    (I thought the mod doesn’t store the values, if they higher than this?)
    That could explain why the data is truncated.

    I also disabled UDP, with no affect. I even changed the number of threads in memcached to 1 (I thought that one apache thread was already reading, while the other was still writing), but it does not make a difference (apart from the performance).

    P.S. IIPImage also uses memcached, and doesn’t have this problem.
    (see http://iipimage.sourceforge.net/blog/)

    Any idea?

    thanks in advance!

Leave a Reply