Thread safety with Rails.cache and memcache-client

Either Rails.cache or memcache-client (or the combination of the two) is not thread safe. I was seeing symptoms like the wrong data being fetched for a given key (getting hash when expecting an array and vice versa). What’s worse is that some data even got written to the wrong keys.

At first I was going to make a SynchronizedMemCacheStore in the same spirit as SynchronizedMemoryStore in ActiveSupport, but we debated over whether a single lock would become a bottleneck. Our threaded asynchronous task processor isn’t running nearly at capacity, but maybe it will someday in the future.

So the fix was to override Rails.cache to use thread local variables.

# inside Rails::Initializer.run in environment.rb
config.after_initialize{ require "threadsafe_rails_cache" }

# threadsafe_rails_cache.rb
module Rails
  def self.cache
    Thread.current["Rails.cache"] ||= begin
      ActiveSupport::Cache.lookup_store(configuration.cache_store).tap do |cache|
        cache.logger = Rails.logger
      end
    end
  end
end

Some ramifications of this…

• RAILS_CACHE is still defined internally by Rails. Obviously you shouldn’t use it and use Rails.cache instead. I’m not sure if Rails uses RAILS_CACHE internally though.
• You’re going to have a lot more connections open to Memcached.

Here’s a nifty graph to show that the fix worked. Red indicates failed tasks, and green indicates successes. The fix was deployed around 12:00.

Versions

• Ruby 1.9.2
• Rails 2.3.10
• memcache-client 1.8.5