Remigiusz ZalewskiRemigiusz Zalewski

Memory cache in .NET

memory-cache-in-dotnet
Sponsor this newsletter →

Introduction

Calls to external APIs or to the database are slow, and if every request pays that price, those calls become the bottleneck of your service. The simplest fix is to keep the result in the web server's memory with IMemoryCache from Microsoft. This video walks through how the cache mechanism works, the two kinds of expiration, and then adds caching to a real API that fetches commodity prices from an external service.

🎬 Watch the full video here:

How the cache mechanism works

Without a cache, the flow looks like this: the client calls your API asking for some resource (say, the population of a specific city), the API goes to the database, gets the data and returns it. Every single request pays for that database call.

With a cache, the data is stored under a cache key - it can be anything, in this example it's the city name. On the next request with the same key, the API returns the data straight from memory and skips the database call entirely. One fewer round trip, and a big jump in performance.

Absolute vs. sliding expiration

There are two types of expiration time:

  • Absolute - the entry is removed when the time is reached, no matter how often it was requested in the meantime. It is never extended.
  • Sliding - if the entry is requested before the time runs out, the countdown resets and the entry lives for another full period. It is evicted only once it goes unused.

The starting point: a slow API

The demo app returns the current price of commodities (wheat, corn, ...) using the Commodities API (you can get a free API key there). The controller calls a service, the service calls the external API, deserializes the response and returns a DTO to the user. The API key and URL are injected using the options pattern from a StockPriceApi section in appsettings.json.

Calling it from Swagger for wheat and corn works, but it's slow - and it doesn't need to be, because the price of corn doesn't change every second.

Step 1: register the memory cache

One line in Program.cs, before builder.Build():

builder.Services.AddMemoryCache();

Step 2: inject IMemoryCache and build a cache key

Inject IMemoryCache into the service through the constructor. Then add a small method that builds a single cache key for a given set of resources:

private static string GetCacheKey(IEnumerable<string> resources)
    => string.Join("-", resources);

Joining the resource names gives one key per combination, so a request for the same pair of resources hits the same cache entry.

Step 3: wrap the call in GetOrCreateAsync

public async Task<ResourceDto> GetStockPriceAsync(List<string> resources)
{
    return await _cache.GetOrCreateAsync(GetCacheKey(resources), async entry =>
    {
        entry.SlidingExpiration = TimeSpan.FromMinutes(2);

        // existing logic: call the Commodities API,
        // deserialize the response and build the DTO
        return await FetchFromApiAsync(resources);
    });
}

GetOrCreateAsync checks the cache first and only runs the factory on a miss, storing the result before returning it. The existing code moves inside the factory unchanged. Sliding expiration fits here because the price doesn't change quickly - as long as the resource keeps being requested, it stays cached.

The result

The first call is as slow as before, because the cache is empty and the external API has to be called. Calling the same endpoint again returns in about 8 milliseconds. Every following request for the same resource comes from the cache, and each one extends the expiration because of the sliding window.

Common pitfalls

  • Registering the service too late. AddMemoryCache() must be called before builder.Build(), otherwise the app fails to start.
  • A cache key that's too coarse or too specific. The key must identify exactly the data being cached - here, the full list of requested resources.
  • No expiration at all, letting the cache grow unbounded over time.
  • Using only absolute expiration for hot data, which evicts it on schedule even though it's being used constantly. Prefer sliding when "keep it while it's used" is the goal.
  • Assuming the cache is shared across instances. IMemoryCache lives in one process's RAM; with multiple instances behind a load balancer, each has its own cache.

Key Takeaways

  • Slow external API and database calls are bottlenecks; caching their results in memory removes the round trip.
  • AddMemoryCache() plus an injected IMemoryCache is all the setup needed.
  • GetOrCreateAsync with a well-chosen cache key is the idiomatic pattern - compute only on a miss.
  • Absolute expiration evicts at a fixed time; sliding expiration extends on every access.
  • In the demo, caching dropped a slow external call to roughly 8 ms on repeated requests.

Get the Full Source Code

The complete runnable solution - the commodities API service with IMemoryCache wired in - is available to Patreon supporters. If you want it ready to drop into your own project instead of rebuilding it from the walkthrough above, you can find it on Patreon.

Resources