Improving your user experience with Edge Caching

In the ever-evolving landscape of web and mobile applications, speed and efficiency are paramount to deliver seamless user experiences. As businesses mature and expand, they are often confronted with a paradoxical scenario where growth, a sign of success, ushers in an era of increased technological complexity and scalability concerns.

At Grafbase, we aim to help with these concerns by providing a feature rich GraphQL Edge Gateway that unifies multiple data-sources into a single API with speed and resilience at its core.

Caching is crucial for enhancing data retrieval latency and optimizing user experience. It reduces the round trip time and network strain while ensuring quicker response times. In the digital age, where immediate, seamless access to information is expected, effective caching is indispensable. Beyond user experience, caching can lessen the load on origin servers, improving their performance, making it a key strategy for efficient and cost-effective digital operations.

Edge caching, an integral facet of edge computing, involves caching data closer to where it was requested, at the edge of the network. By placing cached content in strategic locations across the globe, edge caching reduces the round-trip time for data retrieval and significantly improves response times.

With a developer experience first mindset, you can effortlessly enable caching for specific GraphQL queries, or types, and focus on building exceptionally performant applications without the complexities of cache management.

Grafbase Edge Caching is globally available and strategically performed in independent edge locations. This approach makes it extremely efficient as it provides access to data based on proximity. However, an important aspect to consider, is that cached data does not replicate across edge locations. It is possible that, the same request, handled in different locations may return different results for the same cache key. Or not return any data at all. Ultimately, cached data depends on the origin response and when it was requested.

When handling traffic in our Edge Gateway, we verify if your deployment is leveraging cache. If it is, we deterministically compute the cache key for the request and check if there's any entry in cache that corresponds to the computed key.

The cache key plays an important role and it's a combination of several attributes of the request:

• GraphQL query
• GraphQL variables
• Authentication
• Hostname

If @cache is not used, caching will not be in your request's path and will be forwarded directly to the origin.

Now that we've briefly covered how caching works, let's move forward to how long data remains cached.

When using @cache, you have to specify at least the maxAge attribute in order to define how long a value will remain in the cache and therefore served to incoming requests. Any request that yields the same cache key during this period will see a HIT in the x-grafbase-cache response header.

The following snippet shows how to cache any Post for 60 seconds.

import { config, graph } from '@grafbase/sdk'

const g = graph.Standalone()

g.model('Post', {
  title: g.string(),
  slug: g.string().unique(),
}).cache({
  maxAge: 60,
})

export default config({
  graph: g,
})

A recurrent scenario we encounter is serving a stale cached entry that is being refreshed in the background. To control how long an entry can remain in cache but considered stale, use the optional staleWhileRevalidate attribute. Any request that targets these entries, will experience the following when maxAge has elapsed:

• an UPDATING in x-grafbase-cache
• cache entry refresh by issuing a request to the origin in the background
• any request for the same key while its is being updated will receive a STALE in x-grafbase-cache

This pattern is extremely useful if you can afford the reduced time window of stale data. It keeps low latency consistent while data is refreshed in the background.

The following snippet shows how to cache any Post for 60 seconds and let them remain in cache for an additional 60 seconds for stale behaviour.

import { config, graph } from '@grafbase/sdk'

const g = graph.Standalone()

g.model('Post', {
  title: g.string(),
  slug: g.string().unique(),
}).cache({
  maxAge: 60,
  staleWhileRevalidate: 60,
})

export default config({
  graph: g,
})

You can find more about the other cache statuses here.

Bypassing the cache becomes necessary when cached data is outdated and real-time information is required. By doing so, users can access the latest data directly from the origin server, ensuring accuracy and avoiding potential issues caused by serving stale content.

If you find yourself in need to bypass the cache entirely, you can do so by leveraging the header Cache-Control in the request:

• Cache-Control: no-cache: The no-cache request directive instructs the Edge Gateway to validate the response with the origin server before reuse. It allows clients to request the most up-to-date response even if the cache has a fresh response.
• Cache-Control: no-store: The no-store request directive instructs the Edge Gateway that it shouldn't store the origin response in the cache. Even if the origin server's response could be stored.

You can use both if you wish to completely avoid cache.

While caching brings significant benefits it also poses challenges, such as maintaining cache consistency and handling complex cache invalidation strategies. Cache purging is a critical process in managing cached content to ensure data accuracy and freshness.

At Grafbase, we provide two ways of purging your cached content:

• Mutation invalidation
• Manual

It's important to highlight that purging is done globally, across all edge locations. Although this feature empowers you with full control over your cached content, use it carefully as you might flood your origins.

Purging by mutation is a powerful strategy that will invalidate any cached entry that has certain tags. When using it, our Edge Gateway will inspect the responses to your GraphQL queries and extract the necessary data to decorate cache entries with the appropriate tags.

To use it, all that is necessary is use the optional mutationInvalidation attribute and the mutation response will be used to purge any entry that references the mutated type.

There are three different mutationInvalidation variants available:

• entity: Mutations for the targeted type will invalidate any cache entry that has the chosen field. Uses id by default.
• list: Mutations for the targeted type will invalidate any cache entry that has lists of the type in them
• type: Mutations for the targeted type will invalidate any cache entry that has the type in them

The following snippet shows how to configure @cache with mutationInvalidation.

import { config, graph } from '@grafbase/sdk'

const g = graph.Standalone()

g.model('Post', {
  title: g.string(),
  slug: g.string().unique(),
}).cache({
  maxAge: 60,
  // mutationInvalidation: "entity"
  mutationInvalidation: {
    field: 'title',
  },
})

export default config({
  graph: g,
})

You can learn more about mutation invalidation here.

This method of purging is driven by external action, you will need to actively make a request in order to remove values from the cache. In your Grafbase deployment, there’s an admin GraphQL API available under /admin that allows administrative operations. You can access it via: subdomain.grafbase.app/admin.

This endpoint is subject to API Key authentication, any request sent to it requires a valid api key to be used in the header: x-api-key: grafbase-api-key.

You can request the introspection query and use your favourite GraphQL client to explore.

Presently, you have two mutations at your disposal to purge cached entries:

• cachePurgeTypes: Allows purging specific entries by following the same pattern as the mutationInvalidation attribute (entity, list, type).
• cachePurgeAll: Allows purging the entire cache for your subdomain.

Global purging is a powerful and useful tool but use it thoughtfully. If your servers are not ready to handle the amount of traffic or take some time to scale up, your users will notice. The effects of a global purge are fast but not immediate, it may take a few seconds at most to be noticeable.

When levering the entity purging method, either through manual purging or mutation invalidation, keep in mind that we have some limits in place:

• Responses provide the necessary data that is used to decorate the cached entries with tags. These tags are used on purges and the current limit is 16Kb worth of tags. Anything above that threshold is ignored.
• These tags can only contain UTF-8 encoded characters and cannot contain white spaces
• Tags are case-insensitive. For example, Tag1 and tag1 are considered the same.

Let's move on to something more practical and experience the benefits of caching.

The following will be a small, but caching wise interesting, Grafbase deployment.

It's a rather simple schema that will extend the Query type with a new field that uses a really slow @resolver.

Create the following structure in a GitHub repository and deploy it via Grafbase.

.
  └── resolvers
      └── slow.js
  └── grafbase.config.ts

// resolvers/slow.js
export default function Resolver(_, { seconds }) {
  return fetch(`https://hub.dummyapis.com/delay?seconds=${seconds}`)
}

Once the deployment is done, you should have a fully functioning deployment. Let's issue some requests to make sure it is indeed slowly working.

GRAFBASE_API_URL="https://<REPLACE_ME>.grafbase.app/graphql"
GRAFBASE_API_KEY="<REPLACE_ME>"

curl -v "$GRAFBASE_API_URL" \
  -H 'accept: application/json' \
  -H 'content-type: application/json' \
  -H "x-api-key: $GRAFBASE_API_KEY" \
  --data-raw '{"operationName":"cacheExample","query":"query cacheExample {  slow(seconds: \"1\")}"}' \
  --compressed

If the request was successful you should see the following response:

{ "data": { "slow": "This response has been delayed for 1 seconds" } }

If you measure the above request, you'll notice that it takes at least 1 second. If you look closely to the query being sent, it's the value we're passing in the seconds argument. Inspecting the response headers, you might also notice the absence of x-grafbase-status because we haven't added any cache configuration.

Now, let's make that request much more interesting by leveraging the @cache directive. Change your schema.graphql to:

extend type Query {
  slow(seconds: String): String! @resolver(name: "slow") @cache(maxAge: 60)
}

Notice that the addition of @cache is the only change.

If you issue the same request when the deployment finishes successfully, you'll notice that you should now be getting x-grafbase-status: MISS. This tells us caching is being used, and we attempted to retrieve a cached response for the request but there was none to be served. Issuing the same request again should give you a x-grafbase-status: HIT and a much quicker response.

The following chart highlights the before and after caching. Notice the sharp drop from the constant 1s to below 50ms.

In this post we talked about Edge Caching and how Grafbase leverages it to provide customers extremely fast response times. Additionally, we covered some of the features available through @cache and how easily it is to speed up a slow API. Stay tuned for the next post about caching where we will cover how it can protect your servers and how Grafbase recently introduced changes that mitigate traffic surges when cache keys expire.

For more information about the Grafbase platform, read our documentation and changelog. Follow us on Twitter, GitHub and join our Discord. We're always open for constructive feedback and hearing your ideas!