Idempotency 101
Why, What-for and light introduction with a dotnet webapi.
TL;DR => Go straight to the code :) !
Describing the problem
Idempotency has been a hot discussion topic at my current company (https://www.ohpen.com). We are a fintech company that, among other things, manages large loads of payments and transactions.
From a product point of view, we are asked to design our systems to be idempotent but, for some reasons out of our control, this is not always possible. Either because we inherit a service from another team, we merge 2 teams or we simply have technical debt. Also, as any other company, we need to deal with legacy systems and those are not always idempotent.
So, basically we had to come up with a solution that complies with:
- Repeated write requests (POST, PUT, …) provide the same response.
- Repeated write requests do not load the system. Meaning that, if you do receive several POSTs, the system should respond fast without spending resources and time. In other words, the request should not travel along the system.
- Since we are managing all sorts of systems, we can not depend on any specific implementation. The solution should integrate with any system without modifying it and should require the minimum effort.
- It could happen also that client duplicates a transaction by mistake and sends it twice. We also need to be prepared for that, so we need a fast and distributed system that blocks the SAME requests while the previous one is being processed.
But, … what does it mean “the same request”? I mean, if it is sent later, then it is not the same, right? Not exactly, … I think this is better explained with an example.
Let’s say you have a website where you sell products and you allow your users to pay from that website. A user is about to buy an item but just after clicking the “buy” button he refreshes the page by mistake (or clicks the button twice, who knows …).
In this scenario, we want a middleman (let’s call it idempotency) that:
- intercepts request,
- manages their execution based on a unique key (provided by the website),
- delegates to the actual backend the request processing when needed,
- and blocks duplicated requests.
Implementation in a dotnet webapi
Let’s get concrete. For the sake of this example, I’ll be using an in-memory idempotency storage.
We have an api that allows to pay with transactions. Here you can find the controller with the single endpoint. Note that in line 15 I’m adding an Asynchronous action filter called IdempotencyFilter.
The service, for this example, is dummy. Contains a single method ProcessAsync(Transaction transaction) that simulates a long running process. It returns the processed transaction id once finished. Note that I included an await Task.Delay(1000). This will allow me to properly test concurrent requests.
In the Startup.cs class, I’m injecting all the necessary elements. As you can see, for this demo, I’m using an in-memory storage (line 6) for the idempotency part.
The most important part of the Idempotency project is the filter. It is responsible for:
- Reading the idempotency key from the header.
- Linking the current request execution with the provided key.
- Caching responses when needed.
- Delegating the execution to the actual API when needed.
Note that, to integrate the idempotency service with our API, we don’t need to change any part of our implementation.
Let’s test it!
- Use case: Missing idempotency key
This use case is quite straightforward. If an endpoint is tagged to support idempotent operations, then the x-idem-key becomes mandatory. Not providing it will result in a bad-request response.
- Use case: Provided idempotency key.
In this case, the api processes the request, spends some time on it (remember the Task.Delay, and responds with the new transactions’ id.
During this process, the filter intercepts the request, assigns it to the current execution context (line 4), let’s the api process the transaction (line 6), and intercepts and caches the response (line 8).
Also, after some time, the idempotency storage deprecates the cached response (line 14).
- Use case: Duplicated request
Let’s say we receive a duplicated request (same idempotency key) before the previous one finished to process. In that case, the idempotency filter detects it and responds back with a 409-conflict response type.
In line 14 we see the log of the imdempotency filter. At that moment, another request context is owning the idempotency key and the new request context gets denied the ownership, leading to conflict.
Conclusions
- Idempotency is a necessary feature in systems that deal with highly sensitive data and operations. That sounds very exquisite but most of the systems require that. Games, financial transactions, payments, orders in an ecommerce, … the list goes on and on.
- To make your system idempotent, you can design to be so. That is definitely a best practice since the impact of any new/duplicated request becomes none and you are free to process messages in your system as many times as needed.
- In this article we focus on a specific scenario where idempotency is needed. This is for the most of already existing companies that have legacy (or simply non-idempotent) systems running in production and want to make them idempotent. In this scenario, we proposed a thin layer that acts on top your api and manages the ownership of each request’s execution.
- For the specific implementation, we proved it to be very simple and the actual api implementation didn’t change. That way, we proved to be a valid solution meeting all the requirements.
Happy to hear your feedback about this approach and Idempotency in general :). Do you see any improvements that you’d do? Let me know! #HappyCoding
