Skip to main content

Idempotent Atomic Counter in DynamoDB


Atomic Counters in dynamodb helps to increment the values in dynamodb server instead of performing the calculation at the client application. For example, consider a field called "quantity" with any number, in order to increment or decrement this value, traditionally a get operation should be performed to retrieve the record and the actual value has to be computed by the application and an update statement stores the latest value, instead dynamodb offers a functionality, which increments or decrements the value of the "quantity" by the provided value.

But the update statement issued to dynamodb is not idempotent. Idempotent is a property which provides the same output irrespective of processing the update statement many times. In our case, since we are providing an update statement which just increments the value, processing the same statement multiple times may result in different output, hence the operation is not idempotent.

Why an update statement should be idempotent?

Even though the update statement is issued only once to the dynamodb, the dynamodb client (SDK library) automatically retries atleast 10 times (if the retry policy is not configured) incase of any network latency or timeouts, which causes an over or undercounting as per the dynamodb documentation.


To avoid this issue, AWS suggests a conditional update by adding condition expressions to the update statement, this leads to below issues,

1. The record should be retrieved and application code should perform the computation to find the end result.

2. Application code should retry the update expression with new conditional statements incase of failure.

3. During failure, ConditionalCheckFailedException is thrown, which may be due to the dynamo db internal retry failure or due to other concurrent request trying to update the same row with new quantity, these two failures cannot be differentiated. 

For better understanding scenarios are explained using the below sequencing diagram,

ConditionalCheckFailedException due to DynamoDB retry


ConditionalCheckFailedException due to Concurrent Update

Hence, a conditional expression cannot be used to prevent the false increment of atomic counters.

How to make this Idempotent?

To make this idempotent, DynamoDB Transactions should be leveraged.

Including a client token with transactions will allow to retry a request if it fails without it being considered a new separate request. A client token lasts for 10 minutes and the same request can be retried multiple times within that period without it being considered a new request however after 10 minutes another retry would be consider a second request. Client Token will be included automatically for transactions incase of using official dynamodb SDK library.

Hence transactions can be used to precisely increment atomic counters which solves all the above issues. 

Labels:

Comments

Post a Comment

Popular posts from this blog

How to get 5GB instantly in seedr?

Seedr is a torrent caching site which provides 2GB of caching for free. For those who are not aware of what the torrent is?  Torrent is a peer to peer communication protocol (P2P) for file sharing. It was introduced in the mid of 2001 for decentralizing the file sharing in this real world. How the files are shared via torrent? Files are uploaded to servers as we do traditionally but the interesting part comes here, these files can be downloaded only via torrent client like Bit torrent, µtorrent, Vuze or you can use any other clients available in the market. Just do a google search to find all the available clients for downloading files.  Why they are restricting the download to clients? Because, the concept here is each computer acts as a server, once you start the download, the client will start uploading the data. Someone on the internet will download the same file based on the data you upload, this process is called seeding, thus decentralizing the data and reducing t
14 comments
Read more

Decision making statements - T24

As a programmer, it is essential for you to make your program to work better in all the cases. So, based on the real-time data, you have to decide how the program should work. So, it is important to take decision. Consider a simple case where you want to print the number provided by the user at runtime is even or odd. How to handle that in programs? Decision making statements comes in to play. So, what is the syntax and how to use that? IF condition THEN                 This block will execute when the if condition is satisfied or validated to true. END This is the syntax of the simple IF block. Ok. If the condition is not satisfied, then what will happen. I need to do something when IF condition is failed. How to handle that? ELSE block should be introduced. Here is the syntax, IF condition THEN                 This block will execute when the if condition is satisfied or validated to true. END ELSE                 This block will execute when the i
1 comment
Read more

Hello World - Getting Started with T24

Are you looking to develop great modules and local services for T24? This is the perfect place to start with. Let’s start from the basics. The core banking platform, T24 is developed with INFOBASIC code. This is the proprietary language of Temenos. T24 has the unique compiler to convert the BASIC code to object files with TAFc (Temenos Application Framework c) platform. An another flavour TAFj (Temenos Application Framework java), converts the BASIC to bytecode making it capable to run on any device with the Java runtime installed. The BASIC code remains same for both TAFc and TAFj Platform. Enough Intro! Let’s say hello to this world, How the syntax of this basic program looks like, PROGRAM program_name all the basic statements reside inside this block END Now, it is right time say hello, PROGRAM HELLO                 CRT “Hello World” END Yah, we did that. But, how to run this. You need T24 deployed in your organisation to compile and run
2 comments
Read more