Design Correlation ID for Distributed Tracing

Abhilash Jain
Abhilash Jain
5 min read
Posted on August 08, 2022
Design Correlation ID for Distributed Tracing

As of now most of the tech companies are moving to microservice architecture from monoliths. It comes with some obvious challenges, one of them is maintaining the flow of each request that goes to different microservices. In other words, how to maintain the trace of each request throughout its lifecycle. In the early stages it may be required, but as your system grows bigger and gets too much traffic then it might become cumbersome to debug if any exception occurs or a problem arises. The reason is quite simple:

  • Which microservice is getting affected?

  • To which request that problem belongs to?

  • What is the origin of that request?

If we have answers to the above-mentioned questions, we can easily debug and trace the exception. But the question is how can we connect the broken parts and fit these under one umbrella. The answer is Correlation ID.

 

Design

We need to generate unique ID every time a request lands into our system. Let’s assign this term as x-correlation-id. If we do a deep dive into how we can introduce correlation IDs into an existing system, a system where there are many downstream services. Let's break down some of the complex thoughts into blocks, while keeping in mind that this correlation-id is needed to group each request with all it's connected downstream services. There are four use cases that may arise:

  1. Downstream Services

  2. Cron Jobs/Task Schedulers

  3. Downstream External APIs

  4. Façade/BFF Layer

The flow starts from the user request, and we need to check at the interceptor layer whether any x-correlation-id has been passed in the header to our microservice. If yes, then we can use the same ID or else we need to generate new unique ID (It could be some UUID or any random ID of reasonable length). Now, save it in your thread local or in mapped diagnostics context and make sure to clean it as soon as it goes out of your system.

 

LogBook Changes


    %d{HH:mm:ss.SSS} [%thread] %X{x-correlation-id}%-5level %logger{5} - %msg%n
   

Log message: 02:31:44.419 [http-nio-8090-exec-4] diBJCxGz6A INFO


    %d{HH:mm:ss.SSS} [%thread] %mdc %-5level %logger{5} - %msg%n
   

Log message: 13:40:58.249 [http-nio-8090-exec-9] X-correlation-id=SIX5aEkaFZ ERROR

Zalando Logbook Changes

Adding Dependency

    org.zalando
    logbook-json
    2.1.0

Logbook Config

@Configuration
public class LogbookConfig {
     @Bean
    Logbook logbook() {
        return  Logbook.builder()
                .correlationId(new CustomCorrelationIdServiceImpl())
                .sink(new DefaultSink(new JsonHttpLogFormatter(),
                        new DefaultHttpLogWriter()))
                .build();
    }
}

 

Generating Correlation ID

You must override the existing correlation ID if you wish to generate your own.

public class CustomCorrelationIdServiceImpl implements CorrelationId {
     @Override
    public String generate(HttpRequest request) {
       ------YOUR_CODE_HERE------
    }
}

In the end pass your x-correlation-id in your response header, so that this could be logged in other external services. This could make it easier to trace and debug.