aspnet-core | api | api-management

.NET Core API Gateway Ocelot - Logging HTTP Requests & Response Including Headers & Body

Setup Ocelot API Gateway in ASP.NET Core and configure HTTP request and response logging including headers and body.

Abhith Rajan
Abhith RajanDecember 26, 2019 · 5 min read · Last Updated:

We have many microservices running under one umbrella on production which is Azure Application Gateway and has only one endpoint for all. We are using path-based rules to route to each service.

During development, we may need to simulate the same experience, i.e one endpoint instead of we referring separate URL endpoint for each service.

For the microservices, I used to refer dotnet-architecture/eShopOnContainers. And I read about the Ocelot API gateway from there. So I set up a local API gateway using Ocelot, for that

  1. Created a new ASP.NET Core 3.1 web application and choose the Empty template.
  2. Added the following NuGet packages
<PackageReference Include="Ocelot" Version="13.8.0" />
<PackageReference Include="Serilog" Version="2.9.0" />
<PackageReference Include="Serilog.AspNetCore" Version="3.2.0" />
<PackageReference Include="Serilog.Sinks.File" Version="4.1.0" />
  1. Updated the Program.cs
using System.IO;
using Microsoft.AspNetCore;
using Microsoft.AspNetCore.Hosting;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.Hosting;
using Microsoft.Extensions.DependencyInjection;
using Serilog;
using Serilog.Events;
using Ocelot.DependencyInjection;
using Ocelot.Middleware;
using Microsoft.AspNetCore.Builder;
using OcelotApiGw.Middlewares;

namespace OcelotApiGw
{
    public class Program
    {
        public static IWebHost BuildWebHost(string[] args)
        {
            var builder = WebHost.CreateDefaultBuilder(args);

            // Ocelot configuration file
            builder.ConfigureAppConfiguration(
                          ic => ic.AddJsonFile(Path.Combine("configuration",
                                                            "configuration.json")))
                    .ConfigureServices(s =>
                    {
                        s.AddSingleton(builder);
                        s.AddOcelot();
                    })
                    .UseStartup<Startup>()
                    .UseSerilog((_, config) =>
                    {
                        config
                            .MinimumLevel.Information()
                            .MinimumLevel.Override("Microsoft", LogEventLevel.Warning)
                            .Enrich.FromLogContext()
                            .WriteTo.File(@"Logs\log.txt", rollingInterval: RollingInterval.Day);
                    })
                    .Configure(app =>
                    {
                        app.UseMiddleware<RequestResponseLoggingMiddleware>();
                        app.UseOcelot().Wait();
                    });

            var host = builder.Build();
            return host;
        }

        public static void Main(string[] args)
        {
            BuildWebHost(args).Run();
        }
    }
}

Here we are configuring the Ocelot middleware as well as Serilog for logging. The Ocelot configuration file is added under “configuration” folder in the root directory.

A sample Ocelot configuration file looks like this.

configuration.json

{
  "ReRoutes": [
    {
      "DownstreamPathTemplate": "/{version}/{everything}",
      "DownstreamScheme": "http",
      "DownstreamHostAndPorts": [
        {
          "Host": "localhost",
          "Port": 5010
        }
      ],
      "UpstreamPathTemplate": "/d/{version}/{everything}",
      "UpstreamHttpMethod": ["POST", "PUT", "GET"]
    },
    {
      "DownstreamPathTemplate": "/b/{everything}",
      "DownstreamScheme": "http",
      "DownstreamHostAndPorts": [
        {
          "Host": "localhost",
          "Port": 5011
        }
      ],
      "UpstreamPathTemplate": "/a/b/{everything}",
      "UpstreamHttpMethod": ["POST", "PUT", "GET"]
    },
    {
      "DownstreamPathTemplate": "/{everything}",
      "DownstreamScheme": "http",
      "DownstreamHostAndPorts": [
        {
          "Host": "localhost",
          "Port": 5013
        }
      ],
      "UpstreamPathTemplate": "/c/{everything}",
      "UpstreamHttpMethod": ["POST", "PUT", "GET", "OPTIONS"]
    }
  ],
  "GlobalConfiguration": {
    "BaseUrl": "https://localhost:44390"
  }
}

Apart from that, we are also adding one custom middleware RequestResponseLoggingMiddleware and its purpose is clear from its name, log request and response.

The Logging Middleware

This is a slightly modified version of middleware mentioned in the article, Using Middleware in ASP.NET Core to Log Requests and Responses.

using Microsoft.AspNetCore.Http;
using Microsoft.Extensions.Logging;
using System.IO;
using System.Text;
using System.Threading.Tasks;

namespace OcelotApiGw.Middlewares
{
    public class RequestResponseLoggingMiddleware
    {
        private readonly ILogger<RequestResponseLoggingMiddleware> _logger;
        private readonly RequestDelegate _next;

        public RequestResponseLoggingMiddleware(RequestDelegate next, ILogger<RequestResponseLoggingMiddleware> logger)
        {
            _next = next;
            _logger = logger;
        }

        public async Task InvokeAsync(HttpContext context)
        {
            context.Request.EnableBuffering();

            var builder = new StringBuilder();

            var request = await FormatRequest(context.Request);

            builder.Append("Request: ").AppendLine(request);
            builder.AppendLine("Request headers:");
            foreach (var header in context.Request.Headers)
            {
                builder.Append(header.Key).Append(':').AppendLine(header.Value);
            }

            //Copy a pointer to the original response body stream
            var originalBodyStream = context.Response.Body;

            //Create a new memory stream...
            using var responseBody = new MemoryStream();
            //...and use that for the temporary response body
            context.Response.Body = responseBody;

            //Continue down the Middleware pipeline, eventually returning to this class
            await _next(context);

            //Format the response from the server
            var response = await FormatResponse(context.Response);
            builder.Append("Response: ").AppendLine(response);
            builder.AppendLine("Response headers: ");
            foreach (var header in context.Response.Headers)
            {
                builder.Append(header.Key).Append(':').AppendLine(header.Value);
            }

            //Save log to chosen datastore
            _logger.LogInformation(builder.ToString());

            //Copy the contents of the new memory stream (which contains the response) to the original stream, which is then returned to the client.
            await responseBody.CopyToAsync(originalBodyStream);
        }

        private async Task<string> FormatRequest(HttpRequest request)
        {
            // Leave the body open so the next middleware can read it.
            using var reader = new StreamReader(
                request.Body,
                encoding: Encoding.UTF8,
                detectEncodingFromByteOrderMarks: false,
                leaveOpen: true);
            var body = await reader.ReadToEndAsync();
            // Do some processing with body…

            var formattedRequest = $"{request.Scheme} {request.Host}{request.Path} {request.QueryString} {body}";

            // Reset the request body stream position so the next middleware can read it
            request.Body.Position = 0;

            return formattedRequest;
        }

        private async Task<string> FormatResponse(HttpResponse response)
        {
            //We need to read the response stream from the beginning...
            response.Body.Seek(0, SeekOrigin.Begin);

            //...and copy it into a string
            string text = await new StreamReader(response.Body).ReadToEndAsync();

            //We need to reset the reader for the response so that the client can read it.
            response.Body.Seek(0, SeekOrigin.Begin);

            //Return the string for the response, including the status code (e.g. 200, 404, 401, etc.)
            return $"{response.StatusCode}: {text}";
        }
    }
}

Usually, the request body can be read only once. Here we are making use of the new EnableBuffering extension method to read the request body multiple times. You can read more about the feature here, Re-reading ASP.Net Core request bodies with EnableBuffering().

That’s it, run your projects, make some requests and then check the logs directory.

This logging middleware can be used in any other ASP.NET Core project.

Here we used in the Ocelot API Gateway so that we don’t need to check the logs of each microservices.

Bonus

I am using a free tool Tailviewer to go through my logs. Check it out, it is small in size also an opensource project.

This page is open source. Noticed a typo? Or something unclear?
Improve this page on GitHub


Abhith Rajan

Written byAbhith Rajan
Abhith Rajan is a software engineer by day and a full-stack developer by night. He's coding for almost a decade now. He codes 🧑‍💻, write ✍️, learn 📖 and advocate 👍.
Connect

Is this page helpful?

Related ArticlesView All

Related VideosView All

Why I DON'T use MediatR in ASP.NET Core

Don't Use AutoMapper in C#! Do THIS Instead!

Exploring Source Generation for Logging

Related StoriesView All

Related Tools & ServicesView All

flurl.dev

Flurl

Flurl is a modern, fluent, asynchronous, testable, portable, buzzword-laden URL builder and HTTP client library for .NET.
httpstatuses.com

HTTP Status Codes — httpstatuses.com

HTTP Status Code directory, with definitions, details and helpful code references.