Scaling Microservices: A Robust Node.js API Gateway for Authentication & Rate Limiting

1. Introduction & The Problem

The allure of microservices is undeniable: independent deployments, technological freedom, and enhanced agility. However, as applications grow and the number of microservices proliferates, a significant challenge emerges. How do clients interact with dozens or even hundreds of distinct services? More critically, how do you manage cross-cutting concerns like authentication, authorization, rate limiting, logging, and dynamic routing consistently and efficiently across this distributed landscape?

Without a centralized approach, development teams often find themselves duplicating effort, implementing security policies inconsistently, and struggling with complex network configurations. This fragmentation leads to:

Inconsistent Security Policies: Each service might handle authentication and authorization differently, creating vulnerabilities.
Duplicated Effort: Implementing common functionalities (e.g., rate limiting) in every service is redundant and error-prone.
Performance Bottlenecks: Manual routing and lack of load balancing can strain individual services.
Operational Overhead: Managing, monitoring, and debugging a myriad of endpoints becomes a significant burden.
Client Complexity: Clients need to know about and manage multiple service endpoints and their unique interaction patterns.

These issues not only degrade developer experience but directly impact user experience through slower response times, potential security breaches, and increased operational costs. The solution lies in implementing a robust API Gateway.

2. The Solution Concept & Architecture

An API Gateway acts as a single entry point for all client requests into your microservice ecosystem. It's a facade that encapsulates the internal system architecture, providing a simplified and consistent interface to external clients. This centralization allows you to offload cross-cutting concerns from individual microservices, letting them focus purely on business logic.

Key responsibilities of an API Gateway include:

Request Routing: Directing incoming requests to the appropriate microservice based on predefined rules.
Authentication & Authorization: Verifying client identities and ensuring they have the necessary permissions before forwarding requests.
Rate Limiting: Protecting services from abuse and overload by limiting the number of requests clients can make within a time frame.
Load Balancing: Distributing incoming traffic across multiple instances of a service.
Caching: Storing responses to reduce the load on backend services and improve response times.
Logging & Monitoring: Centralizing request logging and metrics collection.
Protocol Translation: Adapting client-friendly protocols (e.g., REST) to internal service protocols.

Our proposed architecture involves a Node.js API Gateway sitting between clients and various microservices. Node.js is an excellent choice for an API Gateway due to its non-blocking, event-driven I/O model, making it highly efficient for handling a large number of concurrent connections and acting as a reverse proxy. We'll leverage Express.js for our server and `http-proxy-middleware` for efficient request forwarding. For scalable rate limiting, we'll integrate Redis.

High-Level Architecture:

RUST

Client <--> API Gateway (Node.js/Express) <--> Microservice A
           |                                 |--> Microservice B
           |                                 |--> Microservice C
           |--> Redis (for Rate Limiting)

3. Step-by-Step Implementation

Let's build a practical Node.js API Gateway that handles dynamic routing, JWT authentication, and Redis-backed rate limiting.

3.1. Project Setup

BASH

mkdir api-gateway
cd api-gateway
npm init -y
npm install express http-proxy-middleware jsonwebtoken ioredis dotenv

Create a `.env` file for configuration:

INI

PORT=3000
JWT_SECRET=your_super_secret_jwt_key
REDIS_HOST=127.0.0.1
REDIS_PORT=6379
RATE_LIMIT_WINDOW_MS=60000  # 1 minute
RATE_LIMIT_MAX_REQUESTS=100 # 100 requests per minute

3.2. Basic Gateway Structure and Dynamic Routing

First, let's set up the Express server and configure dynamic routing using `http-proxy-middleware`.

Create a `routes.json` file to define our microservice routes:

CODE

[
  {

JSON

[
  {
    "prefix": "/api/v1/auth",
    "target": "http://auth-service:4001",
    "authRequired": false,
    "rateLimit": { "windowMs": 60000, "max": 20 }
  },
  {
    "prefix": "/api/v1/users",
    "target": "http://user-service:4002",
    "authRequired": true,
    "rateLimit": { "windowMs": 60000, "max": 100 }
  },
  {
    "prefix": "/api/v1/orders",
    "target": "http://order-service:4003",
    "authRequired": true,
    "rateLimit": { "windowMs": 60000, "max": 60 }
  }
]

3.3. Gateway Core Implementation (`src/gateway.ts`)

Here is the complete TypeScript implementation of our centralized API Gateway, incorporating JWT authentication, distributed sliding-window rate limiting with Redis, and proxy forwarding:

TYPESCRIPT

import express, { Request, Response, NextFunction } from "express";
import { createProxyMiddleware } from "http-proxy-middleware";
import jwt from "jsonwebtoken";
import Redis from "ioredis";
import crypto from "node:crypto";
import routes from "./routes.json";

const app = express();
const redis = new Redis({
  host: process.env.REDIS_HOST || "127.0.0.1",
  port: Number(process.env.REDIS_PORT) || 6379,
});

const JWT_SECRET = process.env.JWT_SECRET || "default_gateway_secret";

// 1. Correlation ID Middleware for Distributed Tracing
app.use((req: Request, res: Response, next: NextFunction) => {
  const correlationId = req.header("X-Correlation-Id") || crypto.randomUUID();
  req.headers["x-correlation-id"] = correlationId;
  res.setHeader("X-Correlation-Id", correlationId);
  next();
});

// 2. Sliding Window Rate Limiting with Redis Sorted Sets
function createSlidingWindowLimiter(windowMs: number, maxRequests: number) {
  return async (req: Request, res: Response, next: NextFunction) => {
    const clientId = req.ip || "unknown_client";
    const key = `rate_limit:${req.baseUrl}:${clientId}`;
    const now = Date.now();
    const windowStart = now - windowMs;

    try {
      // Redis pipeline: Remove old entries, add current timestamp, count active window
      const pipeline = redis.pipeline();
      pipeline.zremrangebyscore(key, 0, windowStart);
      pipeline.zadd(key, now, `${now}-${Math.random()}`);
      pipeline.zcard(key);
      pipeline.expire(key, Math.ceil(windowMs / 1000));

      const results = await pipeline.exec();
      const requestCount = results?.[2]?.[1] as number;

      res.setHeader("X-RateLimit-Limit", maxRequests);
      res.setHeader("X-RateLimit-Remaining", Math.max(0, maxRequests - requestCount));

      if (requestCount > maxRequests) {
        res.status(429).json({
          error: "Too Many Requests",
          message: `Rate limit exceeded. Maximum ${maxRequests} requests per ${windowMs / 1000}s.`
        });
        return;
      }

      next();
    } catch (err) {
      console.error("Rate limiter Redis error:", err);
      // Fail open to avoid blocking traffic if Redis has transient network blips
      next();
    }
  };
}

// 3. Centralized JWT Authentication Guard
function requireAuth(req: Request, res: Response, next: NextFunction) {
  const authHeader = req.header("Authorization");
  if (!authHeader || !authHeader.startsWith("Bearer ")) {
    res.status(401).json({ error: "Unauthorized", message: "Missing or invalid Bearer token" });
    return;
  }

  const token = authHeader.substring(7);
  try {
    const decoded = jwt.verify(token, JWT_SECRET) as { sub: string; role: string; tenantId: string };
    
    // Inject trusted downstream headers so internal microservices skip JWT verification
    req.headers["x-user-id"] = decoded.sub;
    req.headers["x-user-role"] = decoded.role;
    req.headers["x-tenant-id"] = decoded.tenantId;

    next();
  } catch (err) {
    res.status(401).json({ error: "Unauthorized", message: "Token expired or signature invalid" });
    return;
  }
}

// 4. Dynamic Proxy Route Mounting
routes.forEach((route) => {
  const middlewares: any[] = [];

  // Attach route-specific rate limiter
  if (route.rateLimit) {
    middlewares.push(createSlidingWindowLimiter(route.rateLimit.windowMs, route.rateLimit.max));
  }

  // Attach authentication if route requires protection
  if (route.authRequired) {
    middlewares.push(requireAuth);
  }

  // Attach HTTP Reverse Proxy Middleware
  middlewares.push(
    createProxyMiddleware({
      target: route.target,
      changeOrigin: true,
      pathRewrite: {
        [`^${route.prefix}`]: "", // Strip route prefix when forwarding to microservice
      },
      on: {
        proxyReq: (proxyReq, req: any) => {
          // Pass trusted headers to internal network
          if (req.headers["x-user-id"]) proxyReq.setHeader("x-user-id", req.headers["x-user-id"]);
          if (req.headers["x-user-role"]) proxyReq.setHeader("x-user-role", req.headers["x-user-role"]);
          if (req.headers["x-tenant-id"]) proxyReq.setHeader("x-tenant-id", req.headers["x-tenant-id"]);
          if (req.headers["x-correlation-id"]) proxyReq.setHeader("x-correlation-id", req.headers["x-correlation-id"]);
        },
        error: (err, req, res: any) => {
          console.error(`Proxy error forwarding to ${route.target}:`, err);
          res.writeHead(502, { "Content-Type": "application/json" });
          res.end(JSON.stringify({ error: "Bad Gateway", message: "Upstream service unreachable" }));
        },
      },
    })
  );

  app.use(route.prefix, ...middlewares);
  console.log(`[GATEWAY] Mounted ${route.prefix} -> ${route.target} (Auth: ${route.authRequired})`);
});

const PORT = Number(process.env.PORT) || 3000;
app.listen(PORT, () => {
  console.log(`✅ API Gateway running on port ${PORT}`);
});

API Gateway Architecture Diagram

SCSS

+---------------------------------------------------------------------------------+
|                       Centralized API Gateway Mesh                              |
+---------------------------------------------------------------------------------+
|                                                                                 |
|   [Client Mobile / Web Application]                                            |
|                 |                                                               |
|                 | (HTTPS Request + Bearer JWT + X-Correlation-Id)               |
|                 v                                                               |
|   +-------------------------------------------------------------+               |
|   |                   Node.js API Gateway                       |               |
|   |  1. Correlation ID Generator (x-correlation-id)             |               |
|   |  2. Sliding Window Rate Limiter (Redis Sorted Sets)         |               |
|   |  3. Centralized JWT Verification (Extract sub, role, tenant)|               |
|   |  4. Reverse Proxy Header Rewrite (http-proxy-middleware)   |               |
|   +-------------------------------------------------------------+               |
|         |                     |                     |                           |
|   (x-user-id: 42)       (x-user-id: 42)       (x-user-id: 42)                   |
|         v                     v                     v                           |
|   [Auth Service]        [User Service]        [Order Service]                   |
|   (Port 4001)           (Port 4002)           (Port 4003)                       |
+---------------------------------------------------------------------------------+

4 Rules for Resilient Microservice Gateways

Never Verify JWTs Twice: Verify cryptographic JWT signatures once at the Gateway boundary. Forward trusted plaintext headers (x-user-id, x-user-role) over private VPC networks to save CPU cycles on internal microservices.
Fail Open on Cache Blips: If Redis experiences a transient network timeout, log an alert and let requests proceed rather than taking down 100% of API ingress.
Always Forward Correlation IDs: Pass x-correlation-id into all microservices, message queues, and logs so engineers can trace a user click across 10 distinct service boundaries in OpenTelemetry.
Enforce Request Timeouts: Configure upstream proxy timeouts (e.g. 10 seconds). If a backend microservice hangs, the Gateway cleanly terminates the socket and returns 504 Gateway Timeout.

Production Readiness Checklist

Rate Limiting via Redis: Sliding window counter prevents API spam and brute-force credential stuffing.
Distributed Tracing: X-Correlation-Id injected into all proxy requests.
Secure Private Headers: Gateway strips incoming external x-user-id headers before injecting validated JWT payload values.
Graceful Upstream Error Handling: Unreachable microservices emit 502 Bad Gateway instead of crashing the Gateway Node process.
Health Check Endpoint: Exposed /healthz for Kubernetes liveness and readiness probes.

Conclusion

An API Gateway is the central nervous system of a microservices architecture. By consolidating authentication, rate limiting, and request routing into a performant Node.js gateway layer, engineering teams simplify downstream service code, enforce bulletproof security policies, and scale distributed architectures with complete confidence.