Building for 100k RPS: Custom Rust vs. Edge Middleware

When building high-traffic platforms, routing incoming traffic with high throughput and low cost becomes one of the most important architectural decisions. This article explores how Next.js Edge Middleware compares with dedicated Rust gateways built using Actix Web and Axum, examining latency, scalability, infrastructure costs, developer productivity, and real-world deployment strategies for modern internet-scale systems.

Building for 100k RPS: Custom Rust vs. Edge Middleware

Modern web applications rarely struggle because of databases first.

They struggle because of traffic routing.

Every incoming request must be authenticated, localized, rate limited, routed, logged, traced, and eventually forwarded to the correct service.

At small scale, this layer is almost invisible.

At internet scale, it becomes one of the most expensive components in your infrastructure.

The question many engineering teams eventually face is:

Should routing happen inside Edge Middleware, or should we build a dedicated high-performance gateway?

This decision affects:

• Infrastructure cost

• Global latency

• Throughput

• Operational complexity

• Engineering productivity

• Future scalability

For years, the default answer was to place everything inside reverse proxies like NGINX or HAProxy.

Today, platforms like Next.js, Vercel, and Cloudflare Workers allow developers to execute JavaScript before requests even reach an application server.

At the same time, organizations handling millions of requests every minute increasingly deploy Rust-powered gateways using frameworks like Actix Web and Axum to maximize throughput while minimizing infrastructure cost.

This article explores both approaches, their strengths, limitations, and where each fits in a production architecture.

Understanding 100,000 Requests per Second

Before comparing technologies, it's important to understand what 100k RPS actually means.

100,000 requests every second equals:

Very few startups begin at this scale.

However, systems serving:

• SaaS platforms

• Banking APIs

• Social media

• Gaming backends

• Streaming platforms

• AI inference APIs

• E-commerce marketplaces

can eventually reach this level during peak traffic.

At these volumes, even 1 millisecond of additional latency becomes expensive.

Saving:

• 2 ms

• 5 ms

• 10 ms

across billions of requests produces enormous reductions in compute costs.

What is Next.js Edge Middleware?

Next.js Middleware executes before the request reaches pages or API routes.

Instead of running inside a traditional Node.js server, middleware executes inside an Edge Runtime, allowing request processing closer to end users.

Typical middleware tasks include:

• Authentication

• Authorization

• URL rewrites

• Redirects

• Locale detection

• Cookie inspection

• Security headers

• A/B testing

• Request enrichment

Middleware executes before routing resolution, making it ideal for lightweight request handling. Recent versions of Next.js also support a Node.js runtime option for middleware, though Edge remains the default execution model. oai_citation:0‡Next.js

How Edge Runtime Works

Unlike a traditional server process, Edge Middleware executes inside lightweight runtime isolates.

Instead of:

Incoming Request

      ↓

Node Server

      ↓

Application

the flow becomes:

Incoming Request

      ↓

Edge POP

      ↓

Middleware

      ↓

Destination

This architecture dramatically reduces latency for users located far from your origin server.

Because middleware executes geographically closer to users, it is especially effective for:

• Redirect decisions

• Localization

• Cookie parsing

• Session validation

• Authentication tokens

• Simple routing logic

The tradeoff is that Edge runtimes intentionally expose only a subset of Node.js APIs to keep execution lightweight and portable. oai_citation:1‡Next.js

Advantages of Next.js Edge Middleware

1. Extremely Low Latency

Since execution happens near users, requests avoid unnecessary round trips.

Instead of:

London

↓

Virginia

↓

Response

routing decisions happen inside European edge locations.

The improvement is especially noticeable for:

• Authentication

• Redirects

• Country detection

• Feature flags

2. Excellent Developer Experience

Most frontend engineers already understand TypeScript.

Instead of introducing another language and deployment pipeline, middleware integrates directly into existing Next.js projects.

A single repository can manage:

• Frontend

• Middleware

• API routes

without additional infrastructure.

3. Native Routing

Middleware integrates tightly with:

• NextRequest

• NextResponse

• Rewrites

• Redirects

• Cookies

• Headers

making routing logic concise and maintainable.

4. Automatic Global Distribution

Platforms automatically deploy middleware worldwide.

Developers don't manage:

• Regions

• Load balancers

• Geo-routing

• Edge synchronization

The platform handles deployment.

Limitations of Edge Middleware

Edge Middleware is intentionally optimized for lightweight request processing.

It is not a replacement for high-performance backend services.

Common limitations include:

• Restricted runtime APIs

• Limited execution time

• Smaller memory budgets

• No unrestricted filesystem access

• Reduced support for native Node.js modules

• Platform-specific execution constraints

These constraints encourage middleware to remain lightweight rather than becoming a full application server. oai_citation:2‡Next.js

Introducing Rust API Gateways

Rust approaches the problem differently.

Instead of executing tiny request handlers at the edge, organizations deploy dedicated gateway services built specifically for performance.

Popular frameworks include:

• Actix Web

• Axum

• Hyper

• Tower

Unlike JavaScript runtimes, Rust applications compile directly into native machine code.

There is:

• no garbage collector

• no JIT compilation

• no runtime interpreter

This gives engineers precise control over memory allocation, concurrency, and threading.

Why Rust Became Popular for Infrastructure

Companies increasingly adopt Rust because it combines:

• Memory safety

• High throughput

• Low latency

• Predictable performance

without sacrificing developer control.

Rust's ownership model eliminates entire classes of runtime memory errors while allowing applications to perform close to C and C++ speeds. oai_citation:3‡arXiv

Actix Web vs Axum

Although both frameworks are production-ready, they prioritize different goals.

Axum integrates deeply with the Tower ecosystem, enabling reusable middleware composition and service layers. oai_citation:4‡Docs.rs

Throughput Benchmarks

One of the biggest misconceptions in backend engineering is that frameworks alone determine performance.

They don't.

Performance is a combination of:

• Runtime architecture
• Network stack
• Serialization
• Database latency
• Memory allocation
• Kernel networking
• Async scheduler
• Hardware
• Load-balancer configuration

That said, the underlying runtime still has a measurable impact.

For simple HTTP workloads, Rust frameworks consistently rank among the fastest production-ready web frameworks because they compile to native code and execute on the Tokio asynchronous runtime with minimal overhead. Independent benchmarking projects also show that Actix Web and Axum often perform nearly identically, suggesting framework choice is more about ergonomics than raw speed. oai_citation:0‡NpgsqlRest

Why Rust Performs So Well

Unlike managed runtimes that depend on garbage collection or JIT compilation, Rust applications execute as native binaries.

A request typically flows through:

TCP Socket
      │
      ▼
Linux Kernel
      │
      ▼
Tokio Runtime
      │
      ▼
Hyper HTTP
      │
      ▼
Axum / Actix
      │
      ▼
Business Logic

Every layer is optimized for asynchronous, non-blocking execution.

Tokio schedules thousands of concurrent tasks efficiently while Hyper provides a low-overhead HTTP implementation. Axum builds on top of Hyper and Tower rather than introducing an entirely separate middleware ecosystem, keeping overhead low. oai_citation:1‡Axum

Latency Comparison

When evaluating routing layers, engineers should look beyond average latency.

Important metrics include:

A system with:

• 8 ms average
• 300 ms P99

may deliver a noticeably worse experience than one with:

• 15 ms average
• 25 ms P99

Predictability matters as much as speed.

Memory Consumption

Memory usage is often ignored until infrastructure costs begin to grow.

Edge middleware generally operates within platform-defined memory budgets designed to encourage lightweight execution.

A dedicated Rust gateway, however, runs under your own operating system and infrastructure.

Advantages include:

• predictable allocation
• no garbage collector pauses
• efficient stack usage
• fine-grained control over heap allocations
• minimal runtime overhead

These characteristics are a major reason Rust is increasingly adopted for networking infrastructure and high-performance services. oai_citation:2‡arXiv

CPU Utilization

CPU efficiency directly affects cloud spending.

Imagine two gateways:

Gateway A

• 70% CPU utilization

Gateway B

• 25% CPU utilization

If both deliver the same throughput, Gateway B allows significantly higher request density per instance before scaling out.

Rust's compiled execution model and asynchronous runtime often provide excellent CPU efficiency, especially for network-bound workloads. oai_citation:3‡NpgsqlRest

Cost Analysis

Infrastructure cost isn't only about server pricing.

It includes:

• Compute
• Memory
• Bandwidth
• Scaling
• Operations
• Monitoring
• Engineering time

Edge Middleware Costs

Advantages

✅ No server maintenance

✅ Automatic scaling

✅ Global deployment

✅ Zero infrastructure management

Tradeoffs

• Usage-based billing
• Runtime limits
• Platform lock-in
• Higher costs at very large request volumes

For many startups, these tradeoffs are worthwhile because they eliminate operational complexity.

Rust Gateway Costs

Running a Rust gateway on virtual machines or Kubernetes shifts responsibility to your team.

Advantages include:

• fixed infrastructure pricing
• predictable scaling
• full runtime control
• custom networking
• lower marginal cost at sustained high traffic

Tradeoffs include:

• patch management
• deployment pipelines
• observability
• autoscaling configuration
• operational expertise

Example Monthly Cost Comparison

Illustrative architecture comparison (actual costs vary based on traffic patterns, providers, regions, cache hit rates, and business logic).

The key insight is that developer productivity often outweighs infrastructure savings early on, while mature high-volume platforms increasingly optimize for infrastructure efficiency.

Kubernetes Deployment

Large organizations rarely expose application servers directly.

Instead, production traffic commonly follows an architecture similar to:

Internet
      │
      ▼
Global CDN
      │
      ▼
Load Balancer
      │
      ▼
Rust Gateway Cluster
      │
 ┌────┴────┐
 ▼         ▼
Auth     API Router
 │         │
 └────┬────┘
      ▼
Application Services
      ▼
Database

Each gateway instance remains stateless, allowing horizontal scaling.

Authentication Strategy

Authentication is one of the strongest use cases for Edge Middleware.

Typical flow:

User
   │
   ▼
Edge Middleware
   │
Verify JWT
   │
   ├──── Invalid → Reject
   │
   ▼
Rust Gateway
   │
Business Logic
   ▼
Database

Benefits:

• invalid requests stop at the edge
• lower origin traffic
• reduced backend CPU utilization
• improved global latency

Load Balancing

A Rust gateway can integrate directly with:

• Envoy
• HAProxy
• NGINX
• AWS ALB
• Kubernetes Ingress
• Cloud Load Balancers

This enables advanced routing such as:

• weighted deployments
• canary releases
• blue/green deployments
• geographic routing
• service discovery

Observability

At high throughput, monitoring becomes mandatory.

A production gateway should expose:

• request duration
• p95 latency
• p99 latency
• CPU utilization
• memory usage
• connection count
• error rate
• timeout rate
• retry count
• distributed tracing

Popular tooling includes:

• Prometheus
• Grafana
• OpenTelemetry
• Jaeger
• Loki

Failure Scenarios

High-scale systems should assume failures are inevitable.

Examples include:

Edge Failure

• middleware timeout
• provider outage
• deployment rollback
• malformed rewrite

Gateway Failure

• exhausted connection pools
• memory leaks
• slow downstream services
• cascading retries
• network partitions

Designing graceful degradation and retry strategies is more important than maximizing theoretical throughput.

Hybrid Architecture (Recommended)

For most organizations, the best solution is not choosing one over the other.

Instead, combine both technologies.

User
 │
 ▼
CDN
 │
 ▼
Next.js Edge Middleware
 │
 ├── Locale Detection
 ├── Authentication
 ├── Redirects
 ├── Cookies
 └── Feature Flags
 │
 ▼
Rust Gateway
 │
 ├── Rate Limiting
 ├── API Routing
 ├── Business Logic
 ├── Write Operations
 ├── Event Processing
 └── Service Mesh
 │
 ▼
Microservices
 │
 ▼
Database

This approach leverages the strengths of each layer:

Edge Middleware

• request personalization
• authentication
• localization
• lightweight routing

Rust Gateway

• high-throughput APIs
• streaming
• write-heavy endpoints
• complex authorization
• service orchestration
• infrastructure control

When Edge Middleware Wins

Choose Edge Middleware if your application primarily requires:

• global redirects
• localization
• feature flags
• session validation
• authentication
• excellent developer experience
• minimal infrastructure management

When Rust Wins

A dedicated Rust gateway becomes compelling when you need:

• sustained high throughput
• predictable latency
• infrastructure cost optimization
• advanced networking
• custom middleware
• streaming APIs
• long-lived connections
• complete operational control

The Nepdia Verdict

There is no universal winner.

Edge Middleware and Rust gateways solve different layers of the same problem.

Use Next.js Edge Middleware for work that benefits from global proximity to users:

• localization
• authentication
• redirects
• lightweight routing
• feature flags
• request preprocessing

Use a Rust gateway for workloads that demand maximum efficiency:

• API aggregation
• payment processing
• event ingestion
• write-heavy endpoints
• streaming
• high-concurrency business logic
• service-to-service communication

For organizations expecting rapid growth, the hybrid model provides the best balance of developer velocity, operational flexibility, and long-term infrastructure efficiency.

The most resilient architectures aren't built by forcing every request through a single technology stack—they're built by placing each responsibility in the environment where it performs best.

References

1. Next.js Edge Runtime Documentation
2. Next.js Middleware Documentation
3. Axum Documentation
4. Tokio Documentation
5. Hyper HTTP Library
6. Tower Middleware Ecosystem
7. Actix Web Documentation
8. Rust Programming Language
9. "Rust: The Programming Language for Safety and Performance" (Academic Survey)
10. Independent PostgreSQL REST API Benchmarks (Actix vs. Axum)