Alauda Container Platform

Overview

TOC

Understanding networking

In a cloud-native environment, the core role of networking is to act as the central nervous system and circulatory system. It is the foundational enabler for key application characteristics like elasticity, resilience, and observability.

The responsibilities of cloud-native networking can be broken down into several key areas:

Service Discovery & Load Balancing

Responsibility: Automatically discover running microservice instances and intelligently distribute traffic among them.

Implementation:

  • Service Discovery

    • Pods register with central registry (e.g., etcd)
    • Services query registry to find available instances
    • Kubernetes CoreDNS provides built-in discovery via service names

  • Load Balancing

    • Kubernetes Service resources get Virtual IP (VIP)
    • Acts as built-in load balancer to backend Pods (Endpoints)
    • Distributes requests evenly across instances

Traffic Management & Routing

Responsibility: Control traffic with fine-grained rules to enable advanced deployment patterns.

Implementation:

  • Ingress

    • Cluster's "entry gate" for external HTTP/HTTPS traffic
    • Host/path-based routing
    • SSL termination

  • Service Mesh (Istio, Linkerd)

    • Canary Releases: Direct percentage of traffic to new versions
    • Fault Injection: Simulate service failures for resilience testing
    • Timeouts, Retries & Circuit Breaking: Improve fault tolerance
    • Traffic Mirroring: Copy production traffic to testing environments

Network Security

Responsibility: Enforce "Zero Trust" security model for authorized service communication.

Implementation:

  • Network Policies

    • Firewall-like rules for Pod communication
    • Define which Pods can communicate with each other
    • Example: "Only frontend Pods can access backend database Pods"

  • Mutual TLS (mTLS)

    • Encrypts and authenticates all service-to-service communication
    • Ensures data security and identity verification
    • Widely used in service meshes

Observability

Responsibility: Provide rich network data for understanding and diagnosing distributed systems.

Implementation:

  • Metrics

    • Traffic rates, error rates, latency data
    • Integration with monitoring systems (Prometheus)

  • Logs

    • Access logs for all requests
    • Troubleshooting and auditing

  • Tracing

    • Track request paths across multiple microservices
    • Analyze performance bottlenecks and dependencies
    • Service mesh enables distributed tracing

Learn more about Networking Observability.

Core network layers and components

The container network is a comprehensive networking solution designed for cloud-native applications, ensuring seamless east-west communication within clusters and efficient north-south traffic management across external networks, while providing essential networking functionalities. It consists of these core components:

  • Container Network Interfaces (CNIs) for east-west traffic management within the cluster.
  • Ingress Gateway Controller ALB (Deprecated), NGINX Ingress Controller or Envoy Gateway (Recommended) for managing HTTPS ingress traffic.
  • MetalLB for handling LoadBalancer type Services.
  • Additionally, it provides robust network security and encryption features to ensure secure communication.

GatewayAPI

Gateway API is an official Kubernetes project focused on L4 and L7 routing in Kubernetes. This project represents the next generation of Kubernetes Ingress, Load Balancing, and Service Mesh APIs. From the outset, it has been designed to be generic, expressive, and role-oriented.

The overall resource model focuses on 3 separate personas and corresponding resources that they are expected to manage:

Most of the configuration in this API is contained in the Routing layer. These protocol-specific resources (HTTPRoute, GRPCRoute, etc.) enable advanced routing capabilities for both Ingress and Mesh.

Gateway API for Ingress

When using Gateway API to manage ingress traffic, the Gateway resource defines a point of access at which traffic can be routed across multiple contexts -- for example, from outside the cluster to inside the cluster (north/south traffic).

Each Gateway is associated with a GatewayClass, which describes the actual kind of gateway controller that will handle traffic for the Gateway; individual routing resources (such as HTTPRoute) are then associated with the Gateway resources. Separating these different concerns into distinct resources is a critical part of the role-oriented nature of Gateway API, as well as allowing for multiple kinds of gateway controllers (represented by GatewayClass resources),

Gateway API concepts

The following design goals drive the concepts of Gateway API. These demonstrate how Gateway aims to improve upon current standards like Ingress.

  • Role-oriented - Gateway is composed of API resources which model organizational roles that use and configure Kubernetes service networking.
  • Portable - This isn't an improvement but rather something that should stay the same. Just as Ingress is a universal specification with numerous implementations, Gateway API is designed to be a portable specification supported by many implementations.
  • Expressive - Gateway API resources support core functionality for things like header-based matching, traffic weighting, and other capabilities that were only possible in Ingress through custom annotations.
  • Extensible - Gateway API allows for custom resources to be linked at various layers of the API. This makes granular customization possible at the appropriate places within the API structure.

Some other notable capabilities include:

  • GatewayClasses - GatewayClasses formalize types of load balancing implementations. These classes make it easy and explicit for users to understand what kind of capabilities are available via the Kubernetes resource model.
  • Shared Gateways and cross-Namespace support - They allow the sharing of load balancers and VIPs by permitting independent Route resources to attach to the same Gateway. This allows teams (even across Namespaces) to share infrastructure safely without direct coordination.
  • Typed Routes and typed backends - Gateway API supports typed Route resources and also different types of backends. This allows the API to be flexible in supporting various protocols (like HTTP and gRPC) and various backend targets (like Kubernetes Services, storage buckets, or functions).

For more detailed descriptions of the Gateway API, please refer to the gateway-api documentation.

Comparison Among Service, Ingress, Gateway API

The Alauda Container Platform supports multiple ingress traffic specifications in Kubernetes ecosystem. This document compares them (Service, Ingress, Gateway API) to help users make the right choice.

For L4 (TCP/UDP) Traffic

Both LoadBalancer-type Services and Gateway API (TCP/UDP Routes) can expose Layer 4 traffic externally. However, they differ significantly in their implementation approach and performance characteristics.

Implementation: Kernel-space forwarding

  • Traffic forwarding is handled directly by the Linux kernel (iptables/IPVS/eBPF)
  • Minimal overhead and near-native performance

Advantages:

  • High performance with low latency
  • Lower CPU and memory overhead
  • Battle-tested and mature technology

Gateway API TCP/UDP Routes

Implementation: User-space proxy

  • Implemented by Envoy Gateway (or other gateway controllers)
  • Traffic must traverse from kernel space to user space and back
  • Additional processing overhead in the application layer

Disadvantages:

  • Performance degradation compared to kernel-space solutions
  • Higher resource consumption (CPU/memory)
  • Additional latency due to user-space context switching

Recommendation

We recommend using LoadBalancer-type Services for L4 traffic routing due to their superior performance and lower resource overhead.

Currently, we support LoadBalancer-type services through MetalLB.

For L7(HTTP/HTTPS) Traffic

While Ingress, GatewayAPI, can all expose L7 traffic externally, they differ in their capabilities.

Ingress

Ingress is the standard specification adopted by the Kubernetes community and are recommended for default use. The Ingress is handled by ALB instances that are managed by the platform administrator.

GatewayAPI(Recommended)

Gateway API is the next-generation routing standard for Kubernetes, designed to address the limitations of Ingress and provide more powerful, flexible, and standardized traffic management capabilities. Compared to Ingress, Gateway API offers the following advantages:

  1. Role-Oriented Design

    • Clear separation of concerns: Infrastructure admin (GatewayClass, Gateway) vs. Application developer (Routes)
    • Ingress mixes infrastructure and application concerns in a single resource

  2. Expressive Routing Capabilities

    • Rich matching rules: HTTP headers, query parameters, methods, etc.
    • Ingress is limited to host and path matching
    • Built-in support for traffic splitting, mirroring, and advanced policies

  3. Protocol Support

    • Native support for HTTP, HTTPS, TCP, UDP, gRPC, and TLS passthrough
    • Ingress primarily focuses on HTTP/HTTPS only

  4. Extensibility

    • Type-safe extension points through CRDs (e.g., HTTPRoute filters, policy attachments)
    • Ingress relies heavily on vendor-specific annotations, leading to portability issues

  5. Cross-Namespace Routing

    • Routes can reference Services in different namespaces (with ReferenceGrant)
    • Ingress is typically limited to same-namespace references

  6. Multiple Listeners per Gateway

    • A single Gateway can handle multiple ports, protocols, and hostnames
    • Ingress typically requires one resource per configuration

  7. Portability and Standardization

    • Vendor-neutral API with conformance tests
    • Reduces lock-in and improves interoperability between implementations
    • Ingress implementations vary widely in capabilities and annotations

  8. Attachment and Selection Model

    • Routes explicitly attach to Gateway listeners via parentRefs
    • Clearer relationships and easier troubleshooting
    • Ingress uses IngressClass which is less flexible

Currently, we support GatewayApi through Envoygateway. For migration from ingress to gatewayapi please refer to migration guide

Network Traffic Flow

Example network traffic flow through an Alauda Container Platform cluster.

External Client (Browser / curl) https://www.example.com
   │
   ▼
DNS Resolution (https://www.example.com) → IP (e.g. 34.23.88.11)
   │
   ▼
External Load Balancer [L4]
   │
   ▼
Envoy Gateway / Ingress NGINX Controller [L7]
   │
   ▼
Kubernetes Service (ClusterIP) [L4]
   │
   ▼
Pod (Application)

Flow explanation:

  1. Client (Browser / curl).

    The user sends an HTTPS request to https://www.example.com. The client works at Layer 7, initiating a DNS lookup.

  2. DNS Resolution.

    DNS translates the domain name into a public IP address (e.g. 34.23.88.11).

  3. [L4] External Load Balancer.

    Operates at Layer 4 (TCP/UDP). It forwards incoming connections to backend nodes in the cluster. Examples: AWS NLB, GCP TCP LB, MetalLB.

  4. [L7] Envoy Gateway / Ingress Controller.

    Operates at Layer 7 (Application Layer). Handles:

    • TLS termination

    • Hostname and path-based routing

    • Policies and authentication

    It routes traffic to the matching Kubernetes Service.

  5. [L4] Kubernetes Service (ClusterIP)

    Acts as an internal Layer 4 load balancer inside the cluster. Distributes requests to backend Pods based on selectors.

  6. Pod (Application)

    The final destination where the app runs and processes the request. The response follows the reverse path back to the client.