Skip to content
Help guide the future of agentgateway — Take the survey

For the complete documentation index, see llms.txt. Markdown versions of all docs pages are available by appending .md to any docs URL.

Connection limits

Page as Markdown

Limit the in-flight requests and client connections that agentgateway accepts on each gateway port.

Attaches to:

Cap the number of requests and connections that clients can open to the gateway proxy at the same time. Over a limit, the proxy rejects new work immediately and does not queue it. As a result, a traffic spike or a client that opens many connections cannot exhaust the proxy’s memory and file descriptors.

You set both limits in the top-level frontendPolicies section of the configuration file. Each limit counts separately for each gateway port. By default, both limits are unset, so the number of requests and connections is not limited.

SettingWhat the limit countsWhat happens over the limit
frontendPolicies.http.maxConcurrentRequestsIn-flight HTTP requests, including HTTP/1 requests and HTTP/2 streams. The minimum value is 1.Agentgateway returns a 503 response with the body request limit exceeded, and increments the agentgateway_requests_shed_total metric.
frontendPolicies.tcp.maxConnectionsActive client connections. The minimum value is 1.Agentgateway closes the new connection without a response, and increments the agentgateway_downstream_connections_shed_total metric.

Before you begin

  1. Install the agentgateway binary.
  2. Make sure that you have docker, such as by installing Docker Desktop.

Limit requests and connections

Set a request limit and a connection limit in turn, and send more traffic than each limit allows.

  1. Start the httpbin sample app in Docker. The app listens on port 8000 on your machine.

    docker run -d --name httpbin -p 8000:8080 mccutchen/go-httpbin:v2.15.0
  2. Create a configuration file that allows 2 in-flight requests.

    cat > config.yaml <<EOF
    # yaml-language-server: \$schema=https://agentgateway.dev/schema/config
    frontendPolicies:
      http:
        maxConcurrentRequests: 2
    gateways:
      default:
        port: 3000
    routes:
    - backends:
      - host: localhost:8000
    EOF
  3. Run agentgateway with the configuration file.

    agentgateway -f config.yaml
  4. In another terminal, send 3 requests at the same time to the httpbin /delay/3 endpoint, which responds after 3 seconds. The first 2 requests take all the request slots, so agentgateway rejects the third request immediately with a 503 response.

    for i in 1 2 3; do
      curl -s -o /dev/null -w "request $i: %{http_code}\n" http://localhost:3000/delay/3 &
      sleep 0.2
    done; wait

    Example output:

    request 3: 503
    request 1: 200
    request 2: 200
    
  5. In the configuration file, replace the frontendPolicies section with the following section. The new section sets a connection limit of 1 and no request limit. Agentgateway reloads the file when it changes.

    frontendPolicies:
      tcp:
        maxConnections: 1
  6. Open a connection with a slow request, and then open a second connection. The first connection takes the only connection slot, so agentgateway closes the second connection without a response.

    curl -s -o /dev/null -w "connection 1: %{http_code}\n" http://localhost:3000/delay/3 &
    sleep 0.5
    curl -sS -o /dev/null -w "connection 2: %{http_code}\n" http://localhost:3000/headers
    wait

    Example output:

    curl: (56) Recv failure: Connection reset by peer
    connection 2: 000
    connection 1: 200
    
  7. Check the metrics that count the rejected requests and connections. The bind label names the gateway port.

    curl -s localhost:15020/metrics | grep -E '^agentgateway_(requests|downstream_connections)_shed'

    Example output:

    agentgateway_downstream_connections_shed_total{bind="bind/3000"} 1
    agentgateway_requests_shed_total{bind="bind/3000"} 1
    

Cleanup

  1. Stop agentgateway by pressing Ctrl+C in the terminal where agentgateway runs.

  2. Remove the httpbin container and the configuration file.

    docker rm -f httpbin
    rm config.yaml
Was this page helpful?
Agentgateway assistant

Ask me anything about agentgateway configuration, features, or usage.

Note: AI-generated content might contain errors; please verify and test all returned information.

Tip: one topic per conversation gives the best results. Use the + button in the chat header to start a new conversation.

Switching topics? Starting a new conversation improves accuracy.
↑↓ navigate ↵ select esc dismiss

What could be improved?

Your feedback helps us improve assistant answers and identify docs gaps we should fix.

Need more help? Join us on Discord: https://discord.gg/y9efgEmppm

Want to use your own agent? Add the Solo MCP server to query our docs directly. Get started here: https://search.solo.io/.