Running in a container

GATool has two response caps, gatool.api.max-response-size and gatool.results.max-characters. The first decides the memory a call holds. The figures below were measured with gatool.api.max-response-size raised to 10MB. One refused 9 MB result allocates about 87 to 111 MiB. It holds about 120 MiB of heap while in flight. The body is read, parsed and written as text before gatool.results.max-characters refuses it. Eight concurrent 9 MB responses peaked at 1,088 to 1,331 MiB. gatool.api.max-response-size=256KB brings one such call to about 10 MiB. JDK 21 in a 512 MiB container with two CPUs picks SerialGC with a 128 MiB heap. One 9 MB response under a cap raised to 10MB fills that heap.

The default of gatool.api.max-response-size is 1MB. That is the measured safe point for a 512 MiB container. The measurement used MALLOC_ARENA_MAX=2 in the environment and -XX:MaxRAMPercentage=60. It reached 350 MiB RSS for 64 concurrent 1 MB results. Add -XX:+ExitOnOutOfMemoryError, because an OutOfMemoryError on a Tomcat or client thread leaves the health endpoint answering UP while calls fail. Heap per call in flight is about ten times max-response-size during the parse. Multiply that by the concurrency the deployment allows. Tomcat allows 200 threads by default. The 350 MiB figure above was measured at 64 concurrent calls. A deployment on Tomcat's 200 threads lowers server.tomcat.threads.max to stay near that figure. GATool does not limit concurrent calls in this release, so the deployment bounds them with server.tomcat.threads.max or at the gateway.

gatool.api.schema.max-size bounds the schema fetched from a URL, at 10MB by default. A schema of a few megabytes therefore loads while gatool.api.max-response-size stays at the size the heap allows.

On the stateful transport of the Model Context Protocol (MCP), a connected client holds a listening stream open. Spring Boot's graceful shutdown counts that stream as an active request. GATool ends the sessions before that shutdown begins. A stop with connected clients therefore does not wait out spring.lifecycle.timeout-per-shutdown-phase. A client that was connected opens a new session against the next instance.