Skip to main content

HTTP/2 Regional Services

· 2 min read

Regional Services attached to proxiML jobs can now use HTTP/2 (including gRPC) in addition to HTTPS, so you can connect workers to private APIs and backends that speak those protocols. For background on how services attach to jobs, see the CloudBender reference and the Regional Services announcement.

How It Works​

This extends the Regional Services model: services are still defined and attached in your CloudBender configuration, but you can now select the HTTP2 service type. For workloads that expose gRPC over TLS, use the same service attachment flow as for HTTP2-backed services; specify target_port when multiple services are attached so traffic is routed to the correct listener (see Multi-Service Endpoint Jobs).

To support full HTTP/2 end-to-end encryption, a TLS certificate and private key are required. Inside the job environment, the certificate is located at "$ML_SECRETS_PATH/ml/<service_id>/cert.pem" and the private key at "$ML_SECRETS_PATH/ml/<service_id>/privkey.pem".

When to use what

  • HTTPS — Default for REST-style HTTP APIs and health-checked HTTP endpoints.
  • HTTP2 — Useful when your upstream expects HTTP/2 end-to-end (for example, gRPC endpoints).