Protocol

Triton Inference Protocol

KServe · V2 inference protocol

Lab ready Full lab

HTTP/gRPC model-inference APIs popularized by Triton/KServe V2: readiness/liveness, infer, and model-repository control. Tensor contents, datatypes, and shared-memory extensions dominate latency. It standardizes serving regardless of the backend framework.

Domains

Tags

inferenceservingtensorsml

Sources

  • KServe V2 inference protocol
  • NVIDIA Triton documentation

Related protocols

gRPC

Lab ready

RPC framework on HTTP/2 with protobuf stubs, streaming RPCs, deadlines, and metadata. Widely used as the control plane for ML serving, mi…

aiinternet

HTTP/2

Lab ready

Multiplexed binary framing over a single TLS/TCP connection with streams, HPACK headers, and flow-control windows. It underpins gRPC and …

internetai

Arrow Flight

Lab ready

gRPC protocol for bulk Apache Arrow record-batch transfer with DoGet/DoPut and FlightDescriptors. Columnar data moves with minimal copies…

aiinternet