AI Inference Engineer

Senior AI Inference Engineer opportunity - 12 month initial engagement - government customers. Amstelveen or Eindhoven based, hybrid working. Currently supporting a number of Government customers across the Netherlands, now seeking to add an AI Inference Engineer for their team within Benelux & Nordics, extending an enterprise AI platform for mission-critical workloads. Working with the PM, PO and AI engineers in an autonomous Scrum team, you'll own AI tooling and infrastructure lifecycles and decide which models run on which hardware, at what capacity and quality, balancing delivery and operations.

Key responsibilities: benchmark an 8x H200 node: tensor parallelism versus replicas with llm-d cache-aware routing; measure throughput, time to first token and concurrency per GPU; translate results into customer-facing capacity models: tokens/minute and requests/minute per model/GPU; build catalogue benchmarks covering performance, fixed-set quality evaluation and hardware requirements; design input/output content filtering using self-hosted open-weight models; assess model licences, NVFP4/FP8/INT8 quantisation, memory fit and serving configurations.

You'll bring enterprise ML platform experience, production GPU LLM serving with vLLM or similar, and knowledge of KV cache, batching, prefix caching, quantisation and NVLink versus Ethernet. Python, practical Kubernetes, reproducible performance testing and written architecture decisions (ADRs) are key.

Hybrid: 1–2 days/week in Amstelveen/Eindhoven; flexibility required.