Skip to content
AccelerAI, home

Machine Learning Engineer, Inference

Engineering · San Jose, CA (hybrid) · Full-time

Make Acceler faster on customer hardware.

About the role

You will own the speed of Acceler, our inference model, on the hardware our customers run themselves. You will find where time and tokens go, fix it, and prove the gain with benchmarks the whole team trusts.

What you will do

  • Profile and optimise model serving on GPUs and CPUs, from kernels to batching and caching.
  • Apply quantisation, distillation and speculative decoding where they hold accuracy.
  • Build benchmark suites that measure latency, throughput and cost per request on real workloads.
  • Work with platform engineers to ship optimisations into customer deployments safely.
  • Write clear notes on what changed, why, and what it saved.

What you will bring

  • 3+ years building or serving machine learning models in production.
  • Strong Python, plus C++ or Rust for performance-critical code.
  • Hands-on experience with PyTorch and an inference stack such as vLLM, TensorRT-LLM or ONNX Runtime.
  • A habit of measuring before and after every change.

Nice to have

  • CUDA or Triton kernel work.
  • Experience running models on-premises or in air-gapped environments.
  • Open-source contributions to inference or serving projects.

Apply for this role

How we handle your details: Privacy Policy.