About
Hey i'm Varun Singh, a Backend and Systems Engineer, I love building AI Infrastructure and retrieval systems, agent workflows, and the services underneath them with a bias toward scaling, measurable performance, and reliability. At Oracle I identified a gap and architected Composer, an AI workflow platform that turns 5G requirements into structured test plans and automation-ready bundles, and redesigned the Prometheus collection layer for 5G performance reporting, cutting a 96-logical-site, 24-hour benchmark from 3 hours 58 minutes to 45 minutes. Outside work I build in the open: hybrid retrieval, Go concurrency primitives, and infrastructure tooling, I can never sit still and always try to learn something new. Currently going deeper on inference serving batching, routing, and GPU utilization as i strongly believe that the future of AI is on device. I love playing Valorant, roguelike games, fiction, and exploring new places
SDE-1 @ Oracle · IIIT Jabalpur '25 · SIH 2023 national winner · Code for GovTech (C4GT)
Selected results
- 60% less manual authoring across 8+ network-function teams with Composer
- Prometheus benchmark reduced from 3h 58m to 25m across 96 logical sites and 73K+ PromQL requests
- Mitigated recurring SSH and file-descriptor leaks by centralizing Paramiko, JumpSSH, and SFTP lifecycle cleanup saving resources and improving system stability
- Video readiness reduced from 30 minutes to 5 minutes for 400+ videos across 5 HLS rendition on the LMS backend for 20k+ registered users and 250+ peak concurrent users
Projects
repi
local-first log RCA / hybrid retrieval / resumable agent
An autonomous log-investigation engine for distributed systems. It clusters repetitive logs by signature, retrieves evidence with BM25 + pgvector/HNSW fused through Reciprocal Rank Fusion, and investigates root cause through an explicit, resumable state machine. Every step is persisted for auditability, and confidence is constrained server-side by the evidence actually cited. It can run fully locally with Ollama; hosted-model adapters are optional.
Stack: Python, Postgres, pgvector, ParadeDB, FastEmbed
- implemented - Hybrid retrieval: ParadeDB BM25 + pgvector/HNSW fused with Reciprocal Rank Fusion.
- implemented - Resumable investigation: a hand-rolled finite-state machine whose thoughts, actions and observations persist to Postgres, so a paused run resumes from stored state.
- implemented - Deterministic server-side confidence gating that downgrades conclusions the cited evidence does not support.
- measured - Docker image reduced from 5.58 GB to 1.12 GB by migrating embeddings to ONNX with FastEmbed.
- measured - A 21-question retrieval suite plus scripted incident evaluations. This is a functional suite, not a published benchmark against a public dataset.
- planned - Cross-encoder reranking and a vector-only / BM25-only / hybrid ablation. Neither exists in the code today.
github · demo
collapser
gRPC request-collapsing sidecar / 2,000 concurrent identical requests reached the backend as 1 call
An open-source Go sidecar that suppresses thundering-herd traffic. Equivalent unary requests share one backend call while followers wait on the same result. Envoy, which it runs alongside in a mesh, load-balances and retries those N requests but has no notion that they are the same request; this fills that gap. The proxy is schema-agnostic - a passthrough codec forwards raw gRPC frames, so no generated stubs are needed - runs backend work on a detached context so one cancelled caller cannot kill shared work, and exposes collapse, cache, in-flight and backend-latency metrics through Prometheus. It ships with Kubernetes and Istio manifests plus a kind-based cluster harness that counts backend calls at the backend rather than trusting the proxy's own accounting.
Stack: Go, gRPC, Kubernetes, Istio, Prometheus
- measured - On a local kind cluster running Istio 1.28.1 with sidecar injection and STRICT mTLS, identical concurrent requests through the deployed proxy reached the backend as a single call: 100 concurrent -> 1 call (99.0% fewer backend calls), 500 -> 1 (99.8%), 1,000 -> 1 (99.9%), 2,000 -> 1 (108ms wall against 50ms of backend work). Backend calls were counted at the backend and cross-checked against Envoy's istio_requests_total{reporter="destination"}, not taken from the proxy's own metrics. This is a local single-node kind cluster under synthetic identical load, not production traffic.
- benchmarked - Go microbenchmarks on an 11th Gen Intel Core i7-1165G7 (8 threads, Go 1.25.5), in-process only - no network, no serialization, no backend time. Cache-hit path 45.7 ns/op, 0 B/op, 0 allocs/op. Follower (collapse) path 54 B/op, 0 allocs/op; its ns/op is dominated by however long the backend takes, so the allocation count is the number that means anything. Leader path 1,594 ns/op, 552 B/op, 10 allocs/op.
- benchmarked - In-process stress with the race detector enabled: 10,000 concurrent requests on one key produced 2 backend calls (5,000:1) with 0 errors; a 60-second sustained run with 100 workers put through 391,400 requests as 392 backend calls (998.5:1) at 6,523 RPS with 0 errors. Zero goroutine leak and 0 MB heap growth across the run. Synthetic contention, not production traffic.
- measured - Test coverage: internal/collapser 97.5%, internal/config 95.8%, internal/proxy 33.7%. The proxy package is the thin one.
- implemented - Detached backend context, so a cancelled leader does not kill the shared call its followers are waiting on.
- implemented - Schema-agnostic collapsing through a passthrough codec over raw gRPC frames: no protobuf descriptors or generated stubs required.
- implemented - COLLAPSER_KEY_HEADERS folds named identity headers into the collapse key, so requests that differ only by tenant or authorization get separate backend calls.
- implemented - Prometheus metrics for requests, collapsed requests, backend calls, cache hits, cached entries, in-flight leaders and a backend-latency histogram; plus a liveness endpoint that ignores the backend and a readiness endpoint that reports the backend channel state.
- implemented - Graceful shutdown draining in-flight calls on SIGTERM, panic recovery published as codes.Internal, and a configurable cache entry cap (10,000 by default) so a key-diverse workload cannot grow the map without bound.
- implemented - Kubernetes and Istio manifests (Deployment, Service, VirtualService, DestinationRule, PeerAuthentication), a kind cluster harness, an in-mesh load generator, and a GitHub Pages project page.
- known limitation - Server-streaming RPCs are treated as unary and return only the first response frame, because without a proto descriptor they are indistinguishable from unary calls on the incoming connection; client-streaming and bidirectional calls are detected and forwarded correctly but bypass collapsing entirely. Backend TLS uses the system trust store only - no mTLS, no custom CA bundles - though inside a mesh Istio's sidecars handle that. The result cache is per-process, so N replicas can produce up to N backend calls for one key. The passthrough codec registers globally as "proto", so internal/proxy cannot be imported into a process needing normal protobuf marshalling.
- known limitation - With the default collapse key (method plus payload) two callers sending identical payloads share one response even if they authenticated as different users. Naming the identity headers in COLLAPSER_KEY_HEADERS is what makes that safe, and it is opt-in.
- planned - Optional proto descriptors, mutual TLS and custom CA bundles, and per-method collapse policy. None of these are in the code today.
github · demo
Fend
fleet-wide Linux hardening / Ansible + Electron
A desktop operator tool for applying repeatable Linux hardening across grouped Ubuntu hosts. Electron/React drives agentless Ansible playbooks over SSH, with key-based authentication and per-run audit logs. The project focuses on operationally legible, idempotent hardening rather than claiming full compliance with a security benchmark.
Stack: TypeScript, Electron, React, Ansible
- implemented - Agentless, idempotent hardening: an Electron/React operator UI driving Ansible playbooks over key-based SSH against grouped Ubuntu inventories, with per-run logs.
- known limitation - Ubuntu-only playbook paths, no central cross-operator audit trail, and coverage scoped to the implemented modules: it is not certified against CIS or any other benchmark.
- planned - Benchmark mapping (CIS), dynamic inventory, in-app script editing, broader distro validation.
github
Skills
Languages
Go · Python · TypeScript · C++ · SQL
AI / LLM
Agent Orchestration · RAG · Hybrid Search (BM25 + Vector) · Reciprocal Rank Fusion · pgvector / HNSW · Retrieval Evaluation
Backend & Systems
FastAPI · Fastify · NestJS · PostgreSQL · Redis · gRPC · RabbitMQ · Bull · Prometheus
Infrastructure
Docker · Kubernetes · GitLab CI/CD · Artifactory · S3 · Cloudflare Workers · Helm