Varun Singh

Go / Python / TypeScript · retrieval / agent orchestration / Go services

About

Hey i'm Varun Singh, a Backend and Systems Engineer, I love building AI Infrastructure and retrieval systems, agent workflows, and the services underneath them with a bias toward scaling, measurable performance, and reliability. At Oracle I identified a gap and architected Composer, an AI workflow platform that turns 5G requirements into structured test plans and automation-ready bundles, and redesigned the Prometheus collection layer for 5G performance reporting, cutting a 96-logical-site, 24-hour benchmark from 3 hours 58 minutes to 45 minutes. Outside work I build in the open: hybrid retrieval, Go concurrency primitives, and infrastructure tooling, I can never sit still and always try to learn something new. Currently going deeper on inference serving batching, routing, and GPU utilization as i strongly believe that the future of AI is on device. I love playing Valorant, roguelike games, fiction, and exploring new places

SDE-1 @ Oracle · IIIT Jabalpur '25 · SIH 2023 national winner · Code for GovTech (C4GT)

Selected results

Experience

Associate Software Developer (SDE-1) - Oracle

Jul 2025-now · Hybrid

Architected Composer, an AI workflow platform that turns 5G requirements into structured test plans and automation-ready bundles, reducing manual authoring effort by 50% across 6+ network-function teams. Redesigned the Prometheus collection layer for 5G performance reporting using bounded concurrency, persistent sessions, and coordinated retry/backoff. Led a backward-compatible migration from an overloaded log server to Artifactory.

  • Composer: built intent-driven selection over 65k+ domain-organized JSON examples, versioned generation skills, deterministic validation, resumable finite-state workflows, immutable artifacts, and approval gates, reducing manual authoring effort by 50% across 6+ network-function teams.
  • Prometheus telemetry: redesigned collection using bounded concurrency, persistent HTTP sessions, query packing, and coordinated retry/backoff, cutting a 96-logical-site, 24-hour benchmark from 3h 58m to 45m, an 81% reduction and 5.3x speedup, while completing 73K+ PromQL requests with zero terminal failures.
  • Artifactory and reliability: led a backward-compatible migration from an overloaded log server using reusable upload abstractions and direct bastion-to-Artifactory transfers across Robot, network-function, and OSO artifacts while preserving downstream paths. Centralized Paramiko, JumpSSH, and SFTP lifecycle cleanup to eliminate recurring SSH and file-descriptor leaks.

Project Intern - Oracle

Jan-Jun 2025 · Hybrid

Developed worker-utilization reporting for performance runs and refactored dashboard state management, cutting UI-related support tickets by 80% for 75+ engineers.

  • Worker-utilization reporting for performance runs, surfacing resource bottlenecks.
  • Refactored test-scheduling dashboard state management: 80% fewer UI-related support tickets for 75+ engineers.

Consulting Backend Engineer - Trijog

Feb-Apr 2024 · Remote

Owned end-to-end backend delivery for an LMS with 20k+ registered and 250+ peak concurrent users. Built the 7-module NestJS backend from scratch and engineered a Bull + FFmpeg pipeline that cut video readiness from 30 minutes to 5.

  • Built the 7-module NestJS backend from scratch with 3-role RBAC, Metabase reporting, async jobs and private DigitalOcean Spaces: 20k+ registered, 250+ peak concurrent users.
  • Bull + FFmpeg processing for 400+ videos with multipart uploads and 5 HLS renditions, cutting readiness from 30 minutes to 5; a Cloudflare Worker enforced signed, expiring auth on every manifest and segment.

Projects

repi

local-first log RCA / hybrid retrieval / resumable agent

An autonomous log-investigation engine for distributed systems. It clusters repetitive logs by signature, retrieves evidence with BM25 + pgvector/HNSW fused through Reciprocal Rank Fusion, and investigates root cause through an explicit, resumable state machine. Every step is persisted for auditability, and confidence is constrained server-side by the evidence actually cited. It can run fully locally with Ollama; hosted-model adapters are optional.

Stack: Python, Postgres, pgvector, ParadeDB, FastEmbed

  • implemented - Hybrid retrieval: ParadeDB BM25 + pgvector/HNSW fused with Reciprocal Rank Fusion.
  • implemented - Resumable investigation: a hand-rolled finite-state machine whose thoughts, actions and observations persist to Postgres, so a paused run resumes from stored state.
  • implemented - Deterministic server-side confidence gating that downgrades conclusions the cited evidence does not support.
  • measured - Docker image reduced from 5.58 GB to 1.12 GB by migrating embeddings to ONNX with FastEmbed.
  • measured - A 21-question retrieval suite plus scripted incident evaluations. This is a functional suite, not a published benchmark against a public dataset.
  • planned - Cross-encoder reranking and a vector-only / BM25-only / hybrid ablation. Neither exists in the code today.

github · demo

collapser

gRPC request-collapsing sidecar / 2,000 concurrent identical requests reached the backend as 1 call

An open-source Go sidecar that suppresses thundering-herd traffic. Equivalent unary requests share one backend call while followers wait on the same result. Envoy, which it runs alongside in a mesh, load-balances and retries those N requests but has no notion that they are the same request; this fills that gap. The proxy is schema-agnostic - a passthrough codec forwards raw gRPC frames, so no generated stubs are needed - runs backend work on a detached context so one cancelled caller cannot kill shared work, and exposes collapse, cache, in-flight and backend-latency metrics through Prometheus. It ships with Kubernetes and Istio manifests plus a kind-based cluster harness that counts backend calls at the backend rather than trusting the proxy's own accounting.

Stack: Go, gRPC, Kubernetes, Istio, Prometheus

  • measured - On a local kind cluster running Istio 1.28.1 with sidecar injection and STRICT mTLS, identical concurrent requests through the deployed proxy reached the backend as a single call: 100 concurrent -> 1 call (99.0% fewer backend calls), 500 -> 1 (99.8%), 1,000 -> 1 (99.9%), 2,000 -> 1 (108ms wall against 50ms of backend work). Backend calls were counted at the backend and cross-checked against Envoy's istio_requests_total{reporter="destination"}, not taken from the proxy's own metrics. This is a local single-node kind cluster under synthetic identical load, not production traffic.
  • benchmarked - Go microbenchmarks on an 11th Gen Intel Core i7-1165G7 (8 threads, Go 1.25.5), in-process only - no network, no serialization, no backend time. Cache-hit path 45.7 ns/op, 0 B/op, 0 allocs/op. Follower (collapse) path 54 B/op, 0 allocs/op; its ns/op is dominated by however long the backend takes, so the allocation count is the number that means anything. Leader path 1,594 ns/op, 552 B/op, 10 allocs/op.
  • benchmarked - In-process stress with the race detector enabled: 10,000 concurrent requests on one key produced 2 backend calls (5,000:1) with 0 errors; a 60-second sustained run with 100 workers put through 391,400 requests as 392 backend calls (998.5:1) at 6,523 RPS with 0 errors. Zero goroutine leak and 0 MB heap growth across the run. Synthetic contention, not production traffic.
  • measured - Test coverage: internal/collapser 97.5%, internal/config 95.8%, internal/proxy 33.7%. The proxy package is the thin one.
  • implemented - Detached backend context, so a cancelled leader does not kill the shared call its followers are waiting on.
  • implemented - Schema-agnostic collapsing through a passthrough codec over raw gRPC frames: no protobuf descriptors or generated stubs required.
  • implemented - COLLAPSER_KEY_HEADERS folds named identity headers into the collapse key, so requests that differ only by tenant or authorization get separate backend calls.
  • implemented - Prometheus metrics for requests, collapsed requests, backend calls, cache hits, cached entries, in-flight leaders and a backend-latency histogram; plus a liveness endpoint that ignores the backend and a readiness endpoint that reports the backend channel state.
  • implemented - Graceful shutdown draining in-flight calls on SIGTERM, panic recovery published as codes.Internal, and a configurable cache entry cap (10,000 by default) so a key-diverse workload cannot grow the map without bound.
  • implemented - Kubernetes and Istio manifests (Deployment, Service, VirtualService, DestinationRule, PeerAuthentication), a kind cluster harness, an in-mesh load generator, and a GitHub Pages project page.
  • known limitation - Server-streaming RPCs are treated as unary and return only the first response frame, because without a proto descriptor they are indistinguishable from unary calls on the incoming connection; client-streaming and bidirectional calls are detected and forwarded correctly but bypass collapsing entirely. Backend TLS uses the system trust store only - no mTLS, no custom CA bundles - though inside a mesh Istio's sidecars handle that. The result cache is per-process, so N replicas can produce up to N backend calls for one key. The passthrough codec registers globally as "proto", so internal/proxy cannot be imported into a process needing normal protobuf marshalling.
  • known limitation - With the default collapse key (method plus payload) two callers sending identical payloads share one response even if they authenticated as different users. Naming the identity headers in COLLAPSER_KEY_HEADERS is what makes that safe, and it is opt-in.
  • planned - Optional proto descriptors, mutual TLS and custom CA bundles, and per-method collapse policy. None of these are in the code today.

github · demo

Fend

fleet-wide Linux hardening / Ansible + Electron

A desktop operator tool for applying repeatable Linux hardening across grouped Ubuntu hosts. Electron/React drives agentless Ansible playbooks over SSH, with key-based authentication and per-run audit logs. The project focuses on operationally legible, idempotent hardening rather than claiming full compliance with a security benchmark.

Stack: TypeScript, Electron, React, Ansible

  • implemented - Agentless, idempotent hardening: an Electron/React operator UI driving Ansible playbooks over key-based SSH against grouped Ubuntu inventories, with per-run logs.
  • known limitation - Ubuntu-only playbook paths, no central cross-operator audit trail, and coverage scoped to the implemented modules: it is not certified against CIS or any other benchmark.
  • planned - Benchmark mapping (CIS), dynamic inventory, in-app script editing, broader distro validation.

github

Skills

Languages

Go · Python · TypeScript · C++ · SQL

AI / LLM

Agent Orchestration · RAG · Hybrid Search (BM25 + Vector) · Reciprocal Rank Fusion · pgvector / HNSW · Retrieval Evaluation

Backend & Systems

FastAPI · Fastify · NestJS · PostgreSQL · Redis · gRPC · RabbitMQ · Bull · Prometheus

Infrastructure

Docker · Kubernetes · GitLab CI/CD · Artifactory · S3 · Cloudflare Workers · Helm

Achievements

Education

Indian Institute of Information Technology, Jabalpur - B.Tech, Computer Science and Engineering, Dec 2021-May 2025

Contact

Varun Singh

esc

Go / Python / TypeScript

retrieval / agent orchestration / Go services

ai · tool-calling assistant on the edge · 5 questions/day