Profile picture
Profile

Software Engineer with 5+ years across the spectrum — from fast-paced 0→1 product development to enterprise-scale, distributed, petabyte-scale systems. I build AI tooling and backend platforms: MCP servers, Copilot agent skills, grounded (RAG) agents, and reliable data contracts & APIs. Comfortable owning a problem end to end — design, build, ship, operate — and equally at home writing the first line of a greenfield product or hardening a system that thousands of engineers depend on. Author of the open-source Razorpay IPN Django Handler (PyPI).

Skills
Model Context Protocol (MCP), AI Agent Development, GitHub Copilot Extensions, LangChain, RAG, AI Evaluation, Prompt Engineering, Distributed Systems, OCI, System Design, Data Governance, Observability, C, C++, Python, Java, Django, Flask, FastAPI, React, Docker, Kubernetes, Jenkins, Git, AWS, GraphQL, CSS, HTML, Bootstrap, PostgreSQL, MySQL, MongoDB, Nginx, Redis, Selenium, JavaScript, Celery, RabbitMQ, Apache Kafka, Scrapy, WebSockets, Payments Integration, Data Structures & Algorithm, CI/CD, Elasticsearch, Node.js, Go, SQLAlchemy, Datadog, Wireshark, Prometheus, Grafana, OpenTelemetry, Jaeger, Airflow, RedShift, gRPC, Model Behavior Analysis, TypeScript, Azure Data Factory, Kusto (ADX), OneLake, SCOPE/Cosmos, Microsoft Fabric, Reliability Engineering, Mulesoft, Azure OpenAPI, Data Modelling, Vector Search, LLmOps, AI Guardrails, LangSmith
Work Experience
Software Engineer II, Microsoft
03/2025 – present | Noida, India
  • •Always-On MCP (Model Context Protocol) Server — built a stdio transport MCP Server exposing a standardized tool layer of 44 tools across 8 domains (entity/lineage search, Kusto & ClickHouse querying, Scope Playground jobs, restatements, DataCop validation) + ~69 governed Copilot skills, letting agents operate petabyte-scale data infrastructure in natural language — with a runtime data-access guardrail (client/environment gating, read-only enforcement) and per-call telemetry; least-privilege by design, turning 20–40 min manual query/lookup tasks into seconds and removing ~50+ eng-hrs/month of manual data-plane toil.
  • •Helix Agent-Skills (5 shipped) — authored and shipped 5+ Helix agent-skills — Data Validation, Data Skew Triage, Restatement, Pipeline Doc Generator, Fiscal-Year UDM Rollover — composing/governing the MCP tools into reusable, evaluated workflows: Data Validation deploys a test pipeline and MetricDiffs prod vs test (incl. downstream L+1) to prove data correctness and emit PR-ready evidence, automating ~2 hrs/PR of manual validation; also extended the MCP with a create_restatement tool, a governed-dimension change, and a multi-skill portal sync.
  • •Skill Observability & Evaluation Loop — instrumented every Helix skill run with per-event telemetry (host-hook + durable run-log outbox → Kusto/Grafana), then closed a gap-driven improvement loop: a response-aware feedback/learning store mines failing runs into generalized rules, an owner-reviewed gate publishes versioned fixes, and adoption + quality dashboards (escape-rate, MTTR, recurrence) turn skill quality into a measured number — moving skill improvement from tribal knowledge to a regression-tested pipeline across the ~69 governed skills.
  • •Pipeline Intelligence — Observability & Governance Platform — built a full-stack dashboard mapping ~108 entities across ~57 services and 8 tiers with live health, right-click blast-radius/impact analysis, and ICM integration across 5 teams — cutting incident triage from 15 min to 5 sec (~180× faster) — plus a RAG Copilot over 80+ indexed docs for natural-language Q&A by non-technical users (replicable across the 105K+ dataset ecosystem).
  • •PIPA — Pipeline Intelligent PR Assistant — built a long-running, event-driven agentic PR reviewer (hybrid neuro-symbolic) grounding a large-context LLM in a 68K+ node live lineage graph (Nitro MCP), Cosmos-backed history, and 100+ rules from 2,500+ real PRs to catch contract, blast-radius, and up/downstream-compatibility defects — offloading ~10–15 min of first-pass review per PR across ~192 PRs/day (~32–48 reviewer-hrs/day, ~4–6 FTE) at 1,000+ reviews/day (~34K commits / 5,768 PRs / 88 repos / 30 days) and ~$0 marginal inference cost; reviews only, approval delegated to APAS.
  • •APAS — Automatic PR Approval System — built a deterministic 6-gate insider-threat + approval control plane on top of PIPA — auto-approving only when an authorized (non-restricted-region) ticket, human approval, resolved comments, and passing policies all hold and a diff-aware intent analysis confirms the code matches the ticket's intent with no data-exfiltration / PII-scope-creep / hidden pipelines; hardened by a compulsory, fail-closed, prompt-injection-defended AI pass — approval kept deterministic and auditable, never LLM-decided — saving ~10 min per auto-approved PR while eliminating an insider-threat/exfiltration risk class.
  • •Oncall Pulse (+ Oncall Triage) — built a production dashboard + Teams bot (serverless, Azure Static Web Apps) — 6-sweep incident collection (~100% ICM coverage), 4-dimension shift scoring with SLA + fairness, and one-click GPT-4o handover — paired with Oncall Triage, an offline root-cause + outcome-attribution engine with reachability-gated confidence that stages comment-only actions and never auto-resolves. Deployed for my team; the scoring framework is parameterized per-team, so across the org's ~100 on-call teams it addresses ~45–90 min/team/week (~100 hrs/week, ~5,000+ eng-hrs/yr, ~2.5 FTE) of toil.
  • •Compete AI — Competitive-Intelligence Dashboard — designed and built a React dashboard — including the data model — that tracks how Microsoft's AI products compare against market/competitor products; partnered with data scientists to cross-validate metric correctness, and the resulting metrics are reviewed ~weekly by the SLT (Senior Leadership Team) to drive business decisions. Integrated the internal AskIDEAS RAG/Q&A framework for natural-language exploration of the competitive dataset — replacing manual competitive research/deck-building.
  • •UDM-Compliant Data Contracts & APIs — designed and shipped UDM (Unified Data Model)-compliant data contracts & APIs — standardizing governed dimensions, standard measures, and declared lineage so petabyte-scale SCOPE/Cosmos datasets become uniformly discoverable, navigable, and queryable via the Always-On MCP tools (natural-language workflows), without bespoke per-pipeline glue; included schema design, API versioning, backward-compatible migrations (incl. source migrations across multiple APIs), fiscal-year-rollover modeling, and reliability engineering that cut manual recovery 70%. UDM is the enabler; MCP is the consumer that exploits it.

    • Software Engineer (promoted from Associate Software Engineer)

    Trellix/Fireeye-McAfee⁠
    07/2022 – 03/2025 | Bengaluru, India
  • •Developed the SaaS platform for Extended Detection and Response (XDR), migrating customer data from individually owned, on-premises EC2 instances to a centralized architecture leveraging Kubernetes clusters and a unified database. This transition reduced infrastructure management overhead, enabled scalable, pay-as-you-go service offerings, optimized resource utilization, and facilitated modular cybersecurity services.
  • •Enhanced the SaaS platform's SOAR (Security Orchestration, Automation, and Response) efficiency & architected integrations by building a robust CI/CD pipeline — automating testing, deployment, and scaling processes — reducing deployment times by 40% and increasing system reliability and responsiveness. Architected integrations with Tenable, Microsoft Azure, VirusTotal, ServiceNow, and Trellix/McAfee, improving platform efficiency by 25%. These integrations (REST API integrations) handled 3 million RPM and supported 500,000 concurrent playbook executions, enabling faster, more effective threat response.
  • •Engineered a high-performance data segregation engine to identify customer-specific threat impacts across billions of records. By transitioning from traditional database queries to Redis key-value pairs and ultimately optimizing to a bit-string representation, achieved a ~99.88% reduction in memory usage and improved query speeds by over 100×, enabling real-time threat analysis and decision-making at scale. Enabled real-time streaming in the analytical system using Kafka, working with ACLs and Kafka cluster management along with Zookeeper.
  • •Built a Search Microservice, consolidating the search functionalities of three existing Trellix products into a single interface, and implemented Elasticsearch for additional search capabilities. This improved data-retrieval speed by 80% and simplified the user experience by eliminating the need for multiple search portals.
  • •Implemented a dynamic rule engine that enables the SaaS platform to define, manage, and execute complex cybersecurity rulesets stored in the database. By eliminating the need for hardcoded logic, this streamlined the creation and deployment of new threat-detection rules — reducing response times and enhancing agility in addressing emerging threats in a rapidly evolving cybersecurity landscape.
  • •Developed the Trellix IAM microservice to enable seamless authentication across internal cybersecurity platforms, significantly reducing human involvement in the authentication process.
  • •Leveraged Generative AI with LangChain, RAG, and gRPC to automate the generation of SOAR marketplace apps — reducing manual effort through intelligent code generation, playbook creation, and seamless microservice communication.
    • Software Engineer, Gammastack⁠
    02/2021 – 06/2022 | Indore, India

    • Built Django/React real-time betting flows for 50+ white-label platforms, consuming third-party RabbitMQ feeds, decrypting and normalizing sports-betting data, and streaming low-latency odds/event updates via WebSockets and GraphQL subscriptions.

    • Designed and scaled the Betfaro betting platform to 50,000+ concurrent users and $1M+ in daily transactions, introducing the New Double Bet model that drove a 60% increase in client profitability and a 40% surge in traffic.

    • Integrated multiple payment-gateway and crypto-payment flows across region-specific providers, handling wallet states, transaction callbacks, failure cases, reconciliation, and client-facing payment status updates.

    • Built Python analytics and fraud-detection pipelines using Apache Airflow, GCP Pub/Sub, and background workers to process player activity, betting events, and payment signals against rules/ML outputs, exporting datasets to AWS Athena.

    • Provisioned multi-cloud betting infrastructure with Terraform across AWS, GCP, and Oracle Cloud based on regional availability, tenant requirements, and provider constraints.

    • Optimized high-volume database schemas and data workflows (MongoDB, Flask), reducing bug resolution time by 15% and release cycles by 30%.

    Education

    Bachelors of Technology (Computer Science)

    Lakshmi Narain College of Technology & Excellence⁠
    08/2017 – 07/2021 | Bhopal, India

    CGPA: 8.55

    Projects
    JobPilot — AI Job-Search & Application Platform⁠, AI-powered job search & apply assistant
    01/2026

    Technologies: Python | Django | PostgreSQL | Celery | Chrome Extension (MV3) | RAG/LangGraph | Docker | Kubernetes

  • •Full-stack Django platform for job discovery, résumé building (ResumeForge), ATS scoring and AI-assisted applications; one-click admin deploy and CI/CD.
  • •Manifest V3 Chrome extension that scrapes ATS forms (Greenhouse, Lever, Ashby, Workday, LinkedIn) and autofills them from the user's profile and résumé, drafting grounded answers with an LLM — human-in-the-loop, never auto-submits.
  • •Résumé-grounded RAG answer generator and a public shareable portfolio; published to the Chrome Web Store via OAuth.
    • Scrape Optimus⁠, A Versatile Web Scraping Solution
    09/2024 – present

    Technologies: Python | Django | Kubernetes | Docker | Jenkins | Redis | Kafka | RabbitMQ | PostgreSQL | Nginx

    Built and self-deployed a production web-scraping SaaS on VPS with CI/CD, payment integration, and 200+ proxies / 50+ VPNs for reliable data collection across 1,000+ websites.

    Razorpay IPN Django Handler⁠, OpenSource Python Library
    10/2024 – 11/2024

    Technologies: Python | Django | PyPI | GitHub | GitHub Actions

    Published an open-source Django library on PyPI for handling Razorpay IPN webhook events (payments, orders, subscriptions) with signal-based event tracking for easy integration.

    For a comprehensive portfolio of 12+ personal projects⁠, I have built and deployed, visit: arpansahu.space/projects.
  • •Built and self-managed a cost-efficient infrastructure (Hostinger VPS) running PostgreSQL, Redis, Docker/Harbor, Kubernetes, Jenkins, Kafka, RabbitMQ and Nginx with SSL — hands-on production-ops and DevOps experience across multiple personal projects.