Joseph CasilangSenior AI Software Engineer
Profile
Senior AI Software Engineer specializing in highperformance, HIPAAcompliant AI systems, multiagent orchestration, and enterprisescale multimodal pipelines. Architect of production LLM platforms delivering sub120ms latency, 99.995% uptime, and 40% cost reduction across healthcare environments. Expert in hybrid semantic search, multicloud model routing, GPU autoscaling, Zero Trust AI security, and multimodal STT→LLM→TTS systems. Known for driving measurable outcomes including 70% clinician workload reduction, 35% retrieval accuracy gains, and 25% faster release cycles.
Skills
AI & Machine Learning: LLMs (GPT4o, Claude 3.5, Gemini), LangGraph, MCP, multiagent systems-, RAG pipelines, hybrid semantic search (FAISS, pgvector, Pinecone), multimodal embeddings, ONNX optimization, quantization (AWQ, FP8), hallucinationreduction frameworks, evaluation pipelines, synthetic data generation, multimodal classification-, contrastive training
Backend & Platform Engineering: Python (FastAPI, Django), Node.js, Go, Java, microservices, eventdriven architecture (Kafka, EventBridge), highthroughput REST/GraphQL APIs, WebSockets, distributed systems, Zero Trust backend design, RBAC, encryption, audit logging
RealTime & Voice Systems: Whisper, Deepgram, ElevenLabs, LiveKit, WebRTC, lowlatency streaming pipelines, STT→LLM→TTS agents
Cloud & Infrastructure: AWS (EKS, ECS, Lambda, Bedrock, SageMaker), Azure (AKS, Azure OpenAI, Document Intelligence), GCP (GKE, Cloud Run, Vertex AI), Kubernetes, Docker, Terraform, GPU autoscaling clusters, multicloud orchestration, CI/CD (GitHub Actions), Sigstore signing
Data Systems: PostgreSQL, MongoDB, Redis, Supabase, Snowflake, DynamoDB, ETL/ELT pipelines, dbt, Dagster, PySpark, multiterabyte log analytics
Professional Experience
Roger Healthcare, Senior AI Software Engineer
05/2024 – 04/2026 | Berkeley, CA
  • •Architected a multi-agent clinical AI platform using LangGraph + MCP for autonomous documentation, triage, and care-plan generation, reducing clinician manual workload by 70%.
  • •Implemented GPU autoscaling clusters on EKS and GKE for real-time transcription and multimodal inference, supporting 50,000+ users and 2M+ daily clinical events.
  • •Designed a multi-cloud LLM routing layer across Azure OpenAI, AWS Bedrock, and GCP Vertex AI, cutting inference cost by 40% while maintaining 99.995% uptime.
  • •Built a hybrid semantic search engine (Pinecone + pgvector + Azure Document Intelligence) that improved retrieval accuracy by 35% with sub-120ms latency across millions of clinical documents.
  • •Designed multimodal, multilingual embedding-based clinical search (text, documents, imaging metadata) delivering sub-120ms P99 latency and significantly reducing manual chart review by clinicians.
  • •Collaborated on FAISS/pgvector-based ANN index optimization, deriving empirical scaling laws and tuning latency, recall, and capacity; deployed models with ONNX on Azure AKS, yielding a 30% reduction in query latency and a 5% increase in recall.
  • •Evolved retrieval into a clinical AI Assistant by leading prompt engineering and agentic system architecture (documentation assistance, triage, EMR-aware RAG) using Azure OpenAI and Azure AKS, taking the solution from POC to full production and cutting documentation time by ~15%.
  • •Built real-time STT → LLM → TTS agents using LiveKit + WebRTC, reducing response time by 45% and boosting clinician satisfaction.
  • •Delivered Zero Trust AI security with OIDC federation, KMS encryption, SOC2/NIST-aligned controls, and PHI-safe pipelines, passing all compliance audits with zero findings.
  • •Modernized CI/CD with GitHub Actions and Sigstore signing, eliminating deployment errors and accelerating release cycles by 25%.
  • •Led AI research for a large-scale clinical platform and co-authored applied work on intent discovery in clinical dialogue systems using domain-specific self-supervised pre-training and noisy weak supervision, improving intent detection accuracy and integrating the model into production.
  • •Developed evaluation frameworks combining interaction logs in Azure Blob, clinician feedback, and LLM-as-a-judge methods to continuously monitor relevance and post-deployment drift, improving retrieval relevance scores by ~12%.
  • •Optimized multi-GPU training pipelines, reducing end-to-end training time from two weeks to four days, and built LLM-driven synthetic data pipelines for clinical query–document pairs to strengthen long-tail and rare-condition retrieval.
  • •Defined metrics and tooling for embedding-based clinical search with analytics teams, using BigQuery and Python over multi-terabyte logs to quantify impact on documentation time and search success rates across the platform.
  • •Raised internal ML standards by organizing monthly seminars, mentoring five engineers, and presenting healthcare AI findings at industry conferences, increasing adoption of best practices and improving model compliance across projects.
    • NextGen Healthcare, Senior Full Stack / AI Engineer
    07/2021 – 04/2024 | Atlanta, GA
  • •Owned a production clinical order-verification service, architecting asynchronous LLM inference with config-driven routing across self-hosted SGLang (1s p95) and cloud providers (Vertex AI, OpenAI, Claude, Bedrock via LiteLLM, 3s p95), with primary/secondary fallback and speculative retry to keep verification available across all clinics.
  • •Designed a continual-learning pipeline that converted real clinical traces into retraining data using Langfuse instrumentation, a custom PyQt5 labeling tool, and MiniCPM-V LoRA fine-tuning (8×A100), forking LLaMA-Factory to add albumentations; reduced false negatives in order checks from 15% to 3%.
  • •Tuned SGLang serving (AWQ quantization, FP8 KV cache, FlashInfer attention, chunked prefill, CUDA graph capture), then load-tested and optimized batching to scale from a single clinic to 11+ sites without latency regressions.
  • •Built an LLM-powered dispute automation service in FastAPI using Gemini + LiteLLM with hybrid routing, 1–5ms template fast-paths, pytest validation, and Langfuse monitoring, achieving a 75% automation hit rate and 80% inference cost reduction for billing and coding disputes.
  • •Developed an LLM agent service with tool-use and RAG (smolagents + MCP for secure database access), with 95%+ of responses validated as correct and actionable across 50+ hand-evaluated clinical and operational queries.
  • •Built RAG assistants for clinical workflows using LangChain, a vector database, and hybrid semantic search, improving documentation accuracy by 32% and reducing follow-up edits by 60%.
  • •Designed multimodal classification pipelines using LoRA-fine-tuned models for entity extraction across 1M+ clinical records, increasing extraction accuracy by 28%.
  • •Architected a distributed scheduling engine with Kafka, FastAPI, and Redis, processing 1M+ daily scheduling events at 99.99% reliability across multi-site provider networks.
  • •Implemented multi-cloud inference routing across AWS Bedrock and Azure OpenAI, reducing latency by 38% and inference cost by 22% for production clinical assistants.
  • •Built clinician-facing React/Next.js dashboards with real-time WebSocket updates, improving workflow visibility and reducing task turnaround time by 35%.
  • •Migrated 500+ enterprise clients to new clinical data models with zero downtime, improving data consistency and cutting support tickets by 40%.
  • •Led development of a production OCR engine powering biometric identity verification for 20K+ daily sessions, extending support to new countries and document types and reducing end-to-end latency by 25%.
  • •Researched and built generative CV/clinical imaging demos, including style-preserving image synthesis and avatar generation using diffusion models, validating new patient UX concepts and informing future identity-workflow designs.
  • •Mentored junior data scientists and engineers, led sprint planning, and enforced ML/engineering best practices within a 5-person team, resulting in faster model deployment cycles and higher code-quality standards.
    • Carmatec, Full Stack Engineer
    06/2018 – 06/2021 | Los Angeles, CA
  • •Built retail + healthcare AI automation systems serving 50,000+ monthly users, integrating RAG pipelines and multimodal search into customer dashboards.
  • •Designed event-driven retail inventory pipelines (DynamoDB + Kafka) achieving sub-200 ms update latency across global storefronts
  • •Migrated legacy systems to AWS EKS + Docker, reducing infrastructure cost by 30% and improving deployment reliability.
  • •Built internal analytics dashboards (React + Next.js + PostgreSQL), reducing reporting turnaround time by 50%.
  • •Mentored junior engineers through code reviews and pair programming, improving team velocity by 20%
    • Carmatec, Frontend Developer
    05/2016 – 06/2018 | Los Angeles, CA
  • •Built high-performance React/TypeScript dashboards with real-time WebSocket updates and Redux state management, reducing clinician decision latency by 25%
  • •Designed reusable component libraries reducing UI development time by 40%.
  • •Implemented performance optimizations (lazy loading, bundle splitting) cutting load times by 35%
    • Education
    Bachelor’s Degree, University of Santo Tomas
    2012 – 2016