Utkarsh Malaiya

Utkarsh Malaiya

CS Engineering student at VIT Vellore, specialising in AI/ML; building autonomous systems, LLM pipelines, and things that learn.

Reach me at malaiya.utkarsh@gmail.com.

Education

Vellore Institute of Technology, Vellore
B.Tech, Computer Science Engineering — GPA 9.45
Relevant: OOP, Data Structures & Algorithms, Linear Algebra, Discrete Mathematics
July 2024 – July 2028
The Emerald Heights International School, Indore
High School Diploma, CBSE — 10th: 95.2% · 12th: 96.2%
Physics, Chemistry, Maths, Informatics Practices
April 2023

Experience

Robotics Lab, VIT Vellore — Undergraduate Researcher
  • Maintain and develop a ROS 2-based industrial setup for operating a UR5 and conducting safe, efficient human-robot collaboration experiments.
  • Built a multi-view triangulated, gesture-based manipulation pipeline and validated it in Gazebo and on physical hardware, replacing teach-pendant path definition with natural hand gestures.
  • Implemented a C++ and MoveIt 2 action server for hybrid joint-space and Cartesian trajectories, including TF2 transforms and automatic tool orientation.
September 2025 – Present · Vellore, Tamil Nadu
Robotics Research Center, IIIT Hyderabad — Research Intern, Visual Robot Navigation
  • Engineered a modular data pipeline that converts raw RGB, odometry, and pose streams from real-world navigation datasets into standardized pairwise relative 3D topological graphs for MASt3R-Nav.
  • Optimized dense pointmap generation and intra-image graph sparsification, reducing prior-map memory and compute bottlenecks while preserving 3D geometric connectivity.
May 2026 – July 2026 · Hyderabad, Telangana
NovaLinks Technologies — AI/ML Research Intern
  • Trained a domain-specific language model, handling data curation, model configuration, and the pre-training run.
  • Set up and managed the training pipeline using HuggingFace Transformers with custom hyperparameter schedules.
December 2025 · Vellore, Tamil Nadu
Google Developer Groups on Campus, VIT Vellore — Technical Inner Core Member
  • Led TruthSense, an AI-driven speech-coaching platform with real-time speech emotion recognition, as Lead ML Developer and Project Manager from signal-processing pipeline through deployment.
  • Architected and maintained the backend for the 2026–27 Recruitment Portal, supporting thousands of applicants through production debugging and admin-portal optimization.
July 2025 – February 2026 · Vellore, Tamil Nadu
Team VAUV, VIT Vellore — CS & Autonomous Team Member
  • Designed and developed a ROS 2 and MAVLink control stack for a six-degree-of-freedom Autonomous Underwater Vehicle.
  • Built a Stonefish simulation pipeline to validate control and perception algorithms before deployment on physical hardware.
  • Fine-tuned YOLOv11 for multi-gate detection in underwater settings; deployed on Jetson Orin Nano — mAP 0.97, accuracy 85%.
July 2024 – Present · Vellore, Tamil Nadu
xWell Technologies — AI/ML Backend Developer
  • Built a multi-tenant, scalable healthcare-AI platform generating personalised healthcare plans via in-context learning with LLMs.
  • Designed backend architecture integrating LangChain, Pinecone, and Azure Blob Storage with async task scheduling.
  • Developed prompt-engineered RAG pipelines with parallel async execution, improving response latency across client tenants.
  • Implemented task orchestration, error handling, and logging for production-grade distributed AI services.
July 2023 – Oct 2023 · Remote
DataFlair Web Services — Technical Content Developer (Python & AI)
  • Developed and deployed 20+ Python GUI projects using Tkinter for real-world demonstration tutorials.
  • Implemented and documented foundational AI/ML models using scikit-learn and NumPy.
  • Collaborated with SEO and review teams to produce high-quality, publication-ready content.
Jun 2021 – Oct 2021 · Remote

Skills

Languages
Python
C
C++
HTML/CSS
JavaScript
AI / ML
PyTorch
HuggingFace
Scikit-learn
NumPy
Pandas
LangChain
LangGraph
YOLOv11
Backend & Infra
FastAPI
Docker
Git
Linux
ROS
Azure
Pinecone
Streamlit
Hardware & Embedded
Arduino
ESP32
Raspberry Pi
NVIDIA Jetson

Projects

Diffusion Models from Scratch

April 2026
Python PyTorch DDPM Generative AI
  • Implemented denoising diffusion probabilistic models from first principles, reproducing the forward noising and reverse denoising processes in PyTorch.
  • Built a configurable noise scheduler supporting linear, log-linear, cosine, and sigmoid schedules for controlled experimentation.

Dolphin Agent — Local Voice-First AI Agent

April 2026
LangGraph Ollama Whisper Mem0 Textual
  • Built a local voice-controlled agent that transcribes requests, reasons with local models, and chains filesystem actions through a terminal interface.
  • Implemented persistent cross-session memory, visible execution traces, sandboxed tools, and human approval before every file modification.

Controllable Neural Style Transfer

April 2026 · Live demo
React TypeScript TensorFlow.js WebGPU
  • Created a fully client-side neural style transfer app with independent controls for colour, texture strength, and brushstroke scale.
  • Ran inference in the browser with WebGPU and WebGL fallback, using embedding interpolation and luminance-preserving chroma mixing for real-time control.

Transformers from Scratch — Attention Is All You Need

March 2026
Python PyTorch Transformers Weights & Biases
  • Implemented the original encoder-decoder Transformer architecture in PyTorch for English-to-Hinglish translation.
  • Developed the word-level tokenizer, dataset pipeline, attention masks, training and validation loops, metrics, W&B logging, and checkpointing.

TruthSense — Multimodal Communication Coach

2025 · Project Manager, Backend & AI
Project Management FastAPI Multimodal AI MediaPipe Groq
  • Led the project and owned delivery across the backend and AI stack for a system that evaluates communication skills from audio and video.
  • Built the asynchronous FastAPI analysis service and pipelines for transcription, fluency, speaking rate, pauses, posture, eye contact, and gestures.
  • Fused multimodal features with LLM-generated coaching feedback and enforced a validated Pydantic response contract for frontend integration.

StoryGPT — AI-Powered Interactive Story Generator

June 2023
Streamlit OpenAI GPT API Stable Diffusion LangChain
  • Built an AI storytelling platform generating interactive, choice-based narratives and scene visuals using OpenAI GPT and Stable Diffusion.
  • Designed a context-aware generation pipeline with LangChain and Streamlit, enabling seamless text-image synchronisation and real-time story progression.

Writing

My Learnings from Building Deep Learning Models from Scratch

October 16, 2023 · Artificial Intelligence in Plain English

Building deep learning architectures from scratch isn't just a résumé line — it's a great way to truly understand what's happening under the hood.

Read on Medium

Using Reinforcement to Make Machines Human-Like

February 13, 2022

Autonomous vehicles, Watson cracking jokes — exploring how reinforcement learning brings machines closer to human-like behaviour.

Read on Medium