Home
Posts About Me
Home

About Me

Gautam Reddy K

Hi! I am an AI Systems Researcher and Kernels & Performance Engineer — I build deep learning frameworks from scratch and optimize them at every layer of the stack ,,, ,

I write raw CPU and CUDA kernels, hunt down every wasted clock cycle, and engineer AI systems that squeeze every last TFLOP out of the silicon — because making GPUs go brrrrr is both the art and the science of scaling intelligence.

Research & Interests

My work centers around building and scaling deep learning systems from first principles :

  • CUDA & CPU Kernel Development: Designing and implementing custom GPU and CPU kernels for deep learning operations — from matmul libraries and tensor operations to attention mechanisms and optimizers ,,, ,
  • Performance Engineering: Profiling, benchmarking, and optimizing AI workloads to extract maximum throughput and minimum memory footprint from hardware ,,, ,
  • AI Systems & Framework Architecture: Building complete training frameworks from the ground up from first principles ,,, ,
  • GPU Acceleration & Low-Level Systems: Hardware-aware programming in C++ and CUDA, memory layout optimization, and systems-level engineering for compute-intensive workloads ,,, ,
  • Distributed Training & LLM Systems: Multi-mode parallelism strategies for training large language models at scale ,,, ,
  • BluTrain: A C++/CUDA Framework for AI Systems arXiv:2606.24780 A robust, lightweight, and architecture-general training framework built from first principles in standard C++ and the core CUDA programming model. Every layer is implemented natively: a typed tensor module with reverse-mode autograd, a linear-algebra library, a caching allocator, a multi-mode distributed-execution module, and an MLIR-based deep-learning compiler. In formal evaluations training a 124M-parameter GPT-2 in FP32 on an 8-GPU RTX 6000 Ada system, BluTrain outperforms PyTorch in throughput (407K tokens/s vs 395K tokens/s) and memory efficiency (up to 22% footprint reduction), while strictly preserving numerical fidelity.

I created this blog to document technical deep dives, mathematical derivations, CUDA profiling experiments, and summaries of state-of-the-art research papers — sharing what I learn while building AI systems from the ground up ,,, ,

Contact & Links