- OS
- Ubuntu 24.04
- Kernel
- CUDA 13.3
- GPU
- RTX 3090 24 GB + RTX 5080 16 GB
- Shell
- CUTLASS / CuTe
- Lang
- C++, CUDA, Python
- Work
- kernel engineering, inference optimization
emre570@gravitron
-----------------
I write CUDA kernels and make inference fast. This is where the measurements, the dead ends, and the working code end up.
about
hi, i'm emre. ml systems & performance engineer.
i build and optimize inference systems for production: low-latency audio models, custom CUDA / CuTe-DSL kernels, and the infrastructure around them that makes them ship. i care about what the GPU actually does. measured, not assumed.
every problem has a solution. the important part is asking the right questions, learning continuously, and finding the most efficient way to break the problem into its parts. don't waste time over-planning. start somewhere, the rest will come.
links
- github
- github.com/emre570
- linkedin.com/in/emre570
experience
- Emulated Inc.RL Environment Engineer (Contractor)San Francisco, USAJul 2026 - Jul 2026, 2 weeks
- RL Evaluation Environments for GPU-Kernel Agents: Designed and delivered 9 RL evaluation task environments with a three-gate scoring harness (compile, correctness, measured wall-clock) for LLM agents writing CUDA kernels.
- Freya (YC S25)ML Performance EngineerSan Francisco, USAJan 2026 - May 2026
- Bifrost, Custom Inference Engine: Building a purpose-built kernel engine optimizing STT and TTS pipelines (Qwen3-ASR, Spark TTS, vocoder) on H100/B200 GPUs for a voice AI agent platform serving call centers.
- Wrote custom CUTLASS FP8 GEMM kernels with fused prologue (scaling + format conversion), achieving 3.8× speedup over PyTorch on ASR shapes, peak ∼1 PFLOP/s on H100. Eliminated over 63K separate GPU task launches per transcription.
- Migrated STT from dynamic per-token to static per-tensor FP8 scaling, yielding 13.1 ms latency reduction (19.3%) with zero CER regression.
- Converted TTS vocoder from FP32 to BF16, halved latency reduction from 6.94 to 3.5 ms/chunk with zero quality impact.
- Fused residual add + RMSNorm + FP8 quantization into single CUDA kernel, made 5.3x faster, eliminates up to 5,000 GPU task launches per transcription.
- Fused GEMM + GELU + bias into single kernel via CUTLASS EVT, 18% throughput gain and achieved 550 to 649 TFLOP/s
- Wrote custom Triton kernel for TTS Snake1d activation, 2.7x faster (BF16), 3 TB/s memory throughput at scale.
- Profiling & Pipeline: Built custom GPU profiling infrastructure (wall-clock, kernel-level, layer-level) by instrumenting vLLM internals and TTS serving code. Identified all optimization targets including 13.4% of STT GPU time as pure FP8 scaling overhead.
- E2E 385 ms TTFT on 2× NVIDIA B200 at conc 1; sub-500 ms up to conc=12. Throughput changed from 0.77 to 7.39 pipe/s (9.6x) at conc=20.
read: 8.5x Faster Speech-to-Text: From 429ms to 50ms on a Single GPU
- Bifrost, Custom Inference Engine: Building a purpose-built kernel engine optimizing STT and TTS pipelines (Qwen3-ASR, Spark TTS, vocoder) on H100/B200 GPUs for a voice AI agent platform serving call centers.
- Preference ModelML Systems Engineer (Contractor)San Francisco, USAJan 2026 - May 2026
- RL Environment for CUDA Kernel Code Generation: Designed RL environments where LLM agents propose CUTLASS kernels; environment compiles, validates correctness, and rewards measured speedup over PyTorch baseline on NVIDIA H100.
- Curated 54 task suite of fused inference kernels (attention, RMSNorm, GEMM, SwiGLU-MLP, vision tower) spanning prefill/decode shapes from M=1 to M=4096.
- Top agent-generated kernels observed as 16.5x peak / 4-5x median speedup over naive baselines, selected Qwen3.5-9B as production policy model after evaluating multiple open-source LLM backbones.
- RL Environment for CUDA Kernel Code Generation: Designed RL environments where LLM agents propose CUTLASS kernels; environment compiles, validates correctness, and rewards measured speedup over PyTorch baseline on NVIDIA H100.
- DeepHist Eng.AI EngineerTokat, TurkiyeSep 2024 - Dec 2025
- Real-Time Garment Counter: Deployed YOLO12x across 16 concurrent camera feeds at 20 FPS with 5 to 15% counting error in production. Integrated FFmpeg + MediaMTX with auto-reconnect.
- DLModule: Built modular internal pipeline covering dataset ops, training, inference, and utilities; adopted across multiple projects. Created Docker-based deployment flow for production services.
- Model Training: Executed multi-GPU sweeps (120 GPU-hours) raising mAP50–95 from ∼40% to ∼90% via custom augmentations and dataset restructuring.
education
- Tokat Gaziosmanpasa UniversityBachelor's, Computer Engineering, 3.25 GPATokat, TurkiyeSep. 2020 - Jul. 2024
open source
| repo | what |
|---|---|
| cublade | My CUDA kernel library. Hand-written SM86 WMMA matmul at ~86% of cuBLAS, INT8 / FP8 / MXFP4 / NVFP4 quantize and dequantize kernels, benchmarks. |
| NVIDIA/cutlass #3184 | Snake activation as a two-operand EVT functor: fused GEMM + activation for neural vocoders. 2.1x median, 8.5x peak on Freya-TTS WaveGenerator. Merged. |
| Dao-AILab/quack #96 | Consumer Blackwell (SM120 / SM121) support for RMSNorm, softmax and cross-entropy kernels, CuTe-DSL. Merged. |
blog
- [0007]Writing A Megakernel For LLM Decode - A Worklog
- [0006]Unleashing Blackwell's 4-bit: a surgical look at MXFP4 and NVFP4
- [0005]From 429 GB/s to the DRAM wall: writing an FP8 quantizer on an RTX 5080
- [0004]8.5x Faster Speech-to-Text: From 429ms to 50ms on a Single GPU
- [0003]W8A16 Quantization with LLM.int8-Style Outlier Handling
- [0002]How Critical Are Outliers in Transformer Models? A Live Experiment - Phase 1
- [0001]Transformer Architecture: Building Blocks Explained