Aditya Karnam · AI Research Notes
Thoughts / AI Research
Every post, in one place.
29 posts on agents, memory, local inference, and the infrastructure layer behind AI systems.Also Writing
calltothink.com ↗Call to Think
The posts here are about systems. Call to Think is the other half — essays on technology, AI, and society, and how these tools change the way we think. Written slowly, on purpose.All Posts
Latest first
02.09.2026
Tokens Per Paper: DocLang vs Docling vs Raw PDF for Feeding Research Papers to an LLM
I converted a 23-page Nature paper to DocLang, embedded it three ways, and counted every token. DocLang costs 2.7× markdown on a stock tokenizer and loses on retrieval. Docling's structured text wins, and 38% of the paper was bibliography and journal boilerplate the summarizer never needed.Read post01.09.2026
LoRA Fine-Tuning Crashed on Three MoE Models in mlx-lm. The Fix Was One Line, Three Times.
mlx_lm.lora --train crashed on granitemoe, granitemoehybrid, and lfm2_moe with a VJP error on integer routing indices. Found while fine-tuning Qwen3-30B-A3B on an M5 Pro, traced to a missing mx.stop_gradient, fixed and merged upstream as mlx-lm PR #1795.Read post26.08.2026
A Free Local 9B Model Optimized My Rust Agent Harness. Two of Its Three Claims Were Wrong.
ornith-9b, running fully local and free through the Claude Code harness, wrote a real performance PR against quecto with three optimizations. Claude Sonnet 5 verified the diff line by line: one claim was correct, one was mislabeled, and one didn't do what it said — and got fixed for real before merge. Released as quecto-agent v0.2.2.Read post22.08.2026
Speculative Decoding on Apple Silicon: 119% Faster on Qwen3.8, Until It Isn't
Measured on an M5 Pro: pairing Qwen3.8-27B with a 0.8B draft model peaks at a 119% decode speedup, the best result in this series, but it does not work out of the box. mlx_lm refuses to run speculative decoding against Qwen3.8's hybrid attention cache at all; getting a real number required patching it. Includes the original Qwen3-8B result (58% peak) and a Qwen3.6 self-speculative MTP comparison.Read post14.08.2026
The Batch Size That Breaks Local LLM Serving on Apple Silicon
Measured on an M5 Pro: batching concurrent requests raises aggregate throughput right up until peak memory approaches physical RAM, then throughput collapses and the process eventually has to be killed. The wall is unified memory, not compute.Read post13.08.2026
Why Batching Doesn't Fix Decode: GEMV vs GEMM on Apple Silicon
Measured on an M5 Pro: batched matmul (GEMM) holds a flat ~80x throughput advantage over matrix-vector ops (GEMV) at every size tested. That gap is why batching multiplies prefill throughput but barely touches decode.Read post11.08.2026
MLX Errors on Apple Silicon: A Reference
Verbatim MLX and mlx-lm error strings with verified causes: model type not supported, Metal insufficient memory, quantize group size, and float16 NaN.Read post11.08.2026
Why MLX Decode Slows at Long Context
Measured on an M5 Pro: quantizing the KV cache halves how fast decode degrades with context, but halves prefill throughput too. Break-even math included.Read post10.08.2026
Agent Plugins v1.0.0: One Plugin Format Instead of Five
Agent Plugins v1.0.0 standardizes just 2 component types across 7 clients. What the portable core fixes, what it defers, and why Claude Code is missing.Read post25.07.2026
Building QuECTO: From Minimal Agent Harness to Evaluable Coding Agent
The second entry in the QuECTO build series: how a 1.3 MB core grew into a 3.5 MB coding agent with reasoning controls, verification gates, session persistence, subagents, multimodal input, telemetry, and a native evaluation harness.Read post22.07.2026
MEDFIT-LLM Explained: Fine-Tuning Small Language Models for Medical Tasks
A detailed walkthrough of my paper MEDFIT-LLM: a controlled, multi-architecture study of how domain-focused fine-tuning lets small language models compete with much larger general-purpose models on medical tasks.Read post22.07.2026
What Is an AI Agent Harness? A Systems Guide to Building and Evaluating Reliable AI Agents
Learn what an AI agent harness is, how it controls tools, context, memory, retries, verification, cost, and reliability, and how agent harnesses can be evaluated systematically.Read post20.07.2026
How I Built a Persona-Adaptive RAG Chat Into My Portfolio's Hero Section
I replaced my portfolio's standalone /ask page with a grounded, multi-turn RAG chat embedded in the homepage hero, built on Cloudflare AI Search. Here's the architecture and what broke.Read post20.07.2026
I Turned My Portfolio Into an MCP Server (And Gave It a UI)
I shipped a hosted, read-only MCP server for adityakarnam.com, plus an interactive Fit Check and Projects app any MCP client can open. Here's how it works and why I built it.Read post18.07.2026
India's Next AI Unicorns Won't Just Build Models — They'll Build the Agent Infrastructure Layer
Emergent's $1.5B round, HCLTech's ₹3,500 crore data centers, and Karnataka's policy point at one gap: India's missing agent infrastructure layer.Read post08.07.2026
Ollama vs vLLM vs SGLang on Apple Silicon
Latency and response-quality results for Ollama, vLLM Metal and SGLang on one M5 Pro Mac, plus a Qwen 3.5 sweep from 0.8B to 9B judged by Gemma 4.Read post01.07.2026
subagent-fleet: Local AI Compute Control Plane for Coding Agents
I built subagent-fleet to route Claude Code-style subagents across local Ollama machines, with LiteLLM generation, health checks, model warmup, and a live dashboard.Read post11.04.2026
AI Blog Generator: I Built an n8n Bot That Wrote 139 Posts
I built an n8n workflow pulling from 7 RSS feeds, used Google Gemini to write MDX posts, and auto-published 139 blog posts to GitHub. Here's what Google Search Console showed.Read post05.04.2026
embenx: One Python API for 15+ Vector DBs
embenx gives one Collection API over 15+ vector backends including FAISS, pgvector and Qdrant, plus hybrid search, temporal memory and a built-in MCP server.Read post15.09.2025
MLX Non-Determinism on Apple Silicon
Why identical MLX inputs give different outputs: batch-invariance tests from 512 to 4096 matrices, plus bfloat16, float32 and float16 error behavior.Read post11.09.2025
How I Accidentally Built an Overkill ML Lab That Rivals AWS (And Why I'm Not Sorry)
Building a self-hosted ML lab on Mac minis with 64GB unified memory: Ray cluster, arm64 Docker, MinIO object storage and Streamlit, and what it cost versus AWS.Read post04.09.2025
Unlock Your True Purpose: Ancient Wisdom for a Meaningful Life (Stop Postponing Yourself!)
Discover how ancient Stoic wisdom can help you create meaning, embrace virtue, and stop postponing your authentic life. Learn from Viktor Frankl, Seneca, and Marcus Aurelius.Read post25.08.2025
How to Diagnose a Bad Prompt (with Free Grader Tool)
Learn to identify and fix poorly performing AI prompts with our comprehensive guide and free grader tool. Transform weak prompts into high-performing instructions that get 10x better results.Read post25.08.2025
Prompt Grader vs. Prompt Libraries: When to Use Each
Discover when to use prompt graders versus prompt libraries for optimal AI results. Compare approaches, see real examples, and learn which tool fits your specific use case.Read post25.04.2025
PostgreSQL Sharding Example With FDW
Manual sharding in plain PostgreSQL, no Citus: shard tables, routing insert and read functions, a unified view, range reads, and insert/read benchmarks.Read post12.04.2025
The Hike to Hanuman!
Every social media feed is filled with people conquering mountains. Someone is scaling Everest, another person is posting perfect yoga poses…Read post11.04.2025
Friday Coffee OR BEER!
Hey folks! Welcome to another edition of our Friday ritual where we decide between caffeine or hops (let's be real, sometimes both are…Read post04.04.2025
The Motivation: From Suffering to Spiritual Awakening
We all face moments when everything seems to be falling apart. When deadlines pile up, projects fail, relationships strain, and the path…Read post01.04.2025