Claude Code is no longer bae

I used to love Claude Code (Anthropic) and Codex (OpenAI). They fundamentally changed the landscape of coding in the day in the life of software engineers. However, there have been some recent developments that turned me off.

Silent Rate Limiting: Previously, even on a $20/mo plan, I was able to get a significant amount of work done (e.g. 5 coding prompts). However, now, I can barely get 1 coding change in before running out of credits in a 5 hour window. I understand these plans are heavily subsidized, but from a user’s point of view, this plan no longer works for coding. Either I have to switch to a higher tier (where there’s no guarantee that the rate limit will be nerfed again), or come up with some hacky brilliant solution to rotate between multiple accounts. Also from an ethics perspective, I think not letting users know about this publicly is a dick move.
Disappearance of Fable: Anthropic announced their latest Mythos model was all too powerful to be released to the public caused a lot of FOMO. It could break security, etc. Then they released(?) it with some guardrails (called Fable). I would really like to know the reasoning they did here. (This is an all powerful nuke-level AI, but we are gonna release it on a child-proof model lolz?!?) And objectively, this model was very good. Then due to the pressure from the US government, they pulled it off (US national security, the Chinese AI labs were distilling this, etc.) As someone who decided to pay more to use Fable more only to get it taken away, I got rugpulled. Imagine Netflix getting people to sign up to see the latest hot show and then take the show down after 3 days.
Cost: Now that both companies are no longer resetting limits during the week (they would after they had an outage or bug), I keep hitting the rate limit way more than before. Both companies are talking about going IPO this year, and mathematically, these subsidized plans do not help (e.g. $200/mo plan equates to about $8k of compute).

These observations have made me feel shitty about being a customer; I'm using my hard-earned money and I deserve to get treated like a customer!

At the same time, I’ve been paying more attention to open source LLM and local inference. OpenAI and Anthropic are closed-source AI companies, meaning nothing about their model is open to the public (the actual model is just weights, which is just a list of decimal numbers. And as some model name suggests, the number of weights can range from 1B to >1T. All AI models do is with the weights, they just do a series of matrix multiplication and addition). And when we use their AI, the data (input and output) go directly to the AI companies which they can read and use to train their AI models. (exception is Cloud Virtual Private Compute from AWS/Azure where the cloud providers signed an agreement with the AI companies to host their AI on their servers, and only you can access the servers so the data is all self-contained within the server). Also since these models were closed source, this is like talking to a magic eightball and we don’t know this model actually works.

Alternative is an open-source AI model. Lately, Chinese AI labs have been releasing opensourced LLMs (e.g. Deepseek, GLM, Kimi-K2). Meaning on hugging face, anybody can view the weights, and download the model and use it. They spent a good chunk of money to train these models and they are just releasing them for free? (they don’t release HOW they trained the models, which may be the more important part; more on in a separate blog).

These models were decent, but not as good as the frontier AI lab’s models. People say they are about 6 months behind. So for the large part, they were dismissed from frontier agentic stuff like coding. (I would argue that for normal chat capabilities, these LLMs were more than enough).

So by this logic, the current open source models should have reached the level of Claude Opus 4.5 which came out Nov 2025. This is the model that really made Claude Code viable for coding. Also by this logic, within 6 months, they will release a model capable of Claude Fable which is the mindblowing model of today. I’m a believer that technology will always get better (if they regress they can just revert back to the previous good checkpoint). And unlike the frontier AI labs, these other companies have a clear goalpost: reach/exceed the performance of the leading AI models.

There are few things that also made using these AI models more viable:

1. Distillation: Using the big powerful AI model as the teacher model, you try to get a smaller student model to “mimic” the teacher model. Ideally, the student model will behave just like the teacher model albeit being smaller. This gives you the shortcut to getting a similar performance.
2. Quantization: I think about them as rounding the floating point numbers (e.g. 4.3256 -> 4.33). But essentially, you are reducing the size of the model by making each weight less precise.
3. Mixture of Experts (MoE): Regular AI LLM models are “dense”: meaning if i have a 235B parameter model, all 235B parameters are activated. MoE changes this paradigm: if you have a 235B parameter model but 30B active, then only 30B are active at a given time based on the user input. The model has a router that routes to each 30B parameter subsection. Thus, you are more memory-efficient since you have smaller activation.
4. More unified memory hardware: To run the frontier AI models with decent latency, you require a $500k setup with multiple latest Nvidia GPUs hooked up using the NVLink. Lately, I’ve been exploring unified memory hardware like DGX Spark, Mac Studio, or gmktec evo-x2 (starting at $4k). Pretty much, a computer has a different memory setup like RAM. For AI Models, what matters is memory on the GPU (VRAM) and memory bandwidth (how fast do GPUs talk to each other in a multi-GPU setup). And most GPUs have a very small VRAM. In a unified memory structure, your CPU and GPU use the same RAM, and these hardware often have >128GB memory, meaning you can comfortably fit open source models with 300B parameter models. If you are a casual user of AI and AI tools, this is more than enough, and they even support agentic capabilities like tool-calling.

In short, it’s possible to get a good enough AI model on hardware you can buy and have in your room. No data goes to AI labs or cloud providers. You can use them offline. You can finetune these AI models to your liking so it's YOUR custom model. And you are not faced with the outrageous AI bill: all you pay for is electricity! I think this unlocks a whole new avenue of revenue and these areas can’t be touched by the frontier AI labs because it’s not in their best interest to do so. If most companies and people go this route, they will use ChatGPT/Claude less.

Another interesting phenomenon is starting with ios27 in September, if your iphone has >12GB of RAM, you can use Siri/AI on your phone. I’m worried about sandboxing to develop anything in this domain, but if Siri and its local inference model is good, you have a personal AI assistant on your phone that can gather more context than the AI apps.