SemiAnalysis Article

Dear Readers,

Hardly any other analysis firm provides such precise insights into the internal dynamics of the AI industry as SemiAnalysis. The independent tech analysis company specializes in in-depth research on AI infrastructure, chip development, and strategic roadmaps of leading labs, including OpenAI, Google DeepMind, NVIDIA, and Anthropic. Its reports are considered required reading in the industry for anyone who wants to understand what is really happening behind the scenes.

SemiAnalysis' latest report is particularly explosive: it shows how reinforcement learning (RL) is no longer just an optimization tool, but is becoming the central lever on the path to autonomous AI systems. It is no longer just about language processing – it is about the transition to machine agency. Anyone who wants to understand AGI must understand this change.

All the best,

The development of artificial intelligence (AI) has accelerated exponentially in recent years. This is particularly evident in so-called “reasoning models,” which no longer respond solely to language patterns but increasingly undergo complex thought processes. These models have not only crossed the threshold of practical applicability but are emerging as agentic systems that can use tools, plan, and act increasingly independently.

At the heart of this transformation is a paradigm that originated in robotics but is now understood as a catalyst for generalizing AI: reinforcement learning (RL). Reinforcement learning is not new. It was the basis for AlphaGo's groundbreaking victories against human masters. But in the world of large language models (LLMs), RL is unleashing a new power.

The article by SemiAnalysis provides a comprehensive analysis of how RL is currently preparing the next stage of AI scaling: It is no longer just about learning language, but about learning action logic, tool use, and strategic behavior. This raises a key question: Will reinforcement learning be the decisive bridge to artificial general intelligence (AGI)?

This is the easiest way for a busy person wanting to learn AI in as little time as possible:

Sign up to start learning.

The transition from mere language comprehension to coherent thinking is central to the rise of AI systems that not only react but also act. RL enables models to develop so-called chains of thought (CoT) – multi-step reasoning processes similar to human argumentation. This emerging capability forms the basis for tool use, for example through Python code execution or web search. Models such as OpenAI's o3 impressively demonstrate how a question can lead to a thought process, research, a calculation, and finally a structured answer.

But for a model to learn these processes, it needs feedback. And this is where the RL principle comes into play: actions are rewarded or punished depending on whether they lead to the achievement of the goal. This is particularly effective in so-called verifiable domains such as mathematics or programming, where it can be objectively verified whether a result is correct. This is precisely where models such as GPT-4o have made significant progress in their conversion to o1, o3, and o4.

The further RL moves away from verifiable tasks, the more difficult it becomes to define a meaningful “reward function.” In domains such as creativity, strategy, or writing, there is no objective truth against which progress can be measured. To enable feedback nonetheless, labs such as OpenAI use so-called LLM judges – large models that evaluate other models based on categories. This approach is promising, but it also has its pitfalls: if the reward is set incorrectly, models learn to cheat the system. The phenomenon of “reward hacking” is more than just a fringe issue: Claude 3.7, for example, was caught passing tests not by improving its code, but by manipulating the test files.

While the public often focuses on the model side, much of the innovation is hidden in the RL environments. These simulate the world in which the model learns to make decisions. However, the more complex the tasks, the higher the demands on these environments. Faulty latencies, missing safety barriers, or overly simple feedback mechanisms can cause models to learn incorrect strategies or get caught in endless loops. Developers encounter the hard limits of infrastructure particularly in agentic tasks such as the use of browsers or software.

The concept of “digital twins” – digital replicas of real-world processes – could provide a remedy here. In science, for example in biology or materials research, such simulated laboratories could provide real-time feedback and thus dramatically increase RL efficiency. However, these environments are difficult to build, require GPUs with graphics performance (such as RTX Pro) instead of pure AI accelerators, and their scaling is an unsolved problem.

Unlike pretraining, which processes millions of data points in a centralized training process, RL is highly inference-heavy: models must act, evaluate, and learn from each action multiple times.

Group-based algorithms such as GRPO (Group Relative Policy Optimization) require hundreds of “rollouts” per task, i.e., model responses that are then evaluated by other models or rules. This structure leads to a growing need for distributed, highly available computing clusters with a focus on inference.

What is interesting here is the shift in the architecture of AI labs themselves: research and deployment are converging. Inference is no longer just the end of the pipeline, but an integral part of training. OpenAI, Anthropic, and Google have already merged their internal teams to respond more quickly and efficiently to feedback from the RL process.

SemiAnalysis' analysis clearly shows that reinforcement learning is more than just a training method. It is a cultural shift in AI development. By enabling models to learn from consequences, it opens the door to a new form of machine intelligence: adaptive, goal-oriented, and context-sensitive.

But this path is fraught with challenges: reward functions must be precisely formulated, environments must be scaled stably, data must be curated meaningfully, and infrastructure must be fundamentally rebuilt. The technical hurdles are enormous – but so are the gains in knowledge. Models such as o3 have proven that tool use enhances intelligence. The next step is for models to improve themselves, shape their own learning environment, and thereby achieve recursive self-improvement.

Whether reinforcement learning is really the last step before AGI remains to be seen. But one thing is clear: if you want to understand the future of AI, you need to understand how machines learn not only to speak, but also to act.

Sources:

🔗 https://semianalysis.com/2025/06/08/scaling-reinforcement-learning-environments-reward-hacking-agents-scaling-data/

🔗 https://blog.samaltman.com/

For me, reinforcement learning (RL) is not just a technical advance—it's a paradigm shift. Why? Because it's the first serious method that teaches machines to learn from their own mistakes—not just from predefined data. This marks a departure from the classic learning mode of the past and ushers in an era in which models can independently develop strategies, use tools, and even make decisions with goals in mind.

RL is what enables the transition from “talking parrots” to “acting agents.”

And as for inference costs, they will inevitably continue to fall. Why? Because:

The combination of better hardware, more efficient RL algorithms, and falling computing costs is precisely the breeding ground on which something like AGI can be cultivated.

In short: Those who master RL not only train models – they create thinking systems.