PortalsOS

LLMs trained on pre-1915 physics would never have discovered the theory of relativity. Discovering new science requires going beyond the existing training set, something current LLM architectures can't do.

The iterative nature of science requires LLMs to engage in simulations, theoretical calculations, and experiments to discover scientific insights.

The debate between LLMs and other reasoning models highlights the limitations of LLMs in understanding real-world context and predicting the future.

Without a goal, LLMs lack the ability to discern right from wrong actions, unlike reinforcement learning systems that have defined goals and rewards.

The future of AI development may involve LLMs advancing to a point where they can independently discover the next technological breakthroughs.

Current LLMs do not develop true world models; they build models of what a human would say next, relying on human-derived concepts.

LLMs develop deep representations of the world due to their training process, which incentivizes them to do so.

Continual learning is necessary for true AGI, and while it doesn't exist with current LLMs, there may be straightforward ways to implement it.

LLMs are trained on vast amounts of human data, which is an inelastic and hard-to-scale resource, making it an inefficient use of compute.

At the core of LLMs, regardless of their complexity or training methods, is the creation of a distribution for the next token. Given a prompt, LLMs predict and select the next word from this distribution, continuing the process iteratively.