<h1>AI 2027 - 2029</h1>
<p>Recently, a major attempt was made to forecast what short-term ASI<sup>1</sup> <span class="side-citation-wrapper" data-cite="1. ASI: Artificial Superintelligence, i.e. AIs which are wildly more intelligent than the smartest humans"></span> timelines might look like. It was published to <a href="http://ai-2027.com">ai-2027.com</a>, an interactive website providing a hypothetical scenario that represents some of the authors&#39; modal expectation<sup>2</sup> <span class="side-citation-wrapper" data-cite="2. By modal expectation, I mean the specific timeline they consider most plausible, though not necessarily more likely than not"></span>.</p>
<p>Its predictions may be incredibly jarring or even downright silly to most people, but it also may come as a surprise that they are not far from the ordinary expectations of many people in the AI and forecasting circles.</p>
<ul>
<li>Dario Amodei, the founder and CEO of Anthropic recently said “we could have AI systems equivalent to a ‘country of geniuses in a datacenter’ as soon as 2026 or 2027”  in his essay <a href="https://www.darioamodei.com/post/the-urgency-of-interpretability#:~:text=we%20could%20have%20AI%20systems%20equivalent%20to%20a%20%E2%80%9Ccountry%20of%20geniuses%20in%20a%20datacenter%E2%80%9D%20as%20soon%20as%202026%20or%202027.">The Urgency of Interpretability</a>.  </li>
<li>Sam Altman, the founder and CEO of OpenAI, said “it is possible that we will have superintelligence in a few thousand days” in his essay <a href="https://ia.samaltman.com/">The Intelligence Age</a>.  </li>
<li>In 1999, in his book <em>The Age of Spiritual Machines</em>, Ray Kurzweil predicted artificial general intelligence (AGI) in 2029, and beyond-radical changes to society in the years soon after, and he has stuck by this prediction throughout the last 26 years.  </li>
<li>Manifold, a non-monetary betting market that’s popular in these circles of people, is currently at a 33% <a href="https://manifold.markets/IsaacKing/will-there-be-a-superintelligence-b-10fdd93e8092">chance of superintelligence by 2030</a>.  </li>
<li>Metaculus, a well-known forecasting platform where experts and amateurs make long-term predictions, has <a href="https://www.metaculus.com/questions/5121/date-of-artificial-general-intelligence/">a median expectation of AGI</a> that recently fluctuated between 2030 and 2034.</li>
</ul>
<p>Now, assuming you are sufficiently convinced that the authors of <em>AI 2027</em> are not fully alone in their short timelines, let’s dive into their scenario and challenge some of their assumptions. They provide a frankly staggering level of research (relative to what’s possible to research for a topic like this) and reasoning to inform as much of their expectations about the near-future of AI progress as possible.</p>
<h2>Summary</h2>
<p><em>AI 2027</em> predicts slow (relative to what follows) progress in AI capabilities in 2025 and 2026. It is roughly in the same order of magnitude as the capability growth we’ve seen in 2023-2024. The main improvements discovered are semi-reliable AI agentic systems in 2025, and increasingly reliable agents in 2026, especially in the domain of coding and AI research. These can perform tasks such as browsing the internet for detailed research, creating somewhat complex features in code repositories, and operating a computer entirely on a user’s behalf (each with not-uncommon mistakes). The authors also predict that the governments of the USA and China slowly begin to wake up to the impending dramatic changes on the horizon, and the importance of them from a national security perspective.</p>
<p>In contrast to the predicted “slowness” of 2025-2026, as you might expect based on the title of the essay, 2027 is when they predict AI progress will start the “takeoff” scenario. Over the past several decades of discussion around AI capability progress, “intelligence takeoff” was coined to describe a scenario in which increasingly capable AIs are able to contribute to their predecessors’ capabilities, causing a positive feedback loop which accelerates much faster than one might intuitively expect.</p>
<p>By the end of 2027, in their scenario, one single instance of the top AI model is significantly more intelligent than any human that has ever existed, and there are hundreds of thousands of copies of it running in parallel.</p>
<p>Beyond the scope of this summary, it’s worth quickly clarifying what this might mean, for readers not familiar with the topic of superintelligent AI. Imagine 300,000 much-smarter-than-Einstein people working and thinking (100x faster than humans think) to solve disease, aging, scarcity, and any other human issue (or working to kill us all, but that’s a topic for another blog post). Many people predict that this scenario would cause unfathomable changes, perhaps the same level of technological change between hunter-gatherers and today, happening to us in a decade.</p>
<p>However, as with any long-term prediction, the assumptions and other inputs to your formula can drastically modify the end results (in this case, the timelines).</p>
<h2>Analyzing Assumptions</h2>
<p>The authors estimate that having a superhuman AI coder (but not yet a superhuman AI researcher, due to relatively worse “research taste”), would result in a 5x speedup compared to if only humans were involved. They arrive at this via stacked multipliers from various sources of improved efficiency in the research process due to superhuman AI coder. One of these multipliers (copied here from their <a href="https://ai-2027.com/research/takeoff-forecast#from-sc-to-sar:~:text=with%20the%20above%3F-,Less%20waste,-%3A%20With%20SCs">Takeoff Forecast research</a>) is:</p>
<p>Less waste: With SCs [superhuman coders], research teams can use the vast engineering labor to waste less compute.</p>
<ul>
<li>Before every experiment including small-scale experiments, SCs can red-team and bug-test it to reduce the probability that there is some bug which ruins the results.  </li>
<li>Monitor the experiments in real time, noticing and fixing some kinds of problems almost as soon as they happen (by contrast it’s common in frontier companies today for even medium-sized experiments to be running overnight with no one watching them). They can shut down the experiment early as soon as it’s clear that it worked or didn’t work  </li>
<li>Do the same experiments but with less compute e.g. via higher utilization</li>
</ul>
<p>They guess that this results in a multiplier between 1.2x to 2x. I want to specifically highlight their second bullet point here, the idea of AIs monitoring, fixing, and shutting down the experiments early. I find this unconvincing for two main reasons. Firstly, I find it difficult to believe that the humans in charge in 2027 will be willing to provide this much power to AIs. Secondly, I doubt that a superhuman AI coder (which, by their definition, is not yet a superhuman AI researcher) would perform well enough at this job for humans to even want to <em>autonomously</em> employ them for it. However, the other two specific speedups listed make sense to me. As a result, I reduce their multiplier for less waste to be 1.1x to 1.5x, and we arrive at a hypothetical total progress speedup of 4x, rather than their 5x, for the superhuman AI coder.</p>
<p>Another instance of the authors, in my opinion, overestimating the trust that the AI lab will place in their AI agents is their assumption that the improvements in the agent’s capabilities will immediately be applied to the AI research. In their takeoff simulation code, they use <a href="https://github.com/uvafan/timelines-takeoff-ai-2027/blob/eadc43f1dded3e7b3ef597d13af7eea56215e7e2/takeoff/forecasting_takeoff.py#L155">timesteps of one single calendar day</a> during the speedup calculation, which means that the improvements to the model on Tuesday will begin manifesting itself in a speedup on AI research on Wednesday. I am doubtful that the frontier AI labs, who regularly discuss fear of catastrophic misalignment, and whose internal staff often have very high values of “p(doom)”<sup>3</sup> <span class="side-citation-wrapper" data-cite="3. p(doom) is a common term utilized in this topic to refer to the probability of extinction-level catastrophic events caused by powerful AI, often mentioned in the context of what a person believes p(doom) to be."></span>, will be willing to allow an AI that was trained yesterday (literally) to influence AI research today. I believe a time step of 30 days is more appropriate, which gives AI labs time to internally red team the new version of the model.</p>
<h2>Compute Production and Capacity</h2>
<p>The authors estimate 4 million H100-equivalent<sup>4</sup> <span class="side-citation-wrapper" data-cite="4. An H100 is a modern, high-performance data center GPU, i.e. an AI computer"></span> computing power (they refer to these as &quot;H100e&quot;) existed globally at the end of 2023, but it seems like Epoch AI (their own source) estimated that number to be true at mid-2024. That pushes the authors’ total compute timeline back by 6 months. The authors expect compute production to grow 2.25x per year<sup>5</sup><span class="side-citation-wrapper" data-cite="5. This is their own estimate, but also in line with Epoch AI’s estimate"></span>. This amounts to ~7% growth per month.</p>
<p>However, it may be a mistake to assume 2.25x growth in compute capacity/production every year. <a href="https://chatgpt.com/share/68003bab-7eec-8010-bcc5-cf8693541b1f">ChatGPT o3 DeepResearch predicts a 3-5x increase</a> in compute capacity over the next 2 years, and <a href="https://chatgpt.com/c/68003111-6a20-8010-93e4-c4d70f9f2249">o3 normal predicts a 4x increase</a>, in line with each other. This amounts to about a 6% MoM increase. Thus, assuming 6 million H100e in EOY 2024, and a 4x increase from then until EOY 2026, we arrive at 24 million H100e.</p>
<p>Assuming it slows down a bit further, perhaps to 5% per month in 2027<sup>6</sup> <span class="side-citation-wrapper" data-cite="6. This is still “break-neck” speed: a yearly growth rate equal to [the growth of the entire US economy](https://fred.stlouisfed.org/series/GDPC1) over the last 27 years"></span>, we can expect 32 million H100e by mid 2027, instead of the 60 million that the authors predict.</p>
<p>At first, we might predict that this correction should delay the predicted advent of a superhuman coder AI, since, as we’ll see in the next section, training compute is one of the four major variables in AI progress speed. However, the authors’ actual <a href="https://ai-2027.com/research/timelines-forecast#:~:text=We%20first%20show,best%20AGI%20company.">two methods of forecasting the timing of superhuman coders</a> do not involve their predicted amount of available compute.</p>
<h2>Automated AI Researchers</h2>
<p><a href="https://youtu.be/htOvH12T7mU?si=PhZxTG7HeeBe3q9E&t=2044">In the Dwarkesh Podcast on April 3rd, he asks the authors,</a> if the mental labor toward improving the software of AI models is so important, why aren’t the major labs increasing headcount as quickly as possible? Daniel Kokotajlo, the first author, responds by saying that they see four inputs into AI progress speed: number of researchers, researcher serial speed (relatively constant for humans), research taste (i.e. the quality of selected experiments), and training compute. He says that, by the time they’ve fully automated AI research, the main bottlenecks will be training compute and research taste. In other words, the reason for the 25x multiplier, rather than something higher, is lack of training compute and research taste. If that’s the case, then cutting our available GPUs almost in half (from their expectation), would mean that training compute would be even more of a bottleneck, and having those extra hundreds of thousands of AI researchers working at 40x serial speed does not have as much gain as it otherwise would.</p>
<p>This matches up with a recent quick exchange I had with Daniel:</p>
<p>Me: I was curious as to how you all came up with the &quot;6% of compute will go to research automation?&quot;<br>Daniel: It was just an educated guess! I&#39;m lately thinking it should have been a smaller fraction probably, maybe more like 1%, to free up more compute for experiments.</p>
<p>They did not go into detail on answering Dwarkesh’s specific question (likely because he paired it immediately with another question/statement) about AI labs not hiring more humans today. However, I think it’s a very important string to pull on here, so let me humbly try to answer for them.</p>
<p>One way to look at it is “If OpenBrain wanted to hire an additional *really good* AI researcher today, the marginal cost is a yearly total compensation in the millions of dollars. That researcher might begin contributing strongly within a few weeks or months. Alternatively, hiring non-AI researchers, but very smart programmers and math/physics experts, would perhaps cost slightly less, but they might not begin contributing strongly for, say, 6-12 months. As most things in life, we need to look at things in terms of tradeoffs and opportunity cost. By allocating $3 million to pay Researcher #999 over the 2 years, OpenBrain could purchase 23 GB200 computers and get 6.53 x 1024 total INT8 FLOPs over that same timeframe (assuming a Model FLOP Utilization of 40%). Thus, rather than training ~5 more non-AI-researchers to become AI researchers, they would have enough <a href="https://epoch.ai/data-insights/models-over-1e25-flop">FLOPs to recreate GPT-4o</a> (slightly rounding down due to likely architecture improvements since the original GPT-4o) and more, since GPUs last longer than just 2 years of usefulness.</p>
<p>However, the marginal cost of adding a researcher goes down if the researcher is an AI. Using the authors’ unmodified numbers, if <a href="https://ai-2027.com/research/takeoff-forecast#:~:text=Our%20compute%20forecast,the%20best%20humans.">the leading AI lab uses 500,000 H100e to run 50,000 superintelligent AI researcher agents</a>, then the hardware to run one AI researcher (which accomplishes tasks at 30x human speed) costs $150,000 upfront (assuming <a href="https://ai-2027.com/research/compute-forecast#:~:text=%2420k-,%2415k,-Total%20AI%20datacenter">each H100e costs $15,000</a>), and they can continue running it for years. This is about an order of magnitude less expensive than paying a top-tier human AI researcher for just one single year.</p>
<h2>Running an Updated Takeoff Simulation</h2>
<p>Taking our updated parameters from Analyzing Assumptions (4x speedup for superhuman coder, and 30-day timesteps), and an updated value of 10x for the superhuman AI researcher speedup (and 100x for superintelligent AI researcher and 800x for general superintelligence, using the same methodology as <em>AI 2027</em>) due to the reasoning in the Automated AI Researchers section, the authors’ <a href="https://github.com/uvafan/timelines-takeoff-ai-2027/tree/main">takeoff simulation</a> gives us updated estimates.  </p>
<div class="flex flex-col items-center my-6">
  <img src="/images/takeoff_timeline.png" alt="Takeoff Timeline" class="rounded-lg shadow-md" style="width: 75%; height: auto;">
  <img src="/images/phase_durations.png" alt="Phase Durations" class="rounded-lg shadow-md mt-4" style="width: 75%; height: auto;">
</div>

<p>To highlight some differences we see compared to their estimate: the median expectation for the time between:</p>
<ol>
<li><strong>The entire takeoff scenario (Superhuman coder -&gt; General Superintelligence): 23 months, or almost 2 years (instead of 9.5 months in <em>AI 2027</em>)</strong>  </li>
<li>Superhuman Coder -&gt; Superhuman AI Researcher: 7 months (instead of 4 months)  </li>
<li>Superhuman AI Researcher -&gt; Superintelligent AI Researcher: 10 months (instead of 3.5 months)  </li>
<li>Superintelligent AI researcher -&gt; General Superintelligence: 6 months (instead of 2 months)</li>
</ol>
<p>Although this might not seem like much of a difference (2 years is still arguably unacceptably fast to go from humans being the smartest entities on Earth to computers leaving us in the dust [or <em>as</em> dust, if they are misaligned]), the updated numbers, if true, would give us over twice as long to: set up systems of human oversight, use the intermediate AIs to solve mechanistic interpretability or alignment, and/or have changes in major word leaders (I swear something important happens in January 2029, but I can’t quite place it).</p>
<p>It’s worth noting that this post is <em>not</em> an indictment of the authors of <em>AI 2027</em>. They clearly put much, much more thought, effort, and skill into their work than I’ve done here, and, most importantly, these takeoff timelines are well within their proposed confidence intervals. This post is exactly the opposite of an indictment, in fact, but more of my attempt at a minor contribution to their work because I find it so inspiring.</p>


AI 2027-2029

<h1>The Future of Software Development</h1>
<p>Most experienced software engineers will recall constantly needing to stay up-to-date with changes in software technologies in order to partake in most of the industry. Today&#39;s landscape is not an exception, but it does come with some notable differences.</p>
<p>The new players in the field (as it relates to this blog post) are not new programming languages, libraries, nor philosophies of code quality. Instead, they are large language models and their ability to code, test, make architectural decisions, and (soon) automatically perform deployment and other necessary actions.</p>
<h2>Coding and Testing</h2>
<p>In any given timeframe, software engineers are now able to create vastly more value, as some of the most time-consuming parts of coding are now essentially automated. Tools like GitHub Copilot, Cursor, and more are able to quickly implement:</p>
<ul>
<li>Boilerplate solutions</li>
<li>Code Refactors</li>
<li>Translation to alternative programming languages</li>
<li>Debugging</li>
<li>Unit test creation</li>
<li>Increasingly, entire features across multiple files</li>
</ul>
<p>This allows mindful software engineers to move quickly, focusing their efforts on:</p>
<ol>
<li>Directing the AI</li>
<li>Making high-level and architectural decisions (in order to do (1))</li>
<li>Carefully reviewing AI output</li>
</ol>
<p>At Vectorial, we take this very, very seriously. Each of our engineers is highly encouraged to take advantage of models and tools such as <a href="https://openai.com/index/openai-o1-system-card/">OpenAI&#39;s o1</a>, <a href="https://www.anthropic.com/news/claude-3-5-sonnet">Anthropic&#39;s Claude 3.5 Sonnet</a>, <a href="https://www.cursor.com/">Cursor</a>, and/or <a href="https://github.com/features/copilot">GitHub Copilot</a>. However, we recognize that these tools often make mistakes, so we also emphasize the engineers&#39; responsibility to review code heavily, and require all delivered code to be peer-reviewed by another human engineer.</p>
<p>The balance we strike here allows us to deliver software and AI solutions at a higher quality <em>and</em> a greater speed than ever previously possible.</p>
<h2>Architectural Decisions</h2>
<p>State-of-the-art models are increasingly becoming useful at more high-level tasks, such as designing entire technical architectures. Models such as o1 and Claude 3.5 Sonnet can often create exceptional first drafts of an architecture for a new product or feature.</p>
<p>These architectures are still prone to mistakes, which have the potential to be more dangerous than code bugs, as they are less detectable and fixable. Following them blindly can often be a one-way door that you can&#39;t easily reverse down the line.</p>
<p>That being said, having a human engineer treat these drafts as mere (possibly faulty) suggestions is often higher-quality and discovered faster than the architectures that a junior or mid-level engineer would come up with on their own.</p>
<h2>Automated Software Engineering</h2>
<p>As frontier AI labs continue to improve these LLMs, we continue to progress toward a future in which these AI models are able to perform every aspect of a modern software engineer&#39;s job. Currently, it does not seem clear to us at Vectorial whether or not software engineers&#39; skills will continue to be useful by &quot;moving up the abstraction ladder&quot; if non-technical people can direct the AIs just as effectively.</p>
<p>Products or technologies like <a href="https://devin.ai/">Devin</a>, <a href="https://github.com/All-Hands-AI/OpenHands/">OpenHands</a>, <a href="https://aider.chat/">Aider</a>, <a href="https://v0.dev/">v0</a>, and <a href="https://www.cursor.com/features">Cursor Agent</a> are all early versions of this. You simply explain in natural words what you want to build and the models continuously run, working on the code, and eventually come back with either questions or results (or failures, hence the &quot;early&quot; label!).</p>
<h2>An Example</h2>
<p>Take a quick look over our website. Look at the particle effects, the landing page, the blog post you&#39;re currently reading (not the words, but the code). This is all nearly 100% created by Claude 3.5 Sonnet (using Cursor) and o1. The particle effects are a custom implementation, fully developed and iterated on by o1. We estimate that building the website to its current state would have taken at least 3-5x as many developer hours without the use of AI. A completely non-technical person could have likely built and deployed this website in just 2-3 days.</p>
<video autoplay loop muted playsinline class="w-full h-auto rounded-lg my-4" id="particle-video">
  <source src="/images/vectorial-site-particles.mp4" type="video/mp4">
  Your browser does not support the video tag.
</video>

<script>
  document.addEventListener('DOMContentLoaded', () => {
    const video = document.getElementById('particle-video');
    const observer = new IntersectionObserver((entries) => {
      entries.forEach(entry => {
        if (entry.isIntersecting) {
          video.play();
        } else {
          video.pause();
        }
      });
    }, { threshold: 0.6 }); // Start playing when at least 60% of the video is visible
    
    observer.observe(video);
  });
</script>

<h2>How to Prepare</h2>
<p>Today, it is a good idea for software engineers to get acquainted with generative AI coding tools and agents. It is common to have a few bad initial experiences with generative AI, and avoid using it entirely, to one&#39;s own detriment. Instead, one can refine their abilities of using the tool (similar to the older concept of &quot;<a href="https://en.wiktionary.org/wiki/Google-fu">Google-fu</a>&quot;) and can remember to push the limits of the newest models as they are released, as previously failed uses may now be within the new models&#39; capabilites. It is also important, for the time being, to maintain existing programming skills, to avoid helplessness in times without access (such as in jobs/interviews where generative AI usage is forbidden).</p>
<p>In the medium-term, software engineers should prepare for the likely case of rapid change followed by near-full automation. This involves the job requirements changing rapidly every year or even every few months. Soon after, it may become the case in <a href="https://www.vectorial.us/blog/short-term-ai-progress">the late 2020s</a> that non-technical people can use these AIs just as effectively as software engineers. In this scenario, it&#39;s best to save as much of your income as possible today, as it may drop significantly (or to zero).</p>
<p>In this same scenario, software businesses should also prepare. &quot;Tech debt&quot; that can stay &quot;unpaid&quot; for a couple years may be worth making, since the cost of fixing it will continue to drop. It may make sense to stop hiring junior software engineers, and eventually mid-level engineers.</p>
<p>And businesses in general can prepare for significant technological disruption. For example, modern CRMs can, at risk of heavily oversimplifying, be described as a large user interface and user experience (UI/UX) operating on a database. Will that paradigm remain the same if a small team is able to simply ask OpenAI&#39;s future o4 or o5 &quot;generate and deploy code that can listen in on users&#39; calls, emails, and messages, and store data related to deals, meetings, etc. for them to ask about later&quot;?</p>
<h2>Key Takeaways for Software Engineers:</h2>
<ul>
<li>Adopt AI tools to stay competitive, but maintain rigorous review processes.</li>
<li>Keep your skills sharp — both technical (for deeper debugging/coding/architecture) and in AI usage.</li>
<li>Prepare for rapid changes in the job market: be financially cautious and professionally adaptable.</li>
</ul>


The Future of Software Development

<h2>AI Progress So Far</h2>
<p>The past few years have seen unprecedented advances in artificial intelligence, particularly through the usage of transformers in autoregressive and diffusion architectures. Since the release of GPT-2 in 2019, we&#39;ve witnessed a cascade of increasingly capable AI models from multiple organizations. These models have progressed from basic text completion to complex reasoning, coding, and even multimodal capabilities combining text, images, video, and audio.</p>
<p>Key milestones include:</p>
<ul>
<li>2019:<ul>
<li>Demonstrated the success of self-supervised training for decoder-only transformers (<a href="https://openai.com/index/better-language-models/">GPT-2</a>)</li>
</ul>
</li>
<li>2020:<ul>
<li>Set new standards for language understanding/generation; demonstrated in-context learning (<a href="https://openai.com/index/language-models-are-few-shot-learners/">GPT-3</a>)</li>
<li>Demonstrated breakthrough protein structure prediction (<a href="https://deepmind.google/discover/blog/alphafold-a-solution-to-a-50-year-old-grand-challenge-in-biology/">AlphaFold 2</a>)</li>
<li>Public launch of driverless taxis in geo-fenced areas (<a href="https://waymo.com/blog/2020/10/waymo-is-opening-its-fully-driverless-service-in-phoenix/">Waymo</a>)</li>
</ul>
</li>
<li>2021:<ul>
<li>First major product release of large language models (<a href="https://github.blog/news-insights/product-news/introducing-github-copilot-ai-pair-programmer/">GitHub Copilot</a> and OpenAI&#39;s <a href="https://openai.com/index/evaluating-large-language-models-trained-on-code/">Codex</a>)</li>
</ul>
</li>
<li>2022:<ul>
<li>Impressive multimodal understanding by processing text and image data jointly (<a href="https://deepmind.google/discover/blog/tackling-multiple-tasks-with-a-single-visual-language-model/">Flamingo</a>)</li>
<li>Compute-optimal scaling laws for transformer large language models (<a href="https://deepmind.google/discover/blog/an-empirical-analysis-of-compute-optimal-large-language-model-training/">Chinchilla</a>)</li>
<li>Significant advancement in text-to-image generation (<a href="https://openai.com/index/dall-e-2/">DALLE-2</a>, <a href="https://imagen.research.google/">Imagen</a>)</li>
<li>Brought conversational AI to the mainstream (<a href="https://openai.com/index/chatgpt/">ChatGPT and GPT 3.5</a>)</li>
</ul>
</li>
<li>2023:<ul>
<li>Massive improvements in reasoning and multimodal capabilities (<a href="https://openai.com/index/gpt-4-research/">GPT-4</a>)</li>
</ul>
</li>
<li>2024:<ul>
<li>Expanded multimodal capabilities, combining text, image, video, and audio input (and output for text and image) into the same models (<a href="https://openai.com/index/hello-gpt-4o/">GPT-4o</a>)</li>
<li>Heavy improvements in reasoning and overall model quality (<a href="https://www.anthropic.com/news/claude-3-5-sonnet">Claude 3.5 Sonnet</a>, <a href="https://openai.com/index/learning-to-reason-with-llms/">o1</a>)</li>
<li>Impressive video generation models (Sora, Veo 2, Runway, Kling)</li>
<li>Impressive music generation models (Suno, Udio)</li>
<li>Massive improvements in structure prediction of biomolecular interactions (<a href="https://blog.google/technology/ai/google-deepmind-isomorphic-alphafold-3-ai-model/">AlphaFold 3</a>)</li>
<li>Significant improvements for driverless cars <em>without</em> geo-fencing (Tesla FSD v12 and v13)</li>
</ul>
</li>
</ul>
<h2>Have LLMs hit a wall?</h2>
<p>At first glance, after going from GPT-2 -&gt; GPT-3 -&gt; GPT-4, it may feel like progress has stalled. However, we think this is the common pitfall associated with desensitization or <a href="https://psychcentral.com/health/hedonic-adaption">hedonic adaptation</a>. We believe there are three major sources of this increased desensitization:</p>
<ol>
<li>People who were previously paying attention are now paying attention <em>much more</em> now, so AI progress feels slower. Recall that it took almost 3 years from the GPT-3 announcement until the GPT-4 announcement. As of this writing, it has not even been 2 years since the GPT-4 announcement.</li>
<li>Model releases are much, much more frequent. This stems from more frontier AI labs (OpenAI, Anthropic, Google Deepmind, Meta), more secondary AI labs (Deepseek, Qwen, Mistral), and a shorter release cycle from each. Thus, there were much fewer models between GPT-3 and GPT-4 than between GPT-4 and GPT-4o</li>
<li>AI models are increasingly saturating both formalized benchmarks and unformalized ones (i.e. &quot;vibes&quot;). As models improve, people resort to harder and harder questions, and thus the rate of &quot;failure&quot; in real-life usage stays relatively constant from 2 years ago.</li>
</ol>
<p>The result of these sources is that AI progress can feel slower than it was 2 years ago, or 4 years ago. However, in reality, the models we have access to today are significantly better than models from 1-2 years ago, such as GPT-4 and GPT-4 Turbo. This is evident in essentially every single benchmark, as well as in anecdotal experience from our team.</p>
<h2>Will LLMs hit a wall?</h2>
<p>Assuming LLMs have not yet hit a wall: will they do so in the near future? Common arguments for &quot;yes&quot; include: running out of data, fundamental limits of autoregressive transformers, and diminishing returns of more/better data and more compute. Let&#39;s visit each point directly.</p>
<ol>
<li><p><strong>Running out of Data</strong></p>
<ul>
<li>Llama 3, the best knowledge we have of modern state-of-the-art LLM training, used 15 trillion tokens.<ul>
<li>Most of this data is from mass internet web crawls.</li>
<li>Some estimates put <a href="https://en.wikipedia.org/wiki/Books_published_per_country_per_year">the number of books published each year</a> at ~2.2 million. Assuming the average number of words per book is 80,000, that&#39;s ~235B tokens (~175B words) published every year. That amounts to 14 trillion high-quality tokens since 1960, matching the amount Llama 3 was trained on.</li>
<li>Google&#39;s YouTube has about <strong>1 million hours of footage uploaded every day</strong>. At 30 frames per second, and <a href="https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/video-understanding">258 tokens per frame</a>, that&#39;s about 27 trillion tokens of video added every single day. That&#39;s about <strong>one quadrillion</strong> tokens for every year of YouTube. This even ignores audio tokens from the videos as well! The tens of trillions of tokens used to train today&#39;s state-of-the-art models pale in comparison.<ul>
<li>Keep in mind, this is just one product. There are similarly crazy numbers of human-generated tokens on other social media services, such as Facebook, Instagram, LinkedIn, TikTok, Twitter, Reddit etc.</li>
</ul>
</li>
</ul>
</li>
<li>Frontier AI labs are finding surprising success with synthetic data approaches.<ul>
<li>For example, generating code or math using LLMs, and, using automated deterministic tests or even LLM judges, choosing high quality responses and performing Reinforcement Learning, fine-tuning, or even pretraining on them.<ul>
<li><a href="https://x.com/polynoamial/status/1834280425457426689">OpenAI has announced</a> that they used this methodology to create o1</li>
<li>Many other major and small labs have claimed they&#39;re seeing success with synthetic data.</li>
</ul>
</li>
</ul>
</li>
<li>In any case, if we really do run out of data, AI scientists have shown that scaling up model size results in better sample efficiency. We are currently in an overtraining paradigm (due to outsized inference costs compared to training costs), but if we transition to being constrained by data, there is a lot of room for improvement by entering an undertrained paradigm (by massively scaling model sizes).</li>
</ul>
</li>
<li><p><strong>Fundamental Limits of Autoregressive Transformers</strong></p>
<ul>
<li><a href="https://arxiv.org/pdf/2411.01992">Recent research</a> has mathematically proven that autoregressive, decoder-only transformers are <a href="https://en.wikipedia.org/wiki/Turing_completeness">Turing-complete</a> if given the correct input</li>
<li>Time and time again, skeptics of the modern generative AI paradigm have predicted that these models would fail at specific tasks, and time and time again they have quickly improved enough to succeed at those tasks.</li>
</ul>
</li>
<li><p><strong>Diminishing Returns of Increasing Data Size, Data Quality, and Compute</strong></p>
<ul>
<li>Of the three, this argument has the best potential to be right. We can never be sure that a certain trend will continue, and this is no different. However, we can take hints from the frontier labs, who have at least a few months headstart on seeing these models in action.<ul>
<li>Noam Brown (OpenAI multi-agent researcher): &quot;<em>We announced @OpenAI o1 just 3 months ago. Today, we announced o3. We have every reason to believe this trajectory will continue.</em>&quot; (<a href="https://x.com/polynoamial/status/1870172996650053653">link</a>)</li>
<li>Dario Amodei (Founder of Anthropic) loosely predicts 2026 or 2027 for &quot;very powerful AI&quot; (<a href="https://x.com/tsarnick/status/1856065476759302167">link</a>)</li>
<li>Sam Altman (Founder of OpenAI): &quot;<em>We are now confident we know how to build AGI as we have traditionally understood it. We believe that, in 2025, we may see the first AI agents “join the workforce” and materially change the output of companies.</em>&quot; (<a href="https://blog.samaltman.com/reflections">link</a>)</li>
</ul>
</li>
</ul>
</li>
</ol>
<h2>What might 2030 look like?</h2>
<p>There are a few major trends to pay attention to:</p>
<ol>
<li>More data and more high-quality data are being used to <a href="https://www.clrn.org/what-is-pretraining-and-post-training/">pretrain and post-train</a> these models.</li>
<li>Progressively increasing compute.<ul>
<li>Massively parallelized computer chips are continuing to improve in speed, memory, and other metrics.</li>
<li>Larger and larger clusters of these chips are being built constantly. <a href="https://blogs.oracle.com/cloud-infrastructure/post/now-ga-largest-ai-supercomputer-oci-nvidia-h200">OCI Supercluster</a> from Oracle, <a href="https://www.tomshardware.com/desktops/servers/first-in-depth-look-at-elon-musks-100-000-gpu-ai-cluster-xai-colossus-reveals-its-secrets">Colossus</a> from xAI, and rumors of <a href="https://www.tomsguide.com/ai/meet-stargate-the-dollar100-billion-ai-supercomputer-being-built-by-microsoft-and-openai">Stargate</a> from OpenAI and Microsoft</li>
</ul>
</li>
<li>Increasing focus and efforts on post-training, using methods such as RLHF, RLAIF, supervised fine-tuning, and other reinforcement learning and synthetic data methods based on provably validity of response.</li>
<li>Improving on the vanilla transformer architecture. Papers like the <a href="https://arxiv.org/abs/2412.09871">Byte-Latent Transformer</a>, <a href="https://sakana.ai/transformer-squared/">Transformer^2</a>, <a href="https://arxiv.org/abs/2412.06769">Reasoning in Continuous Latent Space</a>, and many others are recent, seemingly successful, attempts at this.</li>
<li>Distributed training. As cluster sizes hit physical and infrastructure limits, research is being done on the potential for distributed training of models. <a href="https://www.primeintellect.ai/blog/intellect-1">Prime Intellect&#39;s DiLoCo</a> and <a href="https://github.com/NousResearch/DisTrO">Nous Research&#39;s DisTrO</a> are examples of this.</li>
</ol>
<p>If we extrapolate these trends, 2029 and 2030 may look like enormous (even relative to today), distributed training runs for heavily multimodal (text, image, video, and audio input <em>and</em> output) models. These models may be able to autonomously perform long-horizon tasks, such as: building complex web applications, creating detailed visual media, managing a team of and writing detailed research on computer science, artificial intelligence, mathematics, physics, and more (chemistry and biology may require more human intervention due to laboratory tasks).</p>
<p>This would be the beginning of massive job automation, where most white-collar jobs would be able to be fully replaced by AI. Depending on the manufacturing speed of general-purpose (e.g. humanoid) robots, we may start to see significant blue-collar job replacement as well.</p>
<p>It would also be the beginning of a lack of scarcity (relative to today&#39;s society). After controlling for potential governmental increases of monetary supply, prices for nearly every good or service should drop significantly, as the cost of producing them drops, and as overall efficiency increases.</p>


Blog

AI 2027-2029

The Future of Software Development

AI Progress in the Second Half of the 2020s