Rendered at 11:09:28 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
anthonypasq 16 hours ago [-]
Love to see these crazy kinds of experiments going on. Even if this doesn't 100% work or is prohibitively expensive for now, these are glimpses into the future in the same way people were talking about coding agents in 2023 when we just had tab complete.
gritzko 2 hours ago [-]
I am afraid the folks are still busy playing with familiar toys. "Let's extract more code out of LLM's inner layers!" SQLite we already have. That's where their "spec" came from.
The interesting frontier is elsewhere: how to fit it into the pipeline to build new useful things.
I felt motivated enough to write a follow-up/reply post http://replicated.live/blog/follow-up
sawyers 5 hours ago [-]
It shouldn't be prohibitively expensive because the more you break the work into smaller parts, theoretically the smaller a model can be that completes it.
Therefore your agent swarms don't have to be cutting edge API calls, they can be hosted on clusters of tiny cheap computers with small CPU GPU. For orgs you could basically invest in a local server that has 100s of small computers.
You can use engineers and frontier models to plan and do task subdivision. And then shit that out to the local cluster that's basically handling each small task without knowing what it's supposed to do.
The real issue is the middle. Testing, verifying interconnectivity of mid level abstractions, building that amount of tech debt at such a high rate and actually being aware in any way what it is that you've built.
I think for most MBAs, considering the US and a significant amount of global markets only care about short term market upside, and AI is still in the middle of the hype cycle, that's not a serious issue for anyone who is profit oriented.
It's only a real issue for the losers like you and me who care about sustainability and infrastructure that survives longer than one market hurricane season.
cookiengineer 5 minutes ago [-]
This so much!
For my current setup, the most efficient way is to use larger models for coordination, but a heretic'ed qwen 30B model for the implementations. If you build your agentic environment around specifications and test coverage tied to symbols, you can do a lot of parallelization of agent work.
If you then separate the filesystem and tool read/write access by agent roles and policies (e.g. coder not allowed to modify unit tests, tester not allowed to modify code files) then you can force them to use a centralized per symbol/per contract specification tool.
And then you can just let agents discuss issues, where the messaging threads are also tied to the same symbols.
The twin trends of AI slop being churned out at tremendous rate and deployed in everything, plus frontier AIs becoming really good at exploiting many small security holes to pwn boxen, is going to collide in an amazing cyberstorm that was predicted by every cyberpunk novel from the 80s X-D and we are gonna have front row seats 8-D
htrp 16 hours ago [-]
>The browser swarm from earlier this year peaked at roughly 1,000 commits per hour on Git. The new system peaks at around 1,000 commits per second.
>To facilitate this rate of activity, we built a new version control system (VCS) from scratch. Throughput was not the only reason to own this layer. Every change in the system passes through the VCS, so it is where collisions first become visible, and several of the coordination mechanisms in the next section are implemented directly inside of it.
Talk about inventing the universe to make a button.
gritzko 7 hours ago [-]
Making a high-throughput revision control system is doable, there have been experiments here on HN with git-in-postgres etc. The question is, what is the actual bottleneck here?
I made a VCS for LLM assisted work, bootstrapping it on its own code as it was developed. In fact, if it did not really work, it would not even exist!
In my experience, keeping track of things and evaluating them was the bottleneck and I focused on that https://replicated.live/blog/worktree Effectively, the question is "how to leverage human evaluation and feedback".
The Cursor people concluded their last post with "taste, judgement, and direction came from humans". Sounded surprisingly sane after "don't compile it, commit it faster" some paragraphs earlier.
sawyers 5 hours ago [-]
People often think of tech/business as bottleneck issues ie whats my limiting factor. That's relevant to pre-ai business as humans are relatively self organizing.
The issue with AI is it has no real memory, context, long term reasoning, etc. These are a lot of what we use to determine how well we're doing something as well as why.
The "bottleneck" so to speak with ai is that it will just circularly build junk forever, expand scope infinitely, and just continue building.
At its best it will autobuild your unicorn saas. But on average it's a cancer cluster of ever expanding code with no plan.
The issue with unsupervised AI is that when you increase the throughput you decrease the amount of human supervision. So you have to have some way to deal with that.
If I'm going to give AI a gigantic task that's really complicated and it will probably mess it up 100s of times, how will I compensate for that without constantly looking over it's shoulder?
wordpad 5 hours ago [-]
But that's mostly just vanilla models.
Some critical parts of agentic development are literally months old while most mature ones are merely a couple years old.
Critical thinking was the most critical checkpoint for models to reach.
Now its only a matter of time for the right tooling and orchestration to come along. Once agent memory and recall works well, all of those bottlenecks will go away one by one.
gritzko 5 hours ago [-]
Back in my early career days, CPUs were clocked at 1GHz. That was 20-25 years ago.
Schlagbohrer 33 minutes ago [-]
They probably just vibe coded it with ChatGPT
ViscountPenguin 9 hours ago [-]
Seems very reasonable to me tbh, the VCS was built to facilitate human work structures, rebuilding it to facilitate autonomous work seems appropriate.
jgalt212 15 hours ago [-]
It's hard not read such quotes and immediately think of the Infinite Monkey Theorem.
If at the end Shakespeare-level literature is produced, does it matter whether we arrived there by random keystrokes?
w-ll 10 hours ago [-]
Shakespeare-level literature was produced. We are the infinite monkeies, and one was shakespeare.
timacles 8 hours ago [-]
How much slop you figure Shakespeare produced?
mock-possum 6 hours ago [-]
Honestly judging from the volume of his output, a ton.
It’s rare to run into prodigious artists who simply produce a few perfect pieces - as a rule those few magnum opuses are built on a hill of cutting room floor scraps.
qjack 12 hours ago [-]
The problem is the infinite other literature produced along the way. It doesn't work to throw more monkeys at the problem of verification, and checking for matches against existing Shakespeare plays is cheating.
7 hours ago [-]
subygan 11 hours ago [-]
don't worry, we can get another swarm of random monkeys typing and decide what ones to eliminate
.
tijebs gi yo,
sawyers 5 hours ago [-]
Right. The joke with shakespeares monkeys or borjes library is that you just do that and then filter out the books that match shakespeares works.
If you already know what you want in an infinite library, it is there. If you don't know exactly what you're looking for to the letter, you might not be able to find it.
grey-area 6 hours ago [-]
And if in the end it does not produce masterworks, going faster and breaking things will be seen as a huge and credulous waste of time that poisoned the internet and some codebases beyond redemption.
But I’m glad someone else is running all these experiments so we can see who is right in their guess about the outcome.
gritzko 7 hours ago [-]
2*128 is the UUID namespace, which is astronomical. That is like 21 letters. Making one sentence of Shakespeare by random search is strictly 0% chance. SHA256 is ~43 letters then. "LLM broke all of cryptography and creates random data to fit a hash" -- are we there yet?
Random search does not work at this scale. In all their cases, they played with pre-existing well-studied software, so they had an embedded fitness function. An extremely detailed one.
Monkeys have been reading Shakespeare all their lives (training). Effectively, this is LLM uncompression. Hence, "in Rust, from nothing but its documentation" -- not entirely true.
sawyers 5 hours ago [-]
What embedded fitness fuctions are you thinking of?
gopalv 10 hours ago [-]
> does it matter whether we arrived there by random keystrokes?
Douglas Adams said this beautifully in the Campaign for Real Time & the poet Lallalfa.
And I just found out h2g2.com is no longer online, so no reference.
voidhorse 13 hours ago [-]
Yes because the keystrokes aren't free and the world is finite. We need to pay these monkeys in bananas.
confidantlake 11 hours ago [-]
Yes because it is akin to producing a universe of slop and trying to find the single atom of gold.
leoqa 11 hours ago [-]
I mean why even use VCS at that point?
d5lt5 6 hours ago [-]
imho, to be able to revert and recall as agents loose context due to new spawn or compaction.
edg5000 4 hours ago [-]
I initially thought that getting agents to work for longer and in large groups was the future, but I'm increasingly thinking that, at least for engineering, just one thread makes more sense. The agent pulls things into context as needed. One thing that I've been experimenting with is also letting the agent remove things from context, such as files. But just adding to the context and compacting when it's full seems like it might beat a lot of more advanced options. Because the model is good, it knows what to put in the summary; just enough for the model to be able to rebuild the context from that seed (e.g. pulling in relevant files/data into the context).
Schlagbohrer 41 minutes ago [-]
Amazing test. I wish I could spend $2000 of compute on having my own swarm build some weird piece of software with 1000 commits per second.
handfuloflight 16 hours ago [-]
> To test that progress, we returned to a task the old swarm had struggled with: building SQLite from scratch, in Rust, from nothing but its documentation.
Isn't SQLite's source code in its training data?
overgard 9 hours ago [-]
Of course it is
Romario77 15 hours ago [-]
not in Rust. might be different enough ...
kgeist 14 hours ago [-]
Even if no Rust code for it was seen during training, an LLM can trivially transpile SQLite's C codebase to Rust on the fly. For example, I just asked ChatGPT to write John Carmack's famous Fast Inverse Square Root algorithm in Erlang, without searching online or thinking, and it transpiled it immediately (while also extracting the knowledge in the same step). SQLite's semantics/code are stored in the middle layers of an LLM, and the last layers are able to convert it into any representation, as conditioned by the prompt. Cursor's experiment is deeply flawed because they merely extracted the model's compressed, lossy knowledge of SQLite's codebase and then just ran a bunch of tests/fixing rounds to make up for the lossiness. The claim that the agents built it from scratch is false.
wrs 12 hours ago [-]
Then you would expect the implementation to be structured the same as SQLite, and having glanced at the result, it looks like at least some things aren't. For example, it seems to use an operator-tree executor rather than SQLite's bytecode interpreter.
kgeist 2 hours ago [-]
The knowledge is lossy, and code generation itself is non-deterministic (temperature), so the operator-tree executor must be interference from other DB implementations, because it's uncommon to have a bytecode interpreter
6 hours ago [-]
mrtesthah 6 hours ago [-]
To test this hypothesis you would need to look at the initial version before subsequent agents started refactoring it.
NitpickLawyer 3 hours ago [-]
Why tho? The whole point of agentic is that it takes several passes to get to a result. If step a) is a non-idiomatic transpile, but step b) c) and ... z) make it idiomatic and performant and efficient, why look at step a)? That was just the beginning. The idea with "agents" and "swarms" and dynamic this and that is that you only look at the last step.
This strictly follows the early agentic usage, where people were complaining of code full of errors and such. But having the loop do plan-write-verify is the thing that got us from that state to today. And it works. If you look at the code and it has errors (linting, etc) then your loop is not properly setup. You should only be called upon when all that can and should pass does. Your role is "direction/taste/human-common-sense-at-a-glance".
hiddendoom45 15 hours ago [-]
There's Turso which is a sqlite rewrite in Rust with some additional features. It's open source so I'd imagine it would be in the training data.
shay_ker 14 hours ago [-]
How do we know if these models weren’t trained on Turso’s rewrite of SQLite in Rust?
It seems both likely that they were and impossible to remove that code from pretraining. Doesn’t that make this just about LLM memorization of the training set? What am I missing?
IanCal 13 hours ago [-]
> What am I missing?
They’re testing the same models on the same task but different ways of organising the swarms and the new approach works better.
handfuloflight 14 hours ago [-]
This is more about automated long horizon work.
adamtaylor_13 14 hours ago [-]
Wouldn't that be in the authors' interest to disclose, if true?
whinvik 15 hours ago [-]
I would have loved to see more of the harness engineering shared as code. Instead we are left with only the outcome.
I guess that makes sense since the harness is the product in the case of Cursor.
dakolli 14 hours ago [-]
I call this junk "meta-agentic engineering", it reminds me of people who have the coolest nvim configs, spend hundreds of hours customizing it but ultimately get less work done than the guy with minimal workflows, if any at all.
I look on twitter and it's just people building tools for agents to use agents, some weird customization loop going on in the LLM space right now. Ultimately these are trends pushed on us from model providers because they 10x token consumption. Its literally just BS trends to increase revenue at these companies, most of it is largely useless.
svachalek 14 hours ago [-]
Some of this stuff is ludicrous. I finally tried /loop last week and discovered every loop iteration passes the entire context history to the model. So pretty quickly you're running a full 1M context window, without cache, likely just to check if something is ready or needs to be done. It's miserably terrible engineering unless your entire and only goal is to burn tokens.
Wowfunhappy 14 hours ago [-]
Wait, why isn't context cached with /loop?
pzh 13 hours ago [-]
Because it's typically cached for 5min (1hr is a setting you have to explicitly opt into), and very few people run loops at a cadence of < 5 mins.
Wowfunhappy 13 hours ago [-]
Oh! I thought Anthropic cached for one hour (by default), am I wrong about that? Or is this an OpenAI thing?
poly2it 12 hours ago [-]
Anthropic changed their cache duration for some reason a while back.
ihsw 13 hours ago [-]
[dead]
blmarket 13 hours ago [-]
I don't think there is a lot test harness, given there is a golden implementation (sqlite3), with clear interface (SQL, and sqlite3 db files). Once you slop crappy implementation, then you can create one group of agents finding gap where slop doing wrong vs golden, and another group of agents filling the gap meeting the conformance.
I assume they did similar, as long as test case(or spec) can be parallelized. Having parallel execution / testing environment is crucial if we need 1000 commits/second throughput.
I would not surprised they have 1+ millions of specs.
I would bet they won't release generated specs - actually this will be a valuable asset like https://sqlite.org/th3.html
trevswarm 9 hours ago [-]
Even at a much smaller scale of a few dozen parallel agents working I've found real benefit to a structured hierarchy of agents. Not just keeping the context clean and focused at the right level, but if you find that something has just gone off the rails, you can more easily rip out or even just fully delete a leg of work without damaging the original design. It's also nice when you find a design change is needed and you can communicate that to the top level and have it efficiently propagate down where it matters.
I like it so much I even recorded a video walking through the process!
Anything architectural like this is super interesting. We keep getting more powerful models sure. We're getting agents ok. But this coordination of many agents. This is the part that's really going to scale very fast and across many many machines in parallel. Very cool.
timcobb 10 hours ago [-]
> One reading is that it was more productive. Another is that most of those commits were busywork (thrash, contention, churn).
Kudos
smoyer 11 hours ago [-]
This is almost a year behind Steve Yegge's first post on beads. Gas Town and Gas City provide orchestration for the swarm. So far I haven't seen a perfect implementation but this idea isn't new.
sudb 9 hours ago [-]
I don't think Gas Town was actually all that much ahead of the curve - see Cursor's swarm post from January this year, mentioned at the very top of this blog post.
I also don't know that beads is a very strong example that Yegge knows what he's doing - it's famously been described as (pseudo)malware.
IceDane 6 hours ago [-]
Everything Steve yegge has done has been trash. That's why nobody is talking about beads or gas town. It was clear even in the beginning that it was a borderline AI-psychosis-fueled trash fire.
nullsanity 10 hours ago [-]
[dead]
vessenes 13 hours ago [-]
This is super fascinating, and I loved seeing testing of where exactly you need frontier intelligence -- looks like coordination / planning, but not coding right now -- the article's a bit of a tease, as we can't play with such a harness, or their new version control system or get a workable artifact out of it.
That said, I love the work on figuring out these harness coordination jobs. While there are analogs to human management there is also this enticing feeling that, since the models are broadly deterministic, we might be able to get repeatable science-type lessons about managing them with enough testing.
This post describes ideas found in existing agent orchestrator systems since they've existed. You'll find the same "top level agent that breaks a problem down into spec / plan / implement / verify" pattern in get-shit-done or superpowers.
Is the original part the VCS that works at 1000 commits/second? Or spending a bunch of money comparing models?
pianopatrick 14 hours ago [-]
I wonder what would happen if the agents did not have the spec. Like if you just said "build a simple SQL database that works as a single file", what would the agent swarm come up with.
mccoyb 16 hours ago [-]
I find these blog posts (and the originals, with Anthropic's C compiler and Cursor's browser) somewhat funny, as if they have this enormous power to build ... but they can't build something unique or new. Like the software sucks, but look how powerful the process is (the models are indeed powerful).
And it's a bit of a shame: by virtue of their position (their embedding in the fabric of venture capitalism), it seems like they can only make a subset of things -- what they can make is dictated enormously by capital, as they are engines of capital.
Not sure the point I'm trying to make, I just find it amusing.
Perhaps the point is that it might be more worthwhile to give independent creators a billion dollars to play around with agent swarms if we want to keep diversity in the evolutionary algorithm that is the software industry high.
brap 15 hours ago [-]
Because it's mostly a load of crap.
Most of these reusable, automated workflows sound really cool in theory, but in practice either they require a lot of customization (not really reusable) or a lot of babysitting (not really automated).
AI has improved my productivity significantly, I'm not a hater, but we also have to be honest about its (current) limitations. The whole "software factory" thing is just a pipe dream people sell to execs / investors / online courses.
thewhitetulip 9 hours ago [-]
Yes and this can be proven because what Anthropic says and does is vastly different
They say SaaS is dead and you can vibe code any SaaS using Opus and yet despite having Mythos with them they have to use JS terminal to run Claude Code.
Why don't they just vibe code CC? Why doesn't Dario have a 100000 agent swarm to run entirety of Anthropic or at least software department?
jnwatson 8 hours ago [-]
They've said for some time that Claude Code is 90% LLM generated code.
thewhitetulip 6 hours ago [-]
Yeah and it's still a cli tool running on JS
So it's not a brag
They could just write in Rust directly can't they? If they do that then they're doing what they preach
solumunus 4 hours ago [-]
My app is written in TypeScript. With agents I could rewrite it in Rust, I just don’t need to so I don’t. It doesn’t warrant that level of performance and the business logic is more digestible in its current form. Given the amount of reviewing I need to do now, readability is worth more than performance. There are always tradeoffs.
Are you implying that them attempting a Rust rewrite would fail, and this is why they’re not doing it?
thewhitetulip 41 minutes ago [-]
> business logic is more digestible in its current form.
Well, Anthropic spent many $ teling everyone that code is dead so this doesn't matter to them does it? Their business logic can be written in English!
attentive 3 hours ago [-]
claude code VSS - 70Gb
codex VSS - 600Mb
IceDane 6 hours ago [-]
Also famously terrible. If not for the walled garden and the fact that anthropic can take away our ability to use custom harnesses any day, I would not be using CC. Given some of the features they add I've also started to suspect that their models are making product decisions. Not necessarily intentionally, but more in the sense that I think their engineers just let their models build and add whatever it feels like is a good idea to the codebase and then they just publish that.
maherbeg 15 hours ago [-]
Yes, that's true, but in the post, they even mention that building the spec is the scarce resource.
For that to work, the swarm has to actually follow the spec, which is what much of this post is about. We gave the swarm 835 pages of prose and it came back with a database. What was scarce in this experiment, and what we expect to be scarce in software engineering going forward, is the right description of intent.
inciampati 15 hours ago [-]
but that's what we use the agent "swarm" to do .. it's also not scarce. what's scarce is imagination.
viccis 14 hours ago [-]
We're rediscovering the "architect" job role a decade or so after switching to agile processes and staff engineers largely replaced it. God help us when they rediscover UML for agents.
chrisweekly 10 hours ago [-]
I feel like the scarcity of (idealized) Product Managers is even more acute.
viccis 5 hours ago [-]
Easily the biggest delta between a good one and a bad one I've ever encountered. It's so easy to test too. Just ask a product manager if they can give a talk on <some product they own> in 2 hours. If they respond by slacking all of the engineers that work under them to stop what they're doing and give them the info to give that talk, fire them. If they are ready to talk right now then give them a raise.
14 hours ago [-]
SpicyLemonZest 15 hours ago [-]
I mean, how long are these pages? Assuming 40 or so lines per page, that would be 33400 lines of spec turned into 200,000 lines of Rust, or about 6 lines of code per line of spec. That's a very detailed spec, much more than any software specification I have ever written in my career. It's not obvious to me that writing this spec would meaningfully save work over incrementally building individual features, especially when you think about what it would take for human beings to meaningfully review whether those 835 pages contain the right intent.
FuckButtons 14 hours ago [-]
not only that, but what all of this misses is the fact that the specification that they used was a product of writing the software, not the other way round. It would be much, much more difficult to first write a correct specification than to first write a correct program, because programs are formal systems that can be executed to observe their correctness. You can’t do that with a specification.
podgietaru 14 hours ago [-]
Or to validate the accuracy and efficacy of the actual code produced.
arjie 15 hours ago [-]
Well, the tool works well. I am grateful that Nature has seen fit that the same kind of person who can make it doesn’t also have the ability to identify what needs exist in the market because then all software would flow out of OpenAI and Anthropic and the rest of us would run about their legs and peep about. Whether product development will survive the next generation of models is still unknown.
dakolli 14 hours ago [-]
bro it can't even make a button that doesn't look like it was designed in 2018, everything is ass. Product development isn't going anywhere.
15 hours ago [-]
ta93754829 8 hours ago [-]
it feels kind of like a concept car a large manufacturer would show off at a motor show. Everyone knows there is no intention to actually ship that thing, but they're exploring concepts and technology and trying to make inroads in what comes next.
viccis 14 hours ago [-]
The bottleneck has always been on the product side for 99% of cases. For the 1% where it's engineering, LLMs don't help as much.
dougSF70 12 hours ago [-]
Can a swarm of agents write the 835 page manual?
overgard 9 hours ago [-]
Good news, the new manual will be 83500 pages.
lubujackson 6 hours ago [-]
That's 100x productivity right there.
bofadeez 14 hours ago [-]
The issue is that the only models that could be trusted to work autonomously cost more than a human employee
uf00lme 7 hours ago [-]
Cursor is saying we need to get improve the harness and tooling, but the models are getting better. One would have to assume that will continue and that cheaper/smaller models will continue to become even more useful.
onion2k 7 hours ago [-]
If there's a human who could write SQlite in an hour we should be paying them at least that much.
doctoboggan 13 hours ago [-]
I feel bad for the engineers at Cursor who have to use Grok in these sorts of experiments.
ryankuykendall 8 hours ago [-]
They received $60 billion to use grok and make it better. I think they going to be okay.
alienbaby 9 hours ago [-]
Tinfoil hat: what if the prices are artificially inflated deliberately to price out casual users being able to field large swarms of frontier model agents because it could be just too dangerous.
onion2k 7 hours ago [-]
The fact that the swarm needs high quality docs to work from will keep most of the tech industry safe.
The interesting frontier is elsewhere: how to fit it into the pipeline to build new useful things. I felt motivated enough to write a follow-up/reply post http://replicated.live/blog/follow-up
Therefore your agent swarms don't have to be cutting edge API calls, they can be hosted on clusters of tiny cheap computers with small CPU GPU. For orgs you could basically invest in a local server that has 100s of small computers.
You can use engineers and frontier models to plan and do task subdivision. And then shit that out to the local cluster that's basically handling each small task without knowing what it's supposed to do.
The real issue is the middle. Testing, verifying interconnectivity of mid level abstractions, building that amount of tech debt at such a high rate and actually being aware in any way what it is that you've built.
I think for most MBAs, considering the US and a significant amount of global markets only care about short term market upside, and AI is still in the middle of the hype cycle, that's not a serious issue for anyone who is profit oriented.
It's only a real issue for the losers like you and me who care about sustainability and infrastructure that survives longer than one market hurricane season.
For my current setup, the most efficient way is to use larger models for coordination, but a heretic'ed qwen 30B model for the implementations. If you build your agentic environment around specifications and test coverage tied to symbols, you can do a lot of parallelization of agent work.
If you then separate the filesystem and tool read/write access by agent roles and policies (e.g. coder not allowed to modify unit tests, tester not allowed to modify code files) then you can force them to use a centralized per symbol/per contract specification tool.
And then you can just let agents discuss issues, where the messaging threads are also tied to the same symbols.
[1] Exocomp, highly experimental: https://github.com/cookiengineer/exocomp
>To facilitate this rate of activity, we built a new version control system (VCS) from scratch. Throughput was not the only reason to own this layer. Every change in the system passes through the VCS, so it is where collisions first become visible, and several of the coordination mechanisms in the next section are implemented directly inside of it.
Talk about inventing the universe to make a button.
I made a VCS for LLM assisted work, bootstrapping it on its own code as it was developed. In fact, if it did not really work, it would not even exist! In my experience, keeping track of things and evaluating them was the bottleneck and I focused on that https://replicated.live/blog/worktree Effectively, the question is "how to leverage human evaluation and feedback".
The Cursor people concluded their last post with "taste, judgement, and direction came from humans". Sounded surprisingly sane after "don't compile it, commit it faster" some paragraphs earlier.
The issue with AI is it has no real memory, context, long term reasoning, etc. These are a lot of what we use to determine how well we're doing something as well as why.
The "bottleneck" so to speak with ai is that it will just circularly build junk forever, expand scope infinitely, and just continue building.
At its best it will autobuild your unicorn saas. But on average it's a cancer cluster of ever expanding code with no plan.
The issue with unsupervised AI is that when you increase the throughput you decrease the amount of human supervision. So you have to have some way to deal with that.
If I'm going to give AI a gigantic task that's really complicated and it will probably mess it up 100s of times, how will I compensate for that without constantly looking over it's shoulder?
Some critical parts of agentic development are literally months old while most mature ones are merely a couple years old.
Critical thinking was the most critical checkpoint for models to reach.
Now its only a matter of time for the right tooling and orchestration to come along. Once agent memory and recall works well, all of those bottlenecks will go away one by one.
https://en.wikipedia.org/wiki/Infinite_monkey_theorem
It’s rare to run into prodigious artists who simply produce a few perfect pieces - as a rule those few magnum opuses are built on a hill of cutting room floor scraps.
tijebs gi yo,
If you already know what you want in an infinite library, it is there. If you don't know exactly what you're looking for to the letter, you might not be able to find it.
But I’m glad someone else is running all these experiments so we can see who is right in their guess about the outcome.
Monkeys have been reading Shakespeare all their lives (training). Effectively, this is LLM uncompression. Hence, "in Rust, from nothing but its documentation" -- not entirely true.
Douglas Adams said this beautifully in the Campaign for Real Time & the poet Lallalfa.
And I just found out h2g2.com is no longer online, so no reference.
Isn't SQLite's source code in its training data?
This strictly follows the early agentic usage, where people were complaining of code full of errors and such. But having the loop do plan-write-verify is the thing that got us from that state to today. And it works. If you look at the code and it has errors (linting, etc) then your loop is not properly setup. You should only be called upon when all that can and should pass does. Your role is "direction/taste/human-common-sense-at-a-glance".
It seems both likely that they were and impossible to remove that code from pretraining. Doesn’t that make this just about LLM memorization of the training set? What am I missing?
They’re testing the same models on the same task but different ways of organising the swarms and the new approach works better.
I guess that makes sense since the harness is the product in the case of Cursor.
I look on twitter and it's just people building tools for agents to use agents, some weird customization loop going on in the LLM space right now. Ultimately these are trends pushed on us from model providers because they 10x token consumption. Its literally just BS trends to increase revenue at these companies, most of it is largely useless.
I assume they did similar, as long as test case(or spec) can be parallelized. Having parallel execution / testing environment is crucial if we need 1000 commits/second throughput.
I would not surprised they have 1+ millions of specs.
I would bet they won't release generated specs - actually this will be a valuable asset like https://sqlite.org/th3.html
I like it so much I even recorded a video walking through the process!
https://www.youtube.com/watch?v=efUmcCiRoDU (I do walk through how I do this in our app DevSwarm here)
Kudos
I also don't know that beads is a very strong example that Yegge knows what he's doing - it's famously been described as (pseudo)malware.
That said, I love the work on figuring out these harness coordination jobs. While there are analogs to human management there is also this enticing feeling that, since the models are broadly deterministic, we might be able to get repeatable science-type lessons about managing them with enough testing.
Is the original part the VCS that works at 1000 commits/second? Or spending a bunch of money comparing models?
And it's a bit of a shame: by virtue of their position (their embedding in the fabric of venture capitalism), it seems like they can only make a subset of things -- what they can make is dictated enormously by capital, as they are engines of capital.
Not sure the point I'm trying to make, I just find it amusing.
Perhaps the point is that it might be more worthwhile to give independent creators a billion dollars to play around with agent swarms if we want to keep diversity in the evolutionary algorithm that is the software industry high.
Most of these reusable, automated workflows sound really cool in theory, but in practice either they require a lot of customization (not really reusable) or a lot of babysitting (not really automated).
AI has improved my productivity significantly, I'm not a hater, but we also have to be honest about its (current) limitations. The whole "software factory" thing is just a pipe dream people sell to execs / investors / online courses.
They say SaaS is dead and you can vibe code any SaaS using Opus and yet despite having Mythos with them they have to use JS terminal to run Claude Code.
Why don't they just vibe code CC? Why doesn't Dario have a 100000 agent swarm to run entirety of Anthropic or at least software department?
So it's not a brag
They could just write in Rust directly can't they? If they do that then they're doing what they preach
Are you implying that them attempting a Rust rewrite would fail, and this is why they’re not doing it?
Well, Anthropic spent many $ teling everyone that code is dead so this doesn't matter to them does it? Their business logic can be written in English!
codex VSS - 600Mb