Weekly Links: Autosearch, Metacognition, and Machine Consciousness

This week, Grammarly backtracked on AI "Expert" Reviews, we're about to get agent email, and your future pay packet may include AI compute (funny and dystopian at the same time...).

Here are the main stories I picked out from the week:

• OK, well. I ran /autoresearch on the liquid codebase. This tweet from Shopify's CEO, Toby Lütke, is almost throwaway but seems quite important. He used Andrey Karpathy's Autosearch setup to refactor Shopify's main rendering library (which has been in development for 20+ years). He himself says that probably some of the improvements will turn out to be wrong, but the sheer power of throwing long-running computation with a goal at a codebase is impressive. My guess is that this approach will get stuck in local minima, and you might get faster just by starting from scratch. However, doing this along multiple performance dimensions will probably mean a slew of real changes that can improve things. Taking this approach would be impossible with human programmers: we're too attached to priors, and we don't have the patience to throw mud at a wall for many hours.
• Amazon orders 90-day reset after code mishaps cause millions of lost orders. At the other end of the AI-coding spectrum is an announcement from Amazon about coding practices. The company has been all-in on using AI coding across most of its systems. However, it seems that things have gotten out of hand, and bugs shipped due to these tools have caused numerous outages and missed orders. The company is now mandating tighter controls on software in part with deeper code reviews. It seems unlikely to me that asking more humans to look at AI-generated code will really help: you either stay mostly human-driven or you go all in and really figure out how to do much of the review in automated ways. Amazon's Kiro spec-driven development environment is actually one of the levers to help do this, but much more is needed. If Amazon struggles, we're likely to see similar problems in lots of other places. As an aside... for the last article, I found almost no tech media or even mainstream media coverage of this, but many mentions across more minor outlets and "AI-generated" sites. Starts to make you wonder which news sources are really authoritative and comprehensive anymore.
• Imagining and building wise machines: The centrality of AI metacognition. Interesting paper essentially on the topic of AI "wisdom" and consciousness, with some well-known AI luminaries like Joshua Benigo and Melanie Mitchel on the author list. I like the emphasis on strategy selection v's tactical thinking, and coupling this with robustness. However, it seems to me that these are not wholly distinct or even "countable" layers of cognition; it's really a fine spectrum from tactical solutions all the way to whistful thought about the nature of the universe, with strategy and tactics in between. For this reason, it seems likely that some levels of wisdom will emerge by enabling more and more abstraction in model learning and reasoning. Adding explicit metacognitive checkpoints and lessons into training (and perhaps the architecture) could well give us wins, but it seems to me that the training might bring us more than architectural changes if abstraction abilities improve. Humans are good at taking a single parable and mapping it to all sorts of everyday situations. We also still get stuck applying the wrong strategic framework to the decision right in front of us.
• The Abstraction Fallacy: Why AI Can Simulate But Not Instantiate Consciousness. This Google DeepMind paper then directly contradicts my argument around abstraction. The paper is an interesting read and, in a nutshell, argues. The author is much more versed in this subject than I am, and given that the argument is about machine consciousness, we're unlikely to find out the answer to what is right. However, fundamentally, it seems to me that giving a computational thread enough context, world input, and some ability to act could easily imbue the entity executing that thread with something analogous to consciousness. It's not at all obvious that this isn't exactly what happened to us (humans) as our brains became more complex. To put it in terms of the paper: I'm not sure there needs to be a mapmaker at all (that creates abstractions and links between concepts). Nor do I think it needs to be grounded in reality, be conscious. Internally consistent universes may well be enough. Interesting discussions for future decades!
• Yann LeCun Raises $1 Billion to Build AI That Understands the Physical World. Yann LeCun thinks he and his team can bridge the gap anyway. Looking forward to seeing what they build!

Wishing you a great weekend!