fd93

AI and the Commons

In Vannevar Bush's seminal 1945 essay As We May Think he describes the different roles of the programmer and the computer as such:

[E]very time one combines and records facts in accordance with established logical processes, the creative aspect of thinking is concerned only with the selection of the data and the process to be employed, and the manipulation thereafter is repetitive in nature and hence a fit matter to be relegated to the machines.

AI is often, and perhaps correctly, described by skeptics and opponents as a "plagiarism machine": it ingests human-written text and spits out a facsimile of greater or lesser verisimilitude.

This is probably not a fully accurate picture of what's happening from a technical perspective, since the reality of calculating embeddings in an n-dimensional space is that no single piece of data survives intact. Possibly - although here I'm going beyond my limited experiencd - one could consider embeddings to represent numerous overlaid pieces of input information.

Overall, thought, it's probably more accurate to say that the training process captures both the average of its training set and a very large number of possible variations on that average.

Yet as numerous experiments (note: need to add citation for the green-eyed girl case) have shown, regardless of whether the content of an embedding consists of plagiarized material, its products often do. Garbage in, garbage out, as they say.

One of Bush's major concerns in the essay is with wasted effort in scientific research: a tragedy, as he puts it, that Mendel's work wasn't discovered for a generation.

In the pre-computing era, the main cause of repeated research was an inability to transfer and share information in an easily searchable way. That problem has largely been solved by computers, the internet, and search engines. Nowadays, the problem is one of discoverability in the vast ocean of data we've created. So, well, perhaps search engines didn't solve the problem.

What if we saw present machine-learning technologies as an accomplishment of Bush's call to avoid knowledge work that is "repetitive in nature and hence a fit matter to be relegated to the machines"?

For example, there's little need for every single programmer to be learning how to write quicksort other than as a learning exercise, because this algorithm is very well understood and has been for generations. Thus we use libraries.

For more specific problems, however, there are often no high quality libraries available. For example, the OAuth2 security standard is not represented by any one library (and arguably cannot be), because the number of components required for a complete OAuth2 implementation, and the variations between them, are very high.

But what this results in practically speaking is many billions of dollars of "research" happening haphahazardly in closed-source environments. Other programmers may have solved the same problem as you, but you will never know because their research is inaccessible. And even if you could access it, it's very likely to be so coupled to its implementation that you can't use it anyway.

Machine learning tools which have wide acceptance and with the widest possible array of input data can solve this problem. By remixing and reformulating existing code in their data set, they can eliminate repetitive work and free up human engineers to focus on innovation, research, and other truly creative parts of the work.

To achieve this, some fairly radical changes may be needed to the current meta:

  1. De-emphasize commercial, proprietary and closed-source / weight AI models as counterproductive. Instead, invest the trillions in open source / weight models. Chinese AI companies, as well as Meta in the US,
  2. Rather than treating coding assistant as black boxes which produce good code "magically", focus on making the provenance of training data understandable to human engineers. Focus on attestation that a given algorithm will work the way that it claims to, perhaps by having known-good solutions to problems.
  3. Treat software engineers as engineers: give them work that amounts to creative problem solving, rather than integration of solutions. There's nothing wrong with integration work - it's the bread and butter of most programmers' careers - but if the proposed library of solutions was in place, it should be possible to identify a solution and integrate it from a jigsaw of pre-existing parts.