9/10
This is a summary of Andrej Kaparthy’s appearance on Lex Fridman’s podcast in late 2022.
The future of programming is not humans writing code, but neural nets creating weights.
Capture Neural networks are mathematical expressions with many knobs that can be adjusted to produce different outcomes The future of programming, as captured in his essay Software 2.0, is neural network weights. AIs will feed data into neural nets and produce weights, and that will be the software. Humans are bad coders. Our job will be to ask the right questions and provide quality data sets. Consciousness is an emergent function of having a large enough understanding in the world, and realizing you’re an object in it
Consciousness is like a modeling insight. You know enough about the world to know that you’re something in it.
Andrej Kaparthy
AI Being a stand-up comedian will be a strong baseline for testing an AGI (Lex’s co-idea) GPTs are next word prediction engines that use training data to come up with solutions. Transformers are a type of neural network that are general purpose differentiable computers, which are expressive, optimizable, and efficient. He believes neural nets are already thinking because he defines thinking as information processing and generalization There are limitations to comparing neural networks to the brain, as they use different optimization processes. You can’t do RL from scratch, you have to start with something like GPT
Read more