"Not Even Wrong" Podcast
Investing in fundamentally new concepts and engineering practices with large impact.
Chronopunk. (A novel)
“If you went back in time, what knowledge would you gift the past to save the future?”
Synopsis of novel
Reaction to Herbert Diess interview. Innovator’s Dilemma. My interpretation: when facing disruption, managerial class is buying time by going up market. New moral compass needed. Founder types don’t act like this. Tesla has been disrupting car market by offering software defined car with lower cost structure. Eventually, every company is defined by its cost structure. Now facing disruption in software defined car from Chinese and others. Response is full integration, lower cost, agility.
π0: A Vision-Language-Action Flow Model for General Robot Control. Sergey Levine et. al. Sergey talks about π0 model. Pre-train on VLA on internet scale data. Add robot specific actions for the Action part. Fine tune on specific tasks with experts. General means a lot of things in robotics. Very berkleyesqe = build large, distributed, open source systems (Cyberpunks, Spark, RiskV). Similar to Unix, which was designed as an operating system that is not hardware depended. π0 is robot hardware independent. Mix of experts. VLM + Action expert. Action is diffusion based. Pre-train data should be diverse, not just experts, so robot learns how to fix problems. Tokenization in robotics not optimized yet. Normalization of tokens. How much transfer learning is there from FSD to Optimus? Can there be a more general model? Robotics is in essence a navigation problem.
Tesla Q4 comments. 1/2 FSD competition in China. Best way to build real world AI system is with fully integrated system. From metal to algorithm to data collection. Tesla yes, Chinese no. Patchwork. Why is China not letting Tesla export training data? We let 400,000 Chinese students study in the US. Reciprocity please. 2/2 Battery constraint. Why? Demand or supply. Coordinating different supply chains.
Comments on JD Vance Munich speech and Lanz und Precht. JD Vance draws demarcation line between the US understanding of liberty and Europe. For centuries Europe has been building a culture of elitist liberty. The US starts from the bottom up, from the individual. Since 1945 the US has been trying to impose this idea of liberty on Europe. JD Vance now drew a line. No more funding and support. Europe must build their own sovereign system. That’s where I agree with Lanz und Precht. But Europe is not a political idea, it’s a geographic concept. Return to nation states. Align purse and sword with constituency.
Neurips 2024 Part 16.Return of Unconditional Generation. Kaiming He. The goal of representation learning is to compress input in latent space so it can be retrieved efficiently. Pixels are high dimensional. Model learns how to represent pixels. Feature extraction unsupervised. Presentation by Kaiming. Deep learning = representation learning. Input data is way too high dimensional. Necessity to reduce dimensionality into useful representation. History of deep learning: Recurring Neural Networks, Convolutional Neural Networks, Transformers. Same ideas, more cross attention. Normalization important in end to end learning.
DOGE, Tesla and macro. Doge impact will be like technology - “overestimated in the short run and massively underestimated in the long run.” Trump effect on Tesla. Europe yes, rest of world no. Tesla sales sluggish due to Juniper transition Model Y. DOGE upside. First time since invention of democracy in Athens that an empire is undying corruption. De-Byzantinizing through DOGE. If corruption slows down economy, the inverse accelerates it. We agree with ARK Invest that there is good chance for high single digit real growth in the coming five years. Tug of war between S&P500 and Gold. DOGE succeeds, S&P up, otherwise Gold.
Discussing Essay “From Shakespeare to Move 37”. Creativity evolves with the Zeitgeist. It depends on how we perceive learning. What is learning? What is the purpose of intelligence? Creativity is purposeful exploration. Today, artificial Intelligence opens new, interesting questions of what learning is and what creativity means. Move 37 is a first look at what modern AI system will be able to do. Knowledge and iteration through RL will evolve novel forms of creativity. The purpose of creativity is to build new niches and wealth.