AI at Meta, Open Source, Limits of LLMs, AGI and the Future of AI
https://www.linkedin.com/posts/jacquesludik_yann-lecun-meta-ai-open-source-limits-activity-7171810012837752833-DfsE
Excellent conversation between Lex Fridman and Yann LeCun about AI at Meta, Open Source, Limits of LLMs, AGI and the Future of AI.
Good background for some of the technical discussion on Yann’s proposals and work can be found in some of his papers such as “A Path Towards Autonomous Machine Intelligence” https://openreview.net/pdf?id=BZ5a1r-kVsf
See also my article on “Intelligent Agents, AGI, Active inference and the Free Energy Principle” and questions about reasoning, understanding, goals, planning, agency etc. to make real progress in AI. https://lnkd.in/d4WebpXs and https://lnkd.in/gE_5m9vC.
See also other articles in the “Democratizing AI” Newsletter https://www.linkedin.com/newsletters/democratizing-ai-6906521507938258944/
Some of the key topics addressed in Lex’s latest discussion with Yann:
🎾 Importance of open source AI in countering concentration of power and shaping a positive future for humanity.
See also my book on “Democratizing Artificial Intelligence to Benefit Everyone: Shaping a Better Future in the Smart Technology Era” jacquesludik.com that shares similar sentiments.
🎾 The process of language generation involves auto regressive prediction, distinct from pre-planning speech regardless of language.
🎾 Training joint embedding predictive architecture (JEPA) requires contrastive learning to avoid collapse and produce meaningful representations.
🎾 Innovative AI techniques: I-JEPA and V-JEPA for image and video masking, improving video representation prediction.
🎾 Self-supervised learning in AR LLMs.
🎾 Uncovering the depth of human communication beyond language through implicit cues and humor.
🎾 Development of a new AI system for complex problem-solving using internal world models.
🎾 Recommendations to shift from generative models to joint embedding architectures for improved image representation.
🎾 Utilizing open source platforms to enhance AI systems for diverse applications and industries.
🎾 Ethical considerations in AI development, including defining hate speech and dangerous content, and the limitations of large language models.
🎾 Intelligence is multidimensional, making it impossible to compare entities based on skills. AI doomers’ catastrophic scenarios are based on false assumptions.
🎾 Fear of new technology’s societal impact is rooted in human psychology and resistance to change.
🎾 Challenges in training world models, planning in non-physical systems, and scaling up innovative work in AI learning world models, reasoning, understanding, goals, planning, agency, etc., that are all key to making real progress in AI.
