Hey friend,

The sentiment among many prominent people is that we are seeing the beginnings of AGI.

For instance, here is the CEO of Quora making a rather bold claim:

Adam D'Angelo @adamdangelo AGI (AI at avg human level for any job that can be done remotely) will massively impact GDP growth. We might expect >50%/year. Assuming a continuous run-up to AGI, we should expect GDP growth to start to accelerate as we get closer. When will this be noticeable in GDP metrics? February 21st 2023 62 Retweets 763 Likes

But what do the people who understand the technology think?

Christian Szegedy, a legendary researcher from Google Brain, has this to say:

Christian Szegedy @ChrSzegedy The confusion of people assigning human attributes stems from the fact that humans use language to communicate their internal state quite often. LLMs never communicate their internal state. They just aim to model the distribution of past human outputs in certain contexts. February 16th 2023 3 Retweets 47 Likes

He is highly critical of anything magical happening there.

"This is just a complex statistical model which excels at predicting the next token and is able to make good use of the enormous corpus of data that it was fed".

Yann LeCun, one of the founding figures behind modern Deep Learning, is equally critical.

Yann LeCun @ylecun Language abilities != Thinking. Or why LLMs such as ChatGPT can eloquently spew complete nonsense. Their grasp of reality is very superficial. theatlantic.com/technology/arc… This piece in the Atlantic comments on a paper by the MIT Cognitive Science crowd arxiv.org/abs/2301.06627 The Difference Between Speaking and Thinking The human brain could explain why AI programs are so good at writing grammatically superb nonsense. theatlantic.com February 1st 2023 374 Retweets 1,677 Likes

But how can we reconcile both of these worldviews?

Is ChatGPT the precursor to AGI or is it just an advanced statistical model that does something superficial?

The answer lies in the nature of intelligence and consciousness.

To understand the world around us we have to look deep inside ourselves.

And here there are many opposing takes as well!

For instance, Sir Roger Penrose, who was awarded a Nobel prize for his work on black holes, says that "consciousness must be beyond computable physics".

He claims that consciousness is an emergent phenomenon with its root in quantum physics and makes some very convincing arguments for why it might be so.

But for a contrarian take all it takes to look at the world around us.

For instance, in Poland the ruling party is all about propaganda. They are expert communicators and go to extreme lengths to craft and package their divisive messaging.

If I talk to people who subscribe to the world-view of that party, it is as if I was talking to automatons! They literally spit out the party line word for word!

This is an extreme example but what assurance do we have that this is not how things work at every level?

That my words are expressions of a deep emotional state and are not just something my brain inferred would make sense in a given context based on what I observed in my life?

Maybe when we listen to another person, if we trust and respect them, our "neural network" gets retrained to reproduce what we are hearing?

Maybe that is how we acquire language and behavior as children?

Maybe we are nothing more than very sophisticated machines for predicting the next token?

If that was the case there would be nothing magical to intelligence! Humans would be very much like ants.

The advance of civilization would not be driven by individuals but by reprocessing and refining information between generations.

For instance, people were toying with Infinitesimals for years but then Leibnitz and Newton came along and gave us "calculus".

And they were just a tiny stop in the progression of knowledge all the way since ancient times.

They used their neural networks to refine what was already there and make some additions.

And as we know, they didn't have the last word. Calculus continued to be formalized and extended even after they passed away.

So which is which?

Are we spiritual creatures where what we say and do is governed by the divine spark of our soul or personality?

And in that case the ChatGPTs of the world are but an ingenious, even though very powerful, toy.

Or are we nothing but complex information processing units that are outstanding at predicting the next thing to say and the next action to take?

And in that case ChatGPT is our close relative.

I don't think anyone has a convincing answer.

But pondering such questions is infinitely fun!

My favorite things ❤️

📺 The upcoming GTC conference - I had been in love with GTC for many years before I joined NVIDIA (just a couple of threads I wrote on it in the past). This is probably the best (and most entertaining!) way to stay in touch with the AI field. Self-driving cars but done properly without the hype and in broad collaboration across industry, companies creating digital twins of their factories, real time ray tracing... Absolutely fascinating! A new iteration of the conference will take place on March 20-23 and you can sign up for it for free here. Depending on where you live some of the sessions might be at really weird times, but all of them should have recordings available.

👻 PKIM woes continued - continuing on the theme from the previous newsletter, there is a lot of unfounded optimism surrounding personal knowledge management tools and solutions. But among it all, several books stand out. One of them is certainly How to Take Smart Notes by Sönke Ahrens. If you'd like to learn the history behind the Zettlekasten method, learn how note taking plugs the gaps in our psyche to enable improved performance, this is certainly worth your time!

✍️ How to train and deploy a Two-Tower RecSys solution - there is a big disparity between "RecSys tutorials" and what actually happens in the industry, so materials that cross that divide are very valuable. It just happens that my colleagues in collaboration with a couple of very interesting individuals (Jacopo Tagliabue and Hugo Browne-Anderson) authored such a piece of writing (along with code). How to train a Two-Tower model using Metaflow for orchestration, how to server predictions using AWS Lambda, and more! You can read about it all here.

My new video 📺

I teamed up with Hamel Husain to record a video on one of the most underrated MLOps tools, GitHub Actions.

Here are just a couple of topics we cover in the video:

✅ the ChatOps pattern
✅ deploy on merge
✅ run tests on push
✅ using cron with GitHub actions
✅ ... and much more!
​
Hope you enjoy! 🙂

Quote(s) of the Week ✍️

Sönke Ahrens in How to Take Smart Notes

Michael E. Gerber in The E-Myth Revisited