We're building in-car intelligence.

Essentially, you get in your car and you can talk to your car like you talk to a friend.

I've been wanting the car that will never be responsible for a collision and become your friend for a decade.

Wow.

I don't know if you are familiar with this, but there was a show in the '80s called Knight Rider.

Exactly.

That's it.

That's what we're trying to build.

Dr.

Daniela Rus.

Welcome to Particles of Thought.

Thank you so much.

Yes.

Thrilled to be here.

I'm so happy to have you.

You are at the very cutting edge of what is on everybody's mind these days, and that is AI.

So it looks like there was suddenly, you know, to me, AI just appeared out of nowhere suddenly, and now it's everywhere.

Was there some like technological breakthrough that occurred at that point just a few years ago?

What made it suddenly roll out for everybody?

And now we have, you know, you know how it is when you're the first to market and you market well, everybody knows your name.

So I think in this era it's ChatGPT, even though there's a ton of different competitors.

What changed to allow ChatGPT to appear, what it seems to be, so suddenly?

So AI has been around for a long time, ever since Marvin Minsky and his friends did the roadmap for the field in 1956.

And a lot of the ideas that power today's solutions, the use of multi-layer neural networks, the use of predictive AI, the use of classification methods, the use of reinforcement learning, all of these methods were invented decades ago, 40 years ago.

Back then, they did not work very well because the computers could not store very big models.

And we did not have so much data.

Over time, computers became increasingly more powerful and data became broadly available.

And then all of a sudden, the techniques that did not work back then started working.

Now, of course, there had to be additional innovation.

And, and the critical moment was when something called the transformer architecture for AI models was invented because— So that's a— because there is a physical device called a transformer in electronics, but this is a code, this is a software transformer.

This is an AI architecture.

Architecture of software or architecture of firmware or hardware?

So think of AI as a model.

It's like, think of AI as a huge graph.

And this graph has artificial neurons for nodes and artificial synapses connecting the nodes.

Now inside every node there is usually a simple computation.

So the artificial perceptron, the artificial neuron, takes in some numbers, adds them up, passes them through some very simple math that puts out a 0 or a 1, and then passes along the result.

And this is very complicated because it's happening across millions and billions of computational units, so across a huge number of units.

Now, the artificial neurons and the synapses have all sorts of parameters.

Think of them as weights that exist and that are used in the computation.

The weights are what gets trained, what gets learned through the training process.

So you show such a model an example, an example and a second example and more and more examples.

And based on those examples, the network predicts something.

If that something is consistent with the ground truth that was in the data, you have a good answer.

If there is an error, if the something is different, then you can kind of figure out how to tweak the parameters.

And this is a process that takes a lot of iterations.

Imagine doing this for every piece of text that exists in the world, in the internet.

It's a very long and expensive process, but this is what leads to the kind of powerful models that people now use with chatbots.

I've never heard that neuron synapse metaphor that you made there.

That makes it so much clearer.

Okay, so now we are in a world where we have this story about what AI is.

And what is the architecture upon which it lives.

And it's typically data centers and AI chips.

And I think to myself, well, you know, if computation is a big thing, uh, as a physicist coming down the line, we have quantum computing, which is today not there yet, but it allows you to do all this in parallel.

Sounds like that would be great for AI in the future.

But I'm thinking cutting edge, you know, probably who knows how far that is into the future, but the simpler question about the near future is, is that current model of architecture, data centers plus, you know, special computer chip that is at the current generation of computer chips with this, you know, EUV lithography, what, 3-nanometer node or something like that.

Is that what is going to remain forever?

That's a great question.

What's interesting is that the transformer architecture is a kind of a purely feed-forward memorizing solution.

In other words, you put information in the model and the model computes, and then you keep doing it.

Eventually the model gets to memorize all the information that you put in.

Yeah.

Okay?

And then the model is used to generate using statistical methods.

Right.

Now these models have to be really big because if they need to know a lot of stuff, all of that stuff has to be put inside.

Furthermore, these models are built on statistics, so there is no representation of physics, of constraints, of the real world in the model itself.

It's not part of how you build the model.

And so for this reason, I believe that there is a ceiling to how much you can achieve with these methods.

But we haven't gotten there yet.

Well, we continue to make these models bigger and bigger and bigger.

And for a while, we saw very sharp increase in how the models were able to support various AI-related tasks.

But the most recent data shows diminishing returns.

The most recent data looks like an S-curve, which means you continue to improve, but not as much as we did maybe a year ago or 2 years ago.

Now, there are some new ideas and alternatives to this approach.

And in our research group, we're using inspiration from nature to come up with other ways of building AI models.

And one of our solutions we termed liquid networks.

And liquid networks, were developed using the mathematics of what happens inside the physical neurons and the physical synapses of small species, in particular of a small species called C. elegans.

This is a worm that has been broadly studied by the neuroscience community, and we really understand a lot about how this this worm operates.

And what's interesting is that this C. elegans has a happy life on only 302 neurons.

302—302 neurons.

Compared to humans that have— Billions of neurons.

Okay, so how is this possible and compared to the huge transformer models that we build?

And this worm is able to do so much on these 302 neurons.

So we were very inspired by the humble worm and by the fact that there is so much richness from such a small number of computational units.

And we created a new model, this liquid networks, that is loosely inspired by the mathematics of the computations inside the neurons and synapses of the worm, but also by how the neurons are wired, are connected together.

And in the case of C. elegans, there are loops in the architecture of the model.

And here, by the architecture of the model, I mean how these computational units get connected.

And loops are very interesting because loops allow you to learn from the past.

If you don't have loops, if all your computation is from input to output, then you do not have a way to kind of, you know, go back and learn from the past.

Liquid networks have loops inside them.

And with that, we get a lot of adaptation and we get other benefits.

Most AI models are frozen after training.

That means that when you use them in the wild, they do not continue to improve.

But because of the computations that are happening inside the liquid artificial neurons, these models continue to learn after training and they adapt based on the inputs that they see.

Moreover, the models are much smaller as compared to existing transformers.

Now, if we train a model that is based on a kind of a deep learning transformer architecture, we will need about 100,000 artificial neurons and about half a million parameters to teach a car how to steer and accelerate.

Is there a certain number of videos it has to use as a dataset?

It's a large number of videos, but you will use the same dataset to train traditional models and liquid models.

With liquid models, you can learn the task using only 19 neurons.

So 19 neurons versus 100,000.

Wow.

A couple thousand parameters versus half a million parameters.

So you see, the mathematics actually matters.

Yeah.

And we get a lot of benefit from the new mathematics.

Also, the loops matter.

So how we connect the artificial neurons, all of that also matters.

And we can get a lot of energy efficiency.

Altogether, the liquid network models are on the order of 100 to 1,000 times more energy efficient than traditional models.

So now imagine with this technology, you can have a solution that runs on your phone without the cloud, that runs on the car, that runs on computers, that runs on enterprise machines that does not involve the cloud, that does not require— No data center.

So exactly.

Wow.

No data center needed.

Holy cow.

Well, what happens to the data center economic model?

Does it only go for the large companies now?

So the data centers are still needed.

There are problems that require the kind of huge models that we need data centers for, but not all problems that we solve with AI require that size storage and that size computing.

There are many problems that can bring economic value and personal benefit that you can run on your phone, on your computer, on your glasses, on your car, on satellites, on power grids.

So many applications.

And maybe you will indulge me and I can tell you that most recently our young startup called Liquid AI, which is bringing these ideas and more to the market, has made a partnership with Mercedes.

And so in the future, there will be in-car intelligence powered by the Liquid AI models.

And actually, I have Liquid AI models running on my phone, and they run when my phone is in airplane mode without the cloud and without any additional support, and they don't kill my battery either.

Particles of Thought is supported by Carlisle Companies, a manufacturer of innovative building envelope systems.

With buildings responsible for over a third of total energy use, Carlisle's energy-efficient solutions are built to reduce strain on the energy grid.

For example, Carlisle's UltraTouch denim insulation, made from sustainable recycled cotton fibers, delivers energy efficiency while being safe to handle and easy to install.

Made with 80% recycled denim, UltraTouch diverts nearly 20 million pounds of textile waste from landfills each year.

Operating across North America, Carlisle is working towards a more sustainable future.

Learn more at carlisle.com.

You know, when you talk about this idea of AI leaving the data center and becoming a part of our everyday world, it reminds me of last year.

I don't know when you're listening to this, but I'm referring to the year 2025.

I go to San Francisco and for the first time in my life, I see this car with all these sensors and spinning things And I tell my colleagues like, what is that thing?

And they're like, oh, that's one of these self-driving cars.

It's like an Uber or Lyft, but there's no driver, right?

So I understand that you are collaborating with Mercedes.

Is it about that type of self-driving technology or is AI being incorporated in a more general way in vehicles?

That's an excellent question.

I love the self-driving cars and that is potentially in the future for the collaboration with Mercedes.

But right now, our collaboration is focused on essentially building a car that will be your friend.

So we're building in-car intelligence.

Essentially, you get in your car and you can talk to your car like you talk to a friend.

The car can do things for you.

You could tell your car, put down the windows, put up the windows, make it warmer, make it colder.

What route should I take for my trip to New York?

What is the temperature?

So you can interact with your car at the conceptual level, and you get the conceptual intelligence, you get the word intelligence, but you can also get the car to turn knobs for you.

And so this is very powerful and very exciting.

I've been wanting the car that will never be responsible for a collision and become your friend for a decade.

And with this collaboration, I think we're getting much closer to that vision.

Wow.

I don't know if you are familiar with this, but there was a show in the '80s called Knight Rider.

Exactly.

That's it.

That's what we're trying to build.

Wow.

You're bringing KITT to everyday Americans or everyday humans around the globe.

I have to say we're dating ourselves.

We're getting excited about Knight Rider.

Yeah.

Yeah.

But that's nuts.

That your car is your friend.

That, I, so that also makes me think, you know, cars are machines that are burning fuel.

They have efficiencies and there are already computers in cars, but they're not intelligent.

So it sounds like not only will they be your friend, but they will also be optimized to, for fuel consumption and all these other elements.

You know, if there's a mechanical problem, if there are sensors, it sounds like you're, the car can say, "hey, I need to go to the doctor, take me in."

Exactly.

So we are building this kind of in-car intelligence that is multimodal, that can support speech as a data modality, but then also sensors and actuators and many other data types.

And this will be a very exciting world that was indeed imagined back in the day with Knight Rider.

Now, the other thing that's interesting to know about cars is that cars are safety-critical systems.

That's right.

And in a safety-critical system, you have to be safe and you have to make decisions very quickly.

If you have a car that drives 60 miles an hour and this car needs to wait for the cloud to tell it what to do, imagine having a latency of 5 or 10 seconds.

Oh yeah.

That could be the difference between safe driving and not safe driving.

Right.

And so the on-device models are very good for safety-critical systems where it's important to be fast when you make decisions and where latency can be a problem.

The other area where on-device AI is very powerful is when you have private data.

With AI that runs on device, you don't have to share your data with a cloud.

You don't have to put your data in someone else's container.

Your model is your own.

Your data is your own.

And so you can do a lot of activities that require that you stay in charge and that your data remains protected.

Safety and privacy are very important for driving and for a host of other applications where, it's important for the data to stay local.

And just imagine a future where this idea of on-device AI can work on any device, on your phone, on your glasses, on your suitcase, on devices we haven't even imagined and we haven't even built.

I like to think about this notion of having my own automatic diary where AI could run on my glasses and could record everything, could map the videos into searchable text.

And then maybe 10 years from now, I could ask, who was that very charming colleague with whom I had this nice conversation about AI back in the day in 2026?

So your whole life could become recorded and indexed and searchable.

So imagine that.

So now we're talking about this in a very utopian sort of way, and I'm just wondering, how could it go wrong?

Like, if we— you know, right now I'm scared of self-driving cars.

Um, clearly I know no company would be putting them out there in the wild if they hadn't crossed the threshold of safety.

Uh, but let's say we a decade from now are in a world where self-driving cars are the norm, self-driving 18-wheeler trucks are moving cargo on the highways.

What are the main vulnerabilities of such systems short of an electromagnetic pulse that some enemy nation kicks out?

What— are we aware of what the areas of most likely failure may be and where the dangers are?

Let me answer that.

But first, let me say something about the state of the art on the safety of the technology behind self-driving vehicles.

And so right now, the technology is such that it can be deployed in closed environments and also in environments where the vehicles are not moving too fast and you don't have too many changes in the environment.

That is because the sensors and the algorithms that perceive what is safe in the world require a certain amount of response time.

And we have to match the application and the environment in which we deploy against what the technology is capable of doing.

So deploying self-driving trucks off roads, off public roads, in ports, in rail yards, in campuses, is completely safe.

And it's something that we already have companies that are doing.

For example, our company, Venti Technologies, is automating the Port of Singapore with self-driving trucks that interact with human-driven trucks.

And there are other areas where we are deploying the same kind of technology off public roads.

Because the vehicles can move slowly, and the environments are simpler than, let's say, Boston traffic, where driving is so challenging.

Now, the other piece is the environment has to be very carefully mapped.

And this is how the companies that are deploying self-driving taxis operate.

They go inside the region, which we call closed environment, and they map every feature in that region.

So that the vehicles are able to create very detailed maps in sensor space.

Not the kind of Google Maps that we use, but the kind of laser point cloud maps that are very detailed and that actually helps the vehicles navigate.

Now, we don't see a lot of self-driving vehicles in Boston because here it snows and it rains and the sensors have trouble with that.

It's difficult to see the road when it's covered with snow.

Well, let me ask a question about that.

So, you know, I've attended these robotics competitions for K-12 children, and one of the common contests they have is called Follow the Line.

Exactly.

Where the robot has a— so is it the case that we may re-engineer our environment, the roads and such, in order to provide data for these types of autonomous vehicles to keep them safer?

So there's lines on the road that give different messages or you know, like for example, a rumble strip.

A rumble strip to me is something that you can encode binary data into if you so chose, right?

So is there a way to encode data into the environment that— but again, if you do that, it could be modified by a bad actor.

So yeah.

For sure.

So some approaches involve doing exactly that, embedding markers in the environment.

Some other efforts are about inventing new algorithms and new sensors.

For example, with snow, we can use a completely different technology from what is used today.

We call it ground penetrating radar.

Oh, wow.

Where the car can look at the texture of the ground underneath the road and localize, figure out where it is according to the features there.

That is nuts.

So this is a piece of work we have done with our team at MIT.

Wow.

But let's come back to your question about what can go wrong.

Right.

Because a lot can go wrong when you use AI.

And we have to kind of separate where we operate with AI.

Is it a solution that operates completely in the digital space, or is it a solution that operates in the digital space and in the physical space?

And the problems with the physical solutions incorporate everything that can be wrong in the digital space.

But now there are many other, other ways in which the supervillains can have negative impacts.

And so, for instance, using Wi-Fi, a supervillain could swap the controls in your car.

You think you're turning left, but in fact you're accelerating.

You think you're braking, but in fact you're turning right.

And so these kinds of attacks have already been demonstrated at the research level.

When we started our research project about autonomous driving, this was in 2015, the first thing we did was we convened a meeting that was labeled "Autonomous Cars: What Can Go Wrong?"

And we brought together anybody in the community who wanted to brainstorm, so we brought the engineers, but also philosophers and science fiction writers and ethicists.

And we brainstormed all the different ways in which a physical instantiation of AI could cause harm.

And this is actually important to do because it gives us a sense of what kind of guardrails we need to build into the technology as we develop the technology.

There are enormous benefits we can get from using AI in the digital world and using AI in the physical world.

But we have to be aware of what could go wrong, and we have to put in guardrails for the problems that can arise.

Now, some problems we can name, and we can come up with technical solutions for them.

Other problems are more complicated, and then we need to have societal compacts and laws about how to support the good and prevent the bad.

Within the digital space, there are so many ways in which the supervillains can take advantage of people.

So one example is that right now with AI, the supervillains can create automatically hundreds of phishing attacks every minute without an army of programmers.

And so another important aspect of dealing with the negative side of the benefits of these technologies is educating the population, because everyone needs to know something.

Not everyone needs to be the kind of geeky AI researcher that we at MIT are, but everyone needs to understand at a high level how the technology works, what it can do for you, what it can do, what it cannot do, what you should pay attention to so that you're part of a solution, of a comprehensive solution that starts with technical solutions where they're possible, but also continues with educated consumers and with societal norms, compacts, and laws.

Wow.

So if you're a young person right now, or just a person that's an innovator, so we haven't tapped out the current AI models, but people are building things on top of the AIs that are out there.

So with these liquid models, if I was an innovator, should I be like, hey, I need to get on liquid models and start looking how to build on top of that because that's the wave of the future.

Is that how one should be thinking?

Absolutely.

And in fact, the models are actually open source.

We get about a million downloads a week.

So a lot of people have began to use these models on their devices and there's so many creative solutions that people are building.

So I'm just so excited about the economic opportunities that this can create.

That is amazing.

I didn't know that.

So guess what I'm going to be doing this weekend?

I'm downloading my Liquid model.

Right.

So when you talk about value moving from the cloud to the device, what does that actually mean?

Well, right now to take full advantage of a large model that runs in the cloud, you have to put the data in the cloud and you have to fine-tune or train the model onto your data.

The model still sits in the cloud.

Then to interact with a model, you actually need to have a connection to the cloud.

That connection could be broken into, someone could listen in, so there is a security challenge.

When your data is in someone else's environment, there could be privacy breaches.

And so, if you have your data on-prem, then you avoid privacy concerns and you avoid the security concerns associated with accessing the cloud.

Furthermore, the cost of running your models on device is much, much lower than the cost of running a huge model in the cloud.

And this has economic impact, but it also has environmental impact.

Right now, the carbon footprint of AI is very high.

The water required to cool the AI machines, is very high.

And so if we take the problems that don't need the cloud-size solution and we run them on device at a lower energy cost and a lower cooling cost, we help our Mother Earth, we help our planet.

So I think that the future is very exciting, and the future will be a kind of a hybrid future where some problems will still require the industrial-scale installations, But not all the problems will require that.

And so for the problems that could be done in more energy-efficient ways, these on-device solutions will be powerful both from an economic point of view and from an environmental point of view.

Yes, yes.

It sounds like when, you know, the internet came about, a lot of us at the beginning of it couldn't see what it was going to become, right?

You know, it's like when Amazon was a bookstore, you didn't imagine it becoming, you know, what it became during the pandemic.

I see that, you know, this is one cutting edge.

Another thing that MIT is very famous for at the cutting edge is robotics.

And so this whole idea of intelligent robotics that are humanoid, right?

'Cause now we see robots with facial expressions and these sort of things.

Is that gonna be our world soon?

Is that really happening?

Or, you know, is it still sci-fi?

That is a very exciting question because it points to a new world where we have physical intelligence on many devices in our world, including on robots.

And I'm so excited about where that world is going.

Well, MIT is famous for more than just your institution of CSAIL, which does artificial intelligence.

Robotics is another big area where MIT excels.

Science fiction makes that marriage all the time of robots and AI.

And when I look at the robotic technology, it's becoming, you know, more and more advanced, just like AI.

They're humanoids and they even have facial expressions.

Is that really going to be our future?

Absolutely, Hakeem.

And it's going to be a very exciting future.

But I have to say, CSAIL also has a lot of robotics, although MIT as a whole has a lot of robotics.

Yes.

But some of it is at CSAIL.

So that is our future.

Well, you mentioned how AI is coming into all our devices.

If I have AI in my toaster, it's still going to look like a toaster.

But if I have AI in a humanoid robot, you know, just like there are, uh, fakes, digital fakes with AI in video, it's almost like you're going to be able to fake humans.

Not just yet, Hakeem.

Not just yet.

So, uh, let me, uh, explain where we are.

Yes.

With AI and robotics.

Right now in the field of computer science, AI and robotics are two largely different fields.

AI is amazing us with its decision-making and learning, but it's relegated in the digital world.

It works primarily inside our computers.

Robots, on the other hand, have a physical presence and the ability to execute pre-programmed tasks but they're mostly not intelligent.

Right.

Well, this separation is beginning to change.

And this is something that I am personally very interested in.

My research aims to bridge the separation, and many of my colleagues are working towards the same objectives.

And so the objective is to bring AI's ability to understand text, images, and other online information to make physical machines intelligent.

And if we achieve this, then AI will not just reside in our computers, but it will drive, it will walk, it will fly, it will interact with us in the physical world.

And this is the world of physical AI.

And will it be babysitters and nannies and teachers?

Possibly in the future, but not just yet.

You asked about humanoids.

And I want to say that the field of humanoid robotics is very exciting.

For the first time in decades, we have humanoid platforms that are affordable and that are controllable and we can use.

And the research community is using these humanoids broadly around the world to study how to control them, to study what exactly they can do.

Humanoids are very complex mechanisms that are difficult to control, and today's AI is not so good at making the humanoids really capture the kind of fine interaction with the world that is needed.

The traditional AI solutions we're deploying do not understand physics, but humanoids operate in the physical world and they have to understand physics.

And so we also have a great need for humanoids to understand how humans interact with the world.

There is so much common sense that is not codified in the humanoids.

And that doesn't exist yet.

It doesn't exist inside our AI models.

So as we try to bring AI and humanoids together, we need to make AI better.

Got it.

We need to have AI that runs on device because the humanoid cannot wait for the cloud to tell it what to do.

We also need to get AI to understand physics and common sense reasoning.

Let me give you some examples.

I was at a meeting a few months ago and there was a demonstration of a humanoid.

The humanoid was covered in clothing and it was in a small demonstration space with a shelf and a fake plant and the watering can.

And so I asked the humanoid, hey, what can you do?

And the answer was, I help humans in the home.

And then— Wait, the robot said that to you?

That robot said that to me.

Wow.

And then I saw the watering can and the plant and I said, can you water that plant over there?

And the humanoid slowly delivered the task, did it well.

It took a while, but it completed the task.

And then the humanoid came to face me for the next instruction.

And I said, "Now, can you water my friend over here?"

And the human reached for the watering can and was ready to put water on my friend's shoes.

So this is an example of common sense that has to be part of AI in order for those machines to become part of our world.

Now, the other thing to know is that humanoids have been walking and running and falling for decades.

What's happening now is that the mechanisms, the body of the machine, is getting much better at those tasks.

And just recently we heard of great successful results from the humanoid half marathon where the robots were able to run faster than people.

Wait a minute, they completed a half marathon?

Yes, they did in China.

Like, my instinctive thinking is they wouldn't have a battery that could last that long.

Yeah, but think about it.

So the surface on which they ran was very smooth, so it was not ups and downs.

There were no obstacles.

The movement of the robot had to be very simple.

You basically cycle the same movement that puts one foot in front of the other.

And so the actual running of that marathon was not as complex computationally as what you would find on a winding course.

It was still an extraordinary feat, but the feat was really partly enabled by the fact that we have hardware.

So it's, it's very interesting to think about body morphology and how it connects and combines with a brain.

And the purpose of physical AI is to bring the two together, to develop AI that can run on the body of the machine and that can control that body in a way that is effective.

And for humanoids, we haven't quite figured out what that brain ought to be.

We need to figure out efficient solutions that can run on the humanoid.

We need to figure out how to put physics in there.

We need to figure out how to put common sense.

We need to figure out how to develop solutions that do not make mistakes.

Now, it's in the space.

It's actually kind of interesting because there is a question of what should the body do and what should the brain do?

Yeah.

And this in our field we call embodied intelligence.

The intelligence achieved through body, through the hardware, and then physical AI is the intelligence achieved through the brain.

Let me give you an example.

Wait, before you go further, let's just nail that down for our audience members.

So embodied intelligence is a new phrase, right?

Embodied intelligence is about using the hardware or the body of the machine to offset some of the computation.

Physical AI is using AI's ability to understand text, images, and other online information to make the machines more intelligent.

So you can talk to the machine, so that you can use concepts when you describe what you want the machine to do.

In a traditional robotic solution, if you want to task the robots to pick up this cup, you have to process the pixels and the XYZ coordinates.

You cannot task the machine in terms of concepts, right?

All the tasking is in terms of very low-level constructs.

With physical AI, you can bring language in, and language brings you concepts and abstractions, and that allows us to reason about the world at the level that is more consistent with how humans do it.

Than how machines do it.

Because a machine does not understand cup.

Machine does not understand phone.

The machine understands numbers.

So everything we task a machine to do is represented in numbers.

This physical AI layer allows us to put a higher-level construct on top of that number processing.

Yeah, yeah.

So what else are you working on that, you know, what do we want?

That you can tell us about.

So within MIT CSAIL, our objective is to advance knowledge.

Right.

And our work is published, so it's always in the open source.

We are focused on inventing the future.

We're focused on understanding how to make today's methods better, inventing new methods, and enabling applications using the new algorithms, AI models and other computational solutions that we create.

So that's our objective at CSAIL.

Some of the projects within my research group have to do with building new types of AI models for humanoids.

So we are looking at how do you capture the forces and the torques that are needed as part of the interaction with the physical world to really train machines to do more complex tasks than we can specify from first principles.

And so we have— Through them learning on their own?

Yeah, yeah, yeah.

So we set up a kitchen environment where we instrumented people and we record muscle activity, gaze, pose, and lots of other information, touch.

And from this data, we can teach robots to do tasks less mechanical and more smoothly the way humans do.

And we can teach robots to do tasks that we cannot specify from first principles.

So more complex tasks than is possible to write the formula for or develop an algorithm bottom-up from.

And so as part of this, we have to address a very fundamental question about AI.

How can we create visual language action models that embody the conceptual abstractions through language, the perception which is needed for the robots to understand the world, and the action which is about moving in the world?

And an important point here is that many people are really looking at how do we train robots because we don't have as much data about physical action as we have about language, about text.

We have a lot of videos, so there is a line of work where people are looking at video data to try to build action models.

And some tasks can be trained this way, but the world requires interactions.

And so anything that involves physical movement has to be treated with great care.

Let me give you an example.

Right.

Imagine dragging a carry-on suitcase up a hill and what kind of force and torque you have to put with your hand when you, when you move your suitcase up the hill and how that compares with moving that suitcase on a smooth floor.

Right.

Well, if all you have is video, there's no way you're going to understand the difference between the forces and torques in one example versus the other example.

That's right.

So in our research project, we are capturing that force-torque element through different approaches to collecting data and training our models.

This is one line of work.

Another line of work has to do with building soft robots.

So really thinking about embodied intelligence at the level of how do you build skin for machines, how do you build better sensors, how you use haptics as a way of collecting feedback about the world, how do you invent new materials for making such devices.

We recently built a sea turtle that swims like a sea turtle.

And we have a wide range of other machines that we are building.

And so this is part of an effort to address the embodied intelligence aspect.

Here we are interested in how nature achieves intelligence in species like the octopus, where the intelligence is fully distributed.

The octopus is such an intelligent creature.

And so how does that work?

How do you actually have the brain all over the body?

Right.

And how do you coordinate all these different components?

So we have a research project that studies that problem.

You know what that reminds me of is an idea from science fiction a lot is you have these swarms of nanobots or microbots and, you know, that goes into potentially medicine where you could like have nanobots, you know, finding and destroying cancer cells before they become, you know, malignant tumors.

Is that real?

Is that really gonna happen?

Are we gonna have, you know, supervillains with swarms of nanobots like in the movie Big Hero?

So I don't have the swarm of nanobots yet.

I have swarms of drones.

I have swarms of boats.

I have swarms of other types of robots.

But I do have an ingestible origami mini surgeon.

So this is a— Wait, hold on, hold on.

You can't just say that and then just go into the explanation.

We got to like, you did what?

What?

So we built a little pill-sized robot and you could swallow it.

The robot is an origami robot.

So it unfolds?

So it's actually like an accordion.

So you kind of, you squeeze it together, you put it in ice.

And you encapsulate it in an ice pill-shaped structure.

You swallow it, and when it gets to your stomach, the ice melts and the robot unfolds.

Now, the robot is made out of sausage casing, and it has a little magnet embedded in it.

So you could use an external magnetic field to control where it goes and to really understand its location.

Wow.

And then the concept is that you could use this robot to remove a foreign object.

Like, you know, believe it or not, many people swallow button batteries.

And right now you need actual surgery to remove them.

But you could also collect a tissue sample or you can patch a wound or you can deliver medicine inside your stomach at the precise location.

And when you're done with your task, the robot could either come out through the digestive system or in some cases the robot could just sit there and get digested because it's made out of sausage casing.

And so this is not deployed in hospitals yet.

Yeah.

But it's a concept for a future of surgery without incisions, without pain, and without risk of infection.

And think about trauma care, right?

If you find someone that has suffered a physical trauma, and the, you know, just like when the first responders show up, if somebody's had an opiate overdose, they hit them with Narcan really fast, boom, to save their life.

You know, one of the ways that people often lose their lives is by bleeding out, right?

And so you could, you could address that both from the inside.

Here, swallow this ice pill, right?

And then you go in there and create, you know, fix this bleed spots.

And yeah.

So how do you come up with these ideas?

Are you guys, You just wake up in the middle of the night like, oh, here's an application, or, you know, how does this— are you asked, you know, 'cause quite often we're asked, the government or an agency, an entity says, we need someone to develop this, do it for us and we'll give you this amount of money.

Do you guys, or if you think about how movies and TV shows are made, quite often a network or a studio will say, give us this, but at the same time, you have production companies that are pitching to them.

So do you guys come up with your own ideas and pitch, or are they from top down, bottom up?

How do you come up with this stuff?

We have the best students in the world, and we aim to do things that nobody has done before.

So it's a kind of an interesting process to come up with the next challenging problems.

We aim to develop the science and engineering of intelligence.

And so everything just kind of comes together to ask what should the machines of the future look like from a body point of view?

What should the brain look like?

And how should they interact with people?

And what kind of applications all these different capabilities enable.

That's the kind of question that we ask in my research group.

And the problems that we end up solving are often problems driven from within, from these aspirations to do something new.

I guess we are often inspired by science fiction books and movies because these were the superheroes of our childhood.

Like, you know, back in the day when I was a child, I used to read a lot of Jules Verne books.

Yeah, Jules Verne.

We didn't have Harry Potter back then.

And I really enjoyed the 20,000 Leagues Under the Sea novel.

And so when I joined MIT, I met a student who also read the Jules Verne books and enjoyed this notion of underwater world.

And so together we decided to build our own version of the underwater explorer.

And so that was how some of that work came to be.

So there is serendipity in how some of the topics get picked according to interests within MIT or also according to interests between our world and other people's worlds.

Yeah, so we are— what we're seeing is that robots are becoming ever more capable.

AI is becoming ever more capable.

So what is the world of tomorrow Where does that leave us humans?

Are there dangers?

Are there advantages to this?

What is the future of this constant progression to greater capability of robots and AI?

Think of robots and AI as tools.

They're tools.

And just like any other tool, they're not intrinsically good or bad.

They are what we choose to do with them.

And we can choose to do extraordinary things.

With AI, we can give everyone superpowers.

We can get speed, knowledge, insight, creativity, foresight, mastery, even empathy.

All of this is possible with the support from AI.

With robots, we give people physical superpowers.

We can extend our reach.

We can amplify our strength.

We can refine our precision.

We can develop ways to interact with the world that we've only dreamed about and we read in storybooks.

So this is an extraordinary potential future that we are building.

Of course, as we're building this future, we have to think about how the same superpowers that empower people to do good things can also empower the supervillains to do bad things.

And that means we have to think about how we develop technology.

We have to put guardrails at the very beginning.

We have to think about which problems have technological solutions, which problems require social compacts and new laws or adjustments of the existing laws.

And how do we actually prepare the public for a world where machines can support everyone?

Where machines can take on some of the low-level tasks that we all do to free us to spend time on more interesting activities.

Wow.

One of the concerns that a lot of people have about AI that is in the media a lot is its impact on jobs.

And given the fact that I have a son who's about to complete a computer science degree, I certainly want to get him earning money.

What do you see as the future of jobs and AI?

So there's both the jobs that are going to be created, but also there are things, you know, particular tasks that humans do that are going to be replaced.

What's your input on that?

What I say to people is that it's unlikely you will lose your job to an AI, but you might lose your job to someone who knows how to do AI to be more productive.

So a key lesson here is to get educated and learn about the new tools and figure out how to apply them.

And this doesn't mean you have to be an AI geek, but it means you have to understand what the technology is, what it can and cannot do, what it should and shouldn't do, and you have to know how to apply the tool correctly.

The other thing I want to observe is that it's so much easier for us to imagine what will go away than to imagine what will come.

We're much better at imagining what will go away than what will come.

And in fact, if you look at the job landscape today, there are so many jobs that didn't exist 20 years ago.

Right.

In 2007, 3 critical technologies were introduced: the smartphone, cloud computing, and social media.

These sectors employ so many people and generated so many jobs, and nobody was talking about these jobs in 2000 when the dot-com boom was in full power.

And so I think every new technology enlarges the job field rather than shrinks it.

And I expect that there will be some extraordinary new opportunities for the future.

So I think that there is going to be a change in the job market.

Right.

And the current generation that is in the job market will have to adapt very quickly.

I think our children will be AI natives.

Right.

And the story will be very different from them.

But I really do imagine a very positive future with humans and AI working together to bring value to the individual and to society.

Okay, so I said AI, and, you know, I feel like AI is one of those things, like when people say the Big Bang, right?

It conjures up an idea in people's minds, but the average person isn't an expert in it.

So whatever it's conjuring might not necessarily be accurate.

So as easily as you can, for the general listener, define what AI means to you.

Let me do that by telling you a little bit of the AI history.

Oh yes, I love that.

Which started in 1956 when a young Marvin Minsky was a professor at MIT.

And one summer he decided to gather his friends, his geeky friends, and they headed to the woods of New Hampshire to Dartmouth College where they spent an entire month hiking and talking and no doubt drinking till late hours of the night and really discussing what science needs, really discussing what is the next important academic field of study.

I want to remark that in 1956, the world looked very different than it does today, especially in technology.

There was no email, no mobile phones, no social media, so no distractions.

So these people thought very deeply for a whole month about what they ended up calling artificial intelligence.

Wow.

To denote the science and engineering of making machines with human-like characteristics in how they move in the world, how they play games, how they see the world, and even how they learn.

And the AI community has been on a path to deliver on this aspiration of studying these kinds of machines and building them.

If you think about an idea like that, it's innovative in its time, it doesn't yet exist.

And typically as academics, we have to get funding, which means someone has to recognize the scientific merit of the idea.

So were they able to get funded early on or did they just have to shoestring it to start out?

Well, there was funding and then there was some shoestringing.

What is for sure is that everyone had passion and they laid out a roadmap that we are still working on today because it turns out the challenge was very big.

And since 1956, a lot of new ideas came online and they were studied, but at first, the ideas did not work so well.

And then later on, as computing advanced, as you could do more and more computations in a computer box and as data became available, then all of a sudden ideas that were invented 40 years ago started working.

Yeah.

And today AI is a very rich field that continues to study the science and engineering of machines with human-like characteristics.

What's important to remember is that AI is much more than your chatbots.

So chatbots are examples of AI, but there are so many different paradigms for using AI to interact with machines and to get value for yourself as a person and also for your company.

Well, we always hear about large language models as like the AI algorithm or, you know, software architecture?

Are there competing models?

And is it the case that they're not as useful for us regular folk and they might be in the background somewhere or they're still in the research lab?

What's, what's out there?

Actually, Hakeem, there are several different important AI paradigms, not just the generative AI paradigm, which is behind the large language models and most practical solutions for important real problems.

Involve a combination of these paradigms.

So let me explain, if you'll indulge me.

Yes, ma'am.

So the paradigms that are most dominant today are deciding, optimizing, classifying and predicting, generating, and then there's the agentic AI paradigm.

The deciding paradigm had the first mega successes.

So deciding, the deciding paradigm is what enabled Deep Blue to beat Garry Kasparov in the 1990s.

Yeah.

And the way this works is you have a problem and you have some rules about how that problem is done.

Maybe you have chess rules or maybe you have rules about how your company engages with clients.

Anyway, you have a set of rules.

The computer is then able to create many possible futures and in the end you pick the one that suits you best.

And so in fact, this is how Deep Blue worked.

It imagined the chessboard with many different combinations of moves and eventually picked the best.

I just have to add that that is exactly how Doctor Strange figured out how to destroy Thanos and defeat Thanos, but I don't know if you're into that.

Please continue.

So that's deciding.

We also have the optimizing paradigm.

And the optimizing, you can think of it as trial and error.

You have some policy and you get points that are either positive points towards your policy or negative points.

So concretely, let's imagine you're trying to get a car to steer on a racetrack as fast as possible.

And you don't know anything about the task.

You put the car on the track and you let it try.

I'm gonna steer this way, I'm gonna accelerate in this way.

The car repeats trying to steer one way, accelerate another way, and at first the car will not be able to stay on the track, but over time it will figure out how to drive, how to steer and accelerate.

So this is the trial and error optimization approach behind what is called reinforcement learning.

So this is essentially the paradigm behind reinforcement learning.

You start in simulation, you simulate your problem, and you create a policy that drives the agent towards the movements or the actions that lead to the best outcome.

Right.

Okay, so that's reinforcement learning.

We also have prediction and classification.

Right.

And prediction and classification are about training a large model consisting of artificial neurons and artificial synapses.

Think of that as a big graph using labeled data that humans label, where the model figures out the parameters that go inside the neurons and on the synapses so that when the model is used with new data, the model can classify that data correctly.

So let's say we're trying to teach a model how to recognize cars.

You can give the model a lot of pictures of various brands of cars.

If a new brand gets introduced, the model will still be able to recognize that new brand as being a car, even though it was not part of the training dataset.

And the way this works is purely statistical.

So the way this works is by comparing the pixels in the image of the car that you recognize against pixels in other images that humans said were cars.

Right.

The model doesn't know what a car is.

Right.

It doesn't know the semantics of a car.

It just knows that this picture has the same pixel structures as other images that people said were cars.

So the model doesn't know that you hop in the car to drive to New York.

Right.

Or you can make the car warm or cold, or you can play your favorite music when you ride the car.

So these are all aspects that we kind of naturally understand about what it means to be a car.

But the AI systems do not know that.

Okay, so we talked about deciding, optimizing, and predicting.

Generating is like your chatbot.

So generating essentially trains models to create entirely new data that was not seen inside the training set.

And so you can think of generating as a kind of an inverse of predicting.

So in a predictive model, You show the model the image of a car and you ask the model to say, this is a car.

In a generating situation, you can say, "I want a car," using words, and you can expect the model to give you a picture of a car.

Yes.

The picture is static.

In an agentic AI system, instead of getting a static solution, you get dynamic projects.

So you might say, tell me how to build a car.

And then the system will launch all kinds of computations.

Think of them as assistants that scour the internet and scour your model for what is available with respect to designing a body, costs, physics analysis, and ultimately brings you a project for how to do it.

So in summary, deciding is about rules to build possible futures.

Reinforcement learning is about running in simulation through trial and error to figure out the best way for the future, the best way to move or to act in the future.

Predicting is about using labeled data so that you can recognize future data items that were not part of your training set.

Generating is again training on existing labeled data with the purpose of generating entirely new data.

And agentic AI is about creating not static answers, but entire projects that help you through a complex process.

Wow.

I feel like I just got a bachelor's degree for free.

That was more informative than all of my studies on my own.

So what else might exist like in the, in the behind the scenes?

Are there other ways that instead of taking in all language and creating a statistical prediction, are there other types of non-language-based AIs that exist?

So we can do generative AI solutions on any kind of data.

So we can use video, we can use photos, we can use molecules.

We can use sensor data, we can use financial data, we can use weather data.

Any kind of dataset can be mapped onto a kind of a generative model that will train on past occurrences to spit out new ideas and to maybe create digital twins of whatever application it is that you would like to model with a computer.

Wow.

You just sent my mind down a pathway because, you know, there are certain things like minerals, right?

You know, the flashy science like AI and science gets all the headlines.

But, you know, things that have to do with like what we call condensed matter physics, they're always doing something new and they're finding like in asteroids and things, minerals that can't form on Earth or minerals that can only form over long periods of time.

So when you said that, what I imagined about molecules is that you can now explore the space of all sort of like molecule arrangements to figure out new materials.

And can you make it where you can say, here's the final outcome I want.

I want something with this particular property.

AI, go and tell me how to put molecules together to make that happen.

Absolutely.

In fact, we are already doing that.

We are also generating new drugs that help support disease.

We're applying these kinds of techniques to generate mechanisms, different kinds of robot configurations.

So the application space is really vast.

And interesting you mentioned physics, because one application I want to share is the astrophysics application in understanding the beginning of time.

So in fact, we are collaborating with physicists to use some of our AI models that are particularly good at understanding physics, understanding what happens over very long sequences of time to detect gravitational waves and to really begin to have a deeper understanding of how the space-time universe has evolved.

And that's so exciting because you see, the space is interesting.

We are developing core AI algorithms and models and solutions.

We are analyzing them to understand what they're good for.

And then we're applying them in so many different domains.

And today, any domain that has data can benefit.

Wow.

I have learned so much from you today.

I hope that our— I'm sure our listeners have also.

I want to thank you so much, ma'am.

This has been so exciting and awesome, Ms.

Daniela.

Thank you so much.

Yeah, yeah.

So the next time, if I ever visit CSAIL, I've visited the MIT Media Lab a few times.

If I ever visit CSAIL, I'm going to walk through the door saying, where is she?

I want you to be my tour guide.

Well, let's change ever to when.

So when you visit CSAIL, I would love to be your tour guide.

Thank you, ma'am.

Thank you so much.

This is wonderful.