色中色

Ruud van Sloun is the kind of scientist who enjoys working on entirely new ideas that no one else has touched yet. If that fresh idea is promising but no longer novel, he鈥檚 happy to hand it off to others better suited to develop it further, freeing him up to dive into a new challenge.

So around Christmas 2021, Van Sloun had taken a few weeks off to mess around with ideas freely, and ended up plopped in the middle of his living room. The floor around him was covered with dozens of neuroscience papers.

Van Sloun is an associate professor in the Signal Processing Systems group of the Department of Electrical Engineering. Usually, he focuses on algorithms for medical imaging, with ultrasound being a particular source of inspiration.

But not during that Christmas break. Van Sloun briefly stepped outside his field to take a deep dive into the human brain. He used his time off to learn everything about how the brain makes predictions, processes information, and makes decisions.

Medical imaging

鈥淐an鈥檛 we use these principles for medical imaging, too?鈥� Van Sloun wondered out loud, tossing another paper onto the last bare spot on the floor.

Medical ultrasound could become much better and more accessible that way. Van Sloun dreams of portable, affordable ultrasound devices that deliver excellent image quality 鈥攐ne as small as a smartphone that takes images so you can confidently and reliably make medical decision with them, for every patient.

Flashlight looking for keys

鈥淚magine you鈥檝e lost your keys,鈥� Van Sloun explains the idea behind his thinking. 鈥淵ou walk through a pitch-dark room with a flashlight, shining it on the spots where your keys might be hiding. Something glints in the beam. You quickly move the light back. Gotcha.鈥�

Based on the information (something glinting in the light), your brain just figured out the best next move (shining the flashlight back there). That way, you found your keys quickly without wasting energy.

Your brain predicts the future

Van Sloun explains that this idea is known as active inference in neuroscience. 鈥淵our brain constantly makes predictions about what will happen next and then does everything possible to bring those predictions to life. Without active inference, searching for your keys would look very different.鈥�

For instance, instead of focusing on where something might be, you鈥檇 start at the top left corner of the room and painstakingly sweep the flashlight down centimeter by centimeter. Then, you鈥檇 move to the right and slowly scan upwards again. Only after you鈥檝e scanned the entire room would you finally notice where the keys are.

Reflecting and creating pixels

鈥淭hat鈥檚 exactly how an ultrasound image is formed,鈥� Van Sloun explains. 鈥淭he device sends sound waves, which the body鈥檚 tissues reflect. Then, the software builds the image pixel by pixel. What the machine actually 鈥榮ees鈥� doesn鈥檛 matter 鈥� the program treats every pixel in every frame as equally important.鈥�

Not only does this method waste a lot of energy on measurements that aren鈥檛 informative, but the device also picks up a lot of noise and artifacts from tissues you don鈥檛 want to examine.

AI growth spurt

鈥淚 thought it would be cool to apply active inference to ultrasound,鈥� says Van Sloun. But that鈥檚 easier said than done. All the pixels in an ultrasound add up to an enormous number of parameters and dimensions鈥攎ore zeros than you鈥檇 want to deal with. 鈥淵ou need exact statistical models, and that鈥檚 hard in such high dimensions.鈥�

Then generative AI made a giant leap forward. With a clever algorithm and a precise statistical model, it can create something entirely new out of nothing. ChatGPT uses a language model for this, and there are video models for images. 鈥淭hat鈥檚 exactly what we needed,鈥� Van Sloun says. 鈥淪uddenly, the hopeless problem wasn鈥檛 so hopeless anymore.鈥�

Right there in the middle of his living room floor, everything clicked. Generating ultrasounds with smart algorithms that make the images more efficient, better, and more accessible. Van Sloun went for it. Ambitious? Yes. A bit crazy? By his admission, definitely. Possible? Yes鈥攁t least maybe.

He returned to his role as an electrical engineer, wrote up his plan, and submitted it to the European Research Council. And he succeeded鈥攁t the end of 2022, he was awarded an ERC grant. Their message: 鈥淕o for it.鈥� 鈥淚t was a bold idea鈥攈igh risk, high gain,鈥� Van Sloun says.

Shining more efficiently

Van Sloun and a team of about ten students and researchers have been working on his plan for over two years now. He shows a video to illustrate the project. The smart ultrasound device sends a sound wave, shown in orange, directed at the heart. The tissue there reflects the wave, and the device picks it up again.

That data goes into the algorithm, which then assesses how valuable the information is. Fewer sound waves are sent there if there鈥檚 not much to gain from that spot. However, the device collects more measurements if something interesting happens, like movement.

Just like the flashlight

鈥淚t works like the flashlight in the dark room,鈥� Van Sloun explains. 鈥淚nstead of looking around randomly, the device sends sound waves to the spots where the most information is likely to be.鈥�

His method is more efficient than a traditional ultrasound. It requires less power to produce an image of similar quality. This could be a game-changer for wireless ultrasound devices, which are growing in popularity. They鈥檙e convenient, but you have to recharge the battery every hour, says Van Sloun. With his technology, that battery would last much longer.

Focusing on details

You can also use full power to capture an ultrasound with more detail. That鈥檚 what Van Sloun is currently focusing on in his research. 鈥淚n the image below, you see a heart,鈥� he explains. The heart valves at the bottom are closed. 鈥淭he image on the left is similar to a standard ultrasound. The right image has been enhanced by generative AI, which removes noise and reveals much more detail.鈥�

鈥淗ealthcare providers are often pleasantly surprised when I show them these images,鈥� Van Sloun continues. Cardiologists and sonographers have told him they get a good ultrasound image in only about one out of ten patients. Around seventy percent of ultrasounds are of moderate quality, and the image is even clinically unusable for ten to twenty percent.

鈥淭his problem is only getting worse,鈥� the researcher says. 鈥淣ot because ultrasound technology is getting worse, but because the human body is changing.鈥� People generally have more fatty tissue than before. The more tissue an ultrasound wave has to pass through before reaching, in this case, the heart, the noisier and more out-of-focus it is. This is also a problem for pregnant women.

If an ultrasound is not clear enough to make a good diagnosis, the patient is referred for an MRI scan. This is inconvenient for the patient and expensive for healthcare, says Van Sloun. Better ultrasounds make care more accessible and inclusive. 鈥淭hat means a huge win for reducing pressure on the entire healthcare system.鈥�

Ultrasounds on your smartphone

It鈥檚 a promising prospect, but Van Sloun likes to dream even bigger鈥攍iterally. 鈥淢y passion is smart ultrasound technology, because it can bring medical imaging to many people.鈥�
According to the World Health Organization (WHO), two-thirds of the world鈥檚 population has no access to ultrasound or any other form of medical imaging. When it comes to high-quality imaging like MRI, that number rises to ninety percent.

鈥淭he differences between countries are striking,鈥� says Van Sloun. In Germany, there are 450 MRI scanners for every 13 million people. For those 13 million people, there's just 1 MRI scanner in Africa, and some countries don鈥檛 even have any MRI scanners. 鈥淲orking on MRI technology is rewarding, and I enjoy it, but no matter how advanced the technique gets, nine out of ten people simply don鈥檛 have access to it.鈥�

Combining methods

The key to realizing his vision is merging different techniques to create more efficient measurements and higher-quality ultrasounds. If successful, reliable diagnoses could be made with only a smartphone, a small ultrasound device, a cable, and an internet connection. Today, such imaging still requires bulky, expensive equipment.

The second key is cloud computing, where all the data is processed鈥攕omething a smartphone alone can鈥檛 handle. Van Sloun鈥檚 algorithms demand significant computing power, and he beams at the thought of making this a reality: high-end ultrasounds right on your smartphone. 鈥淭hat way, we bring medical imaging to millions who lack access and miss out on vital care.鈥�

Musical associations

Van Sloun loves algorithm development because it allows him to be creative. He compares it to writing songs鈥攁 hobby he enjoyed as a guitarist in various bands. 鈥淎 song starts with a basic idea, like a few chords and a key. You build on that, adding and tweaking until you get the desired effect.鈥�

In music, that effect is often emotional; in algorithms, it鈥檚 usually functional, solving a problem. Van Sloun explains, 鈥淔or the creative, iterative process, the exact effect doesn鈥檛 matter. There鈥檚 no fixed step-by-step plan in both cases鈥攋ust brainstorming and connecting associative ideas.鈥�

Working with Massimo Mischi

Ultrasound became part of Van Sloun鈥檚 journey when he learned about the work of Professor Massimo Mischi. While Van Sloun was studying in Eindhoven, Mischi was developing ultrasound techniques to detect prostate cancer, reducing the need for MRIs. 鈥淚 thought that was cool. I said to myself: I want to work with this guy.鈥�

And that鈥檚 precisely what happened. Van Sloun did his master鈥檚 project in Mischi鈥檚 group when Mischi was still a university lecturer. Not long after, Mischi invited him to pursue a PhD under his supervision. 鈥淣ow, years later, I find myself in pretty much the same place Massimo was back then.鈥�

Is he a role model now? 鈥淚 hate to answer that question. But I hope so. I hope I can inspire others the way Massimo inspired me.鈥�

The signs are promising. When asked what he鈥檚 most proud of, Van Sloun immediately points to his team. 鈥淲orking with people who want to change the world is a privilege. They do exactly what I hope to give others: the courage to dream big.鈥�