Arizona Horizon Banner

December 14, 2010

Host: Ted Simons

Brain-Computer Interface

  |   Video
  • Researchers at Arizona State University and Phoenix Children's Hospital are working on a brain-computer interface that will allow people to control electronic devices directly with their brain. Stephen Helms Tillery of the ASU School of Biological and Health Systems Engineering will discuss the research on a special Technology and Innovation segment.
  • Stephen Helms Tillery - ASU School of Biological and Health Systems Engineering
Category: Science

View Transcript
Ted Simons: Good evening, and welcome to "Horizon." I'm Ted Simons. Researchers at the Arizona state University in conjunction with Phoenix children's hospital, are working on an interNais will allow for the use much brain signals to control electronics device and artificial limbs. The study is funded by a grant from the Arizona biomedical research commission. I'll talk to the lead researcher, but first, Mike Sauceda tells us more in this edition of Arizona technology and innovation, "Horizon's" multimedia effort that focus oats people, ideas, businesses, and technologies that are shaping Arizona's future.

Narrator: This ball is being controlled directly by the brain of a monkey with a brain implant. It's part of research being conducted by Arizona state University school of biological health systems engineering and Phoenix Children's Hospital. Research on measuring brain waves is taking part at ASU's biodesign institute. ASU student, Remy Wahnoun, is working on the research. He explains how researchers can figure out how to read brain signals.

Remy Wahnoun: Basically in the path that controls the limbs in the brain, you have NEURONS, and those have a direction of movement. When you go to one direction, they will be really excited. When you go to the other one, they will be less excited. So what you do is if you recall from -- you can actually find out where the person is trying to go, and using this we can actually do a movement in treating --

Narrator: Wahnoun says there are two types of brain implants, one that goes into the brain, 1 or 2 millimeters, or another that lies on top of the brain.

Remy Wahnoun: To first one plants in the brain, so we're trying to avoid this because the brain will react to it. And the second sway to use implantation, to show -- they're on the brain, so they don't penetrate the brain. This is pretty safe. They can have it up to a few weeks without any problem. So given this, you could have -- you could use the implants to control your limbs, you could actually use that to remote control -- someone was completely paralyzed will be able to do all of this. And hopefully eventually get full feedback from it. Control your arms and get back feeling from it.

Narrator: The implants are being placed on epileptic children having brain surgery because of their disease. He says the kids learn fast, how to control devices directly with their brains.

Remy Wahnoun: We are -- already have three patients and they got pretty good control of the interface, so we're walking on -- working on making things better, how the brain is working, so this is the thing with the interface, the primary goal is to actually help people who lost mobility, and the secondary goal is to actually understand better how the brain is working.

Ted Simons: Here now to talk about efforts to create a brain computer interface is Stephen Helms-Tillery, an assistant professor at ASU's school of biological and health systems engineering. Good to have you here. Thanks for joining us.

Stephen Helms-Tillery: Thank you, my pleasure.

Ted Simons: Let's define terms here. Brain computer interface. What are we talking about here?

Stephen Helms-Tillery: We're talking about a way to interface the brain, obviously, to an electronic system like a computer. So we have an interface to our body, our brain controls our limbs, through an interface, which interfaces our spinal cords, which projects to our muscles. If you can't control your limb, maybe you need something else to control, like a computer game, or robotic arm. To do that you need to interface to a computer which interfaces with the technology.

Ted Simons: Talk about how long this was developed. I imagine the first step would be to control, you have the interface to the electronic device, the next step would be the electronic device being things that could be used.

Stephen Helms-Tillery: Right. So it's actually a very long development. This comes from work in basic science of how the brain controls movement. It's expanded in 50 years. In that time we've learned what Remy discussed about how newer neurons relate to movements and specifically, and so we learn in the 1980s or so if we had enough cells at one time, enough signals from the brain at once that we could use that to predict or read out what the brain is trying to do with the movement. So once we knew that, then we could take that readout and use it to control any other kind of system. So in the laboratory, in the early part of 2000, we use the signals to control first video game, so -- the virtual reality, and then to control a robotic arm to pick up pieces of food, to feed yourself with, for example. So there's a whole chain of processing and several computers involved and processing those signals, but once we have fast enough computers, we know enough about the brain that we can actually take those signals and do something with them.

Ted Simons: Talk about what's going on in conjunction with Phoenix children's hospital, and using kids so epileptic children that will have to have -- these are kids that will have to have some sort of brain surgery anyway, and they -- the experiments are precursors? How does that work?

Stephen Helms-Tillery: This is a very unique opportunity. These are children who have intractable epilepsy, which is normally treated with medications. They give the kids medications, and at some point it doesn't stop the seizures enough. What is done now is a neurosurgical procedure, often epilepsy caused by sort of an irritation in a very localized part of the brain, so the procedure would be to go in and separate that part of the brain from the rest of the brain, or actually even remove it. But before you do that, you can imagine if you're going to have a piece of your brain separated from the rest, you want to make sure that it's the right piece of brain that gets separated, and that you don't separate other things, like the part of your brain that controls language, or the part that controls movement. So you want to limit accessory damage. So what they do is they place these grids, either on the surface or sometimes deep inside the brain as well, of what's called ECOG, or electrocorticogram – thye are little discs, they're on the surface of the brain and they record the signals. They watch the children when the children have seizures, they can localize where in that grid the seizures originated from, and sort of helps them to guide for the surgery, the treatment surgery. But during that time, the kids sit in a clinical room for a week, 10 days, and they're stuck in this room, because they have to be monitored continuously, they're being videotaped, because seizures can have all kinds of characteristics and they want to catch the seizures, so we have an opportunity to go into that room to say, look, we have a very interesting research problem, which if we have an interface to the brain, what are the characteristics of the signal processing we can do to take signals out of the brain and use that to control a prosthetic system? We talk to the parents and the kids about it, and give them the opportunity to play video games just by thinking about it. So the kids learn, for example, to play a soccer game where they're controlling the motion after soccer game to make it go left and right, but instead of using joystick, they just use signals inside their brain.

Ted Simons: And it works doesn’t it? The kids must be fascinated by this.

Stephen Helms-Tillery: The kids seem to love it so far. We've had three patients, and it's a mix because the kids are not in the best of physical conditions, so -- but they've been very engaged and really enjoyed having this sort of new challenge to occupy their time.

Ted Simons: Can these sensors now, what do they control? Can they control touch, can they control gradients, can they control temperature? Is the sky the limit as far as what these things can do?

Stephen Helms-Tillery: The sky is the limit. So right now what we're doing is we’re taking signals out, using that through something else, but if you want to use your hand to pick up a glass of water, you need to feel the glass. You need to know what you need to feel, how your hand touches it, how hard am I touching it, what’s the temperature, all those things. So the next step in development of these prosthetic systems is going to be to put information back into the brain. So we're also using this environment now to study that. And we're developing tasks where in addition to having the kids control video games, we can maybe stimulate their fingers and look at what the signals look like in the brain.

Ted Simons: This is big news, or at least big possibilities now for victims of stroke, ALS, these things, you can also use this in terms of communication?

Stephen Helms-Tillery: Right. I think in the first patients where this has been applied, these technologies, there have been patients with locked-in syndrome, so these are really sad cases, so this is a person who has, for example, a brain stem stroke, there's a really fascinating French movie called "the Butterfly and the Diving Bell." It's about this. So what happens is people become profoundly paralyzed. The only thing they can control is for eye movements. But otherwise, cognitively they're normal. They have normal as inspirations, and dreams, and thoughts, but they're locked in. In this body they can't control. Those are the first people where implants are actually put into the brain so they can control things. And one of the first things that people in that condition want to be able to do is communicate. So just give them access to like a virtual keyboard on a screen, so they can move the cursor and peck keys on the screen, and write emails. That's sort of the first line of technology development here.

Ted Simons: OK. So we've gotten this far, you've gotten this far, what's next? How much do we still not know about brain signaling, interface and these sorts of things?

Stephen Helms-Tillery: There's a lot that we don't know about how adaptive they are, so how many different kinds of things can you control? If we have an interface to your brain, can you control -- you can control something like an arm, could you control an airplane? Could you control a wheelchair? So what are the limits of what you can learn to do? How many things could you learn to do from a single -- could you control a whole bunch of things at once? And then the big open question, the thing that's really the hard challenge now is, how can we put information back in? What can we use that for? And so we're beginning to develop technologies where we're interfacing multiple brains to computers at once. And to see if one brain can maybe coach another brain in using these sort of implants.

Ted Simons: Wow. The collaboration between ASU and Phoenix Children's Hospital, talk to us about that.

Stephen Helms-Tillery: This is really a super pleasure. I've been working on this sort of working the lab for 10 years at ASU, and when Dr. Addleson came, one of the things he wanted to do very quickly was get involved in research, and especially this kind of research, brain interface. So it's been gratifying to take the technologies we developed in the lab, move them directly into the clinical environment and begin to sort of do that development. And so Dr. Addleson has been amazing in sort of putting these pieces together and allowing these reactions. It's difficult with clinician typically because clinicians are very busy. They have a lot of work that they have to do. And so he's done a very nice job of structuring things so we actually have time to do these interactions and make these experiments happen.

Ted Simons: It's fascinating stuff. And we're so glad to have you on to talk about it. We'll try to get you back on when the next phase comes through. Great work, thanks for joining us, we appreciate it.

Stephen Helms-Tillery: Thank you very much.

Focus on Sustainability: A HORIZON Special

  |   Video
  • Find out how Arizona can benefit from a green economy. We'll take a look at the latest strategies to promote clean energy innovation. And we'll show you how recycled waste water can help the state achieve a sustainable water future. Made possible by ASU Global Institute of Sustainability.
Category: Sustainability

The Science of Magic

  |   Video
  • Magic tricks take advantage of how the human brains pay attention to surroundings. Phoenix Barrow Neurological Institute researchers Stephen Macknik and Susana Martinez-Conde have traveled the world to study how magicians fool the brain and the science behind that. They will talk about their new book based on that research, "Sleights of Mind," as Tony the Magician performs tricks they will analyze.
  • Stephen Macknik - Phoenix Barrow Neurological Institute
  • Susana Martinez-Conde - Phoenix Barrow Neurological Institute
Category: Science

View Transcript
Ted Simons: Vision is one of our most trusted links to the outside world, but you can't always believe everything you see. Magic tricks, for example, take advantage our brain's vulnerability to illusions. Two researchers from Barrow Neurological Institute decided to travel the world and meet with magicians to study why the brain can be fooled by magic tricks. The result of their efforts is a book titled "Sleights of Mind." Here to talk about their findings are Stephen Macknik and Susana Martinez-Conde, along with magician Tony Barnhart, who will perform some magic tricks as the two researchers talk about the science behind the illusions. Good to have you all here. Thanks for joining us.

Susana Martinez-Conde: Thank you.

Ted Simons: Susana why did you write this book?

Susana Martinez-Conde: The book reflects collaboration that we as neuroscientists have been carrying out with magicians for the last five years. And trying to figure out -- magician and neuroscientists, we have a lot of overlapping interests in the understanding of attention and perception and cognition, so we realized that we have a lot to learn from magicians in terms of techniques they use and the conclusions that have arrived. And that has very real implications for everyday life.

Ted Simons: In terms of defining terms, what is cognitive illusion?

Stephen Macknik: Well, so if you think about a visual illusion, would be one of those posters you've seen where something comes out in 3D, or you've seen on the printed page, it's a stationery image but it looks like it's moving. Those would be types of visual illusions, or 3D at the movies when you wear the glasses. Cognitive illusion works on the same principles in the brain, but now instead of operating in the visual system to give you these percepts, they operate in the cognitive parts of the brain, and that allows you to have certain source of cognitive illusions. Difference, you might not see something that's there because you're paying attention to the wrong thing.

Ted Simons: And the idea, Antonio going to do some examples here in a minute, but the idea that we learn how do we learn how the brain works from some of these illusions? These cognitive illusions?

Stephen Macknik: Well, at the Barrow we'll actually study the brain while they actually do various sorts of tasks. And we'll figure out how it is that the brain operates while doing them. And some of these tasks will be things that cause illusions. So, for instance, magic tricks would be a type of cognitive illusion, or we might show certain other source of sequences in which someone has to pay attention very closely to the one thing happening, and they miss something else. The show -- to show, for instance, multitasking doesn't exactly exist.

Ted Simons: Susana, was there a magic moment, if you will, when you realized that these guys hold something that we can learn from?

Susana Martinez-Conde: There was. Steve and I, we were asked to organize the annual conference for the association for the scientific study of consciousness, and we wanted to host it here in Phoenix, but they told us that would be too hot in June, and we should go to Vegas instead to have it there. When the board -- the board wanted to go to Vegas. So we said, OK, we'll do it there. And so we wanted to do something special for our conference, maybe bring some art and science together, something to do with illusions -- as neuroscientists we use illusions a lot to understand what the brain is trying to do. And Vegas basically gave us the answer. We were thinking about how to bring art and science and the public together, and we saw these everywhere, these giant posters of Penn and Teller, and all these great magicians, so we realized that magicians are the masterful manipulators of attention and awareness, and we should be talking to them.

Ted Simons: Let's talk to one right now, master manipulator himself, go by the name Magic Tony?

Tony Barnhart: Magic Tony.

Ted Simons: All right. What are you going to do for us first? What are you going to show us?

Tony Barnhart: I thought since tonight's show is all about science, maybe we should do a quick and dirty experiment. Would you like to play?

Ted Simons: Sure, why not.

Tony Barnhart: Alright. I've got a deck of playing cards. I'm going to run my thumb down the edge of the deck and I want you to tell me to stop anywhere you like.

Ted Simons: Stop.

Tony Barnhart: Right there? All right. You've select add card. I'm going to show it to you, to the camera, I'm not going to look at the monitor. So you had a free choice of any card in the deck. We're going to try a quick experiment. Some people think crystals hold some power. They think that they can magnify telepathic energy. I thought if one crystal has this much power, why don't we try 10,000 crystals, see if we can amp this up. So I spared no expense, and I brought along 10,000 crystal was me this evening. So what I want you to do is concentrate on your card. And I want you to send the thought of the card to the base of the salt shaker, it will shoot up throughout crystals, it will magnified, shoot up in the holes and hit me right in the brain. We could wish for a larger target, sure.

Ted Simons: I'm focusing, I’m concentrating.

Tony Barnhart: You've got it.

Ted Simons: I'm seeing it.

Tony Barnhart: OOOH, OOOH, your card is the color of a cherry. Is this correct? A black cherry.

Ted Simons: There we go. [LAUGHTER]

Tony Barnhart: That was close. It's a club. Is it a club?

Ted Simons: Yes, it is.

Tony Barnhart: In fact is your card the ace of clubs?

Ted Simons: It is the ace of clubs.

Tony Barnhart: So that’s amazing, but what's even more amazing is, if you look through the deck, in fact you will not find an ace of clubs. There's no ace of clubs in this deck of cards because that's the card that I keep under the salt shaker.

Ted Simons: All right. That's fantastic. OK. Now, what did he just do? Obviously Magic Tony has special gifts. But he can't -- what happened here?

Susana Martinez-Conde: What happened is that what he do, when he -- he actually -- he put the card beneath the salt shaker and that was right in front of your field of view, the information was right in your eyes, and you were looking, but you didn't see because you were not paying attention. You were paying attention to the wrong thing. The spreading of the cars is something we called top-down attention, or -- it's a wide movement, very striking, that captures your attention, and you cannot see something that is happening simultaneously that is a more subtle notion, such as sliding a card under the salt shaker.

Ted Simons: You basically did that as were you sliding the cars.

Tony Barnhart: Absolutely. Would you like to see it again?

Ted Simons: I guess so. I've already played the fool once.

Tony Barnhart: I'm not going to tell you how I got your card where it needed to be, but I did have it palmed. And so all it took was a spreading of the cards while I placed the salt shaker --

Ted Simons: I just did it again. I looked at the cards again while you did it again.

Tony Barnhart: Interestingly, it probably didn't fool everybody at home, because there's a big social component too that doesn't travel through the camera.

Ted Simons: Explain.

Stephen Macknik: Well, a lot of the tricks that magicians use involve where they're looking. So if I look at my thumb, for instance, people at home are also wondering why I'm looking at my thumb. If I point up there, this is called joint attention. And magicians pump that like a lab rat, pump as cocaine lever, right? They really want to control you, and where you're looking and where you’re thinking, and one of the most powerful ways is to pay attention to it themselves, but in a false way. They're really not paying attention, they're doing something else.

Ted Simons: Alright, you've got another example here for us.

Tony Barnhart: Sure. We'll do something with some old antique silver dollars. The trick uses four silver dollars and three hands. I only have two. So I'm going to need your help.

Ted Simons: I see this one coming. Sure. All right.

Tony Barnhart: What's going to happen is one at a time I'm going to make the coins travel invisibly through the air into your hand. Would you hold out your hand? Palm up. We'll make the first one, go you'll hear it go. There it is. The first coin traveled invisibly from one hand to the other. That leaves three coins. Let's try it again. We'll make it slower.

Ted Simons: All right.

Tony Barnhart: There it was.

Ted Simons: There we go. All right.

Tony Barnhart: Now, you may be skeptical. Let's try it again. I want to make it as fair as possible. One from there over to here. One coin left. So you're developing hypotheses. You might think that it's traveling up my sleeve. I don't want you to think that. Steve, would you hold on to my sleeve? Hold on -- very nice. That's quite a killer grip. Let's see if we can make it happen. The last coin disappears.

Ted Simons: All right. Do you want to explain what happened here?

Stephen Macknik: Well, there's a couple things going on. One of the things that Tony was doing was he was changing the method every time he did it. So he wasn't actually -- obviously he was moving the coins from one hand no the -- to the other, but he wasn't doing it the same way every time. As soon as you saw it happen one time, you're like maybe he did that. And then the next time it would be something different. But another thing that he was invoking was divided attention. He was actually getting you to apay attention to more than one place at a time, which like a sucker, we all did.

Ted Simons: Thank you.

Stephen Macknik: And this keeps you from allocating your attention effectively to either place. So this gets back to the issue of multitasking. When we create our attention in our brain, or when we have an intentional spotlight where we enhance what we're paying attention, to but everything else is suppressed, and the harder you try the more everything else is suppressed. It's one of the things we've discovered, that this is what's happening. And so what was -- what he was doing was he was getting you to pay attention to one thing, suppressing any of the information that he may have otherwise revealed when he was doing something in the other place.

Ted Simons; And that again is part of how the brain works. In terms of energy, there's only so much energy to focus on one thing? What's the speculation here? What’s the thought?

Susana Martinez-Conde: Our brain has limited resources, first it has to fit inside of a skull, and that's a limited space. So we have limited neurons, limited connections, and the brain does a lot of guestimation on what's there. And most of the times when the brain guesses, it's what's really happening, but magic tricks are the sign to take advantage of the brain's limitations.

Ted Simons: As a magician, Tony, I would imagine things like humor, like other -- as long as have you someone's attention over on X, you can do anything you want on Y.

Tony Barnhart: Absolutely. And humor is integral to magic performance. That's why a lot of magicians become amateur comedians, because the moment someone understands a joke, all of their attention gets suppressed, all their external attention. This is -- there's no evidence for this yet, but hopefully it's coming along soon. So when I'm doing close-up magic with people, and I have to do some piece of sleight of hand, I like to time it so the sleight of hand happens with the punch line of a joke. It seems to reduce the odds that people will detect the sleight of hand.

Ted Simons: This is fascinating. We've got 30 seconds left. What do we take from all this? What should we learn from all this?

Stephen Macknik: I think the main thing that you want to take home from this is that you can't multitask, multitasking is a myth. So don't try to text or email while you drive. Because while you're doing one, you're not doing the other. So -- and the other thing is that you have a spotlight of attention where you enhance things and suppress everything else, and that's how magic works.

Ted Simons: Alright, great stuff. Thank you for joining us. It was fun.