Voice control
Voice control has just recently become interesting to me. Ikuko bought me an Amazon Echo and I’m surprised at how useful I find it. I’m also surprised at how well it works– it can decipher my commands even when I’m whispering or mumbling from across the room. But the usefulness is what surprised me the most. Siri, Cortana, and Google Now were never that compelling to me. Typing is pretty fast and discrete and I prefer that to shouting at my phone. What makes Amazon Echo nice is that it leaves my hands free. It’s always listening and when I’m in the kitchen I often have my hands full. So with no button pressing or fumbling with a device, I can speak instructions that are quickly parsed and executed.
That got me thinking about where it might be useful in rigging for difficult experiments. For example, when making patch clamp recordings under the guidance of two-photon microscopy (for example…), the operator has one hand on the 3-axis control for the imaging system and the other hand is on the 3- or 4- axis control for the pipette manipulator. So there’s no free hand for starting and stopping the imaging, recording the imaging, zooming in and out, etc. It could be handy to have voice control for those functions, and it probably wouldn’t be terribly hard to implement. There are open source packages for speech recognition like CMU Sphinx.
Thoughts?
* Of course, in my scenario, the operator also has a pipette in their mouth, so voice control isn’t always a viable option. But for much of the experimental process, there is a valve closed and the operator can speak, so it’d still be useful.
It’s a nice idea… I just wonder about what the realistic response time is of a system like this? In the time it takes a (reasonably skilled) experimenter to move their hands over to the imaging computer and adjust settings, then move back again, a Siri-type system has just begun to think. I see where you’re going with this though…
I agree that it’s never going to be super low latency. That said, I’ve been pretty surprised at how quickly the Amazon Echo system responds. I can ask for a particular audio stream and I get the response in less than a second (much faster than my experience with some other voice systems). I imagine that latency could decrease further for simple commands that are processed locally (rather than over the internet).
I really like this idea. I’ve seriously thought about adding pedal controls for imaging zoom factor, laser power, and switching from seal test to current clamp. Also, wearing a mission control style headset while patching would be pretty baller.
Yes, we need voice control!
Check out https://wit.ai – haven’t tried it, wonder how fast it is …
[…] an earlier post, we discussed how surprisingly useful well designed voice control can be. There are open source […]