Computer mouse driven by sound

By Richard Seven, Electronic Devices / Consumer & Gadgets

Say "ahh" and the cursor zips toward the northeast corner of the computer screen. "Ooo" sends it shooting straight south. Want it to head southeast? Say "ohh." To make the cursor do a circle or figure 8, let vowel sounds bleed into one another, like eee into ahh into aww and so on. You can make it hurry or slow by regulating the volume of your voice. To open a link, make a soft clicking sound.

So goes the University of Washington's "Vocal Joystick" software, which uses sounds to help people with disabilities use their computers.

Its development has been a multidisciplinary task with faculty and students from several university departments - electrical engineering, linguistics, computer science, as well as the Information School - blending their expertise. It is just one of a series of UW-generated assistive-technology projects ranging from enabling the blind to use touch screens to developing an alternative to the point-and-click method of computer navigation.

Researchers have tested the joystick with spinal cord injury patients at the UW Medical Center and just finished another round of testing with 10 participants with varying levels of disabilities.

Susumu Harada, a computer science and engineering graduate student, administered the tests, putting each subject through 12 hours of training. He evaluated how they learned producing the correct vowel sounds, memorized the directional patterns and manipulated cursor speed.

Sometimes, moving the mouse by voice seemed frustrating, even a bit tiring. If the operator was out of sync with his own sounds as recorded by the software, the cursor might speed past a target in one direction and go so slowly in the other that the subject would have to take a break to catch his breath.

Some sounds came easily. Some seemed a bit unnatural and strained. But when a subject caught the rhythm, the task was easy and natural.

There are several options for people who need accommodations in using computers, but the UW software is distinguished on several levels. For one, it doesn't use standard voice-recognition technology. Instead, it detects basic sounds at about 100 times a second and harnesses them to generate fluid, adaptive cursor movement.

Vocal-joystick researchers maintain the system is easier to use because it allows users to exploit a large set of sounds for both continuous and discrete movement and to make visual adjustments on the fly. Kurt L. Johnson, a professor in the Department of Rehabilitation Medicine at the university, says he believes the software has great potential because it is easy to both learn and use.

"A lot of assistive software doesn't get used because it is too complicated," he says. "But I think they've created something intuitive here. We had some of our higher level of spinal cord patients test it, and one of them learned to use it in about 90 seconds." The Vocal Joystick requires only a microphone, a computer with a standard sound card and a user who can vocalize. The team behind the study, funded by the National Science Foundation, hopes to make a prototype available online this fall.

Vocal Joystick began in the electrical engineering department. Professor Jeff Bilmes and his students, especially Jon Malkin and Xiao Li, created the underlying sound-recognition engine. From there, computer science and engineering professor James Landay and Information School professor Jacob Wobbrock, along with Harada, developed creative ways to apply the technology.

Various offshoots of the Vocal Joystick technology, from playing a video game to operating a robotic arm, have been developed. Ultimately, researchers would one day like to apply this technology to a number of home devices, even electronic wheelchairs.

One of the other applications was "VoiceDraw," which allows hands-free computer drawing. Harada used it to "paint" a portrait of Mount Fuji by sounds alone, and he won second place in a national competition for workplace innovation and design.

Wobbrock, who has been mentoring Harada and is a former first-place winner of the national award, leads a group he calls AIM, which stands for accessibility, interaction and mobility. He is working on a software project that makes a mouse slow and become more accurate as a user tries to enter and click on a target (think how you decelerate when your car begins to take a tight curve).

Another of his projects seeks an alternative to mouse clicking by triggering functions when the device crosses a "goal line." You don't click the mouse; you just cross a threshold.

Before coming to the UW, Wobbrock did cutting-edge work in assistive technology at Carnegie Mellon University in Pittsburgh. He says a parallel goal behind all these projects is to make technology work better for everyone, not just those with physical difficulties.

"Think of sidewalk curb cuts," he says. "They help people in wheelchairs, but they also help me pushing a stroller or a grocery cart or riding a 10-speed bicycle."

Fall Ritual - When is it "cold" enough??

Keep Your Hands Warm this Winter!

from gotsyzygy

I'm sure EVERYONE does it. Delaying turning on the furnace and putting those flannel sheets!! It's almost like by doing these things you are giving up on Summer, and let's face it.....

I DON'T WANT IT TO BE OVER!!!

So I watch the thermostat in the house and think about making comments about the "chill" in the air. This generally results in a variety of responses from Bill, none of which is: "Turn on the heat already!!" Luckily the thermostat is right by the computer, I can look over and see currently it's at 63 degrees. Honestly, it's not too bad since I have on cuddle duds (tops/bottoms), sweats (top and bottom) and slippers. Oh, I forgot the hot beverage, coffee at the moment. I also have a small desk lap, which is handy, as it's second function is a hand warmer when my fingers get too cold to type. You have to use caution, as it's gets really HOT!

This morning when I got up, it was at 59, so hey, it's progress!! I actually like sleeping in the cooler weather, so at least that doesn't me. Being cold during my waking hours isn't something I enjoy!!

I think Bill has some secret bet with the Weather Gods!! I'm not sure if he wins anything, but I know he's playing!! How long can I hold out before I have to turn the heat on? I wonder if he writes the date on a calendar every year? Somehow I wouldn't be surprised.

This year I've decided, I'm not going to say ANYTHING!!! Nope, I'll just put on more layers and pile on the dogs if need be. I'll even wear my coveralls in the house if that's what it takes. FREEZE BABY FREEZE!!!! I guess I'd better go and find Mad and Kruz's coats. Seeker is from up north where it's freezing already, so he's LOVING THIS!!

So, let me know....when do you turn on your furnace???

Chills, drafts and a cold mouse hand

It's that time of year again when the air gets chilled. If you sit and work at the computer using a computer mouse late at night after the sun goes down, then you probably have suffered from a cold mouse hand on occassion. It's a very noticeable problem. Your hand is exposed and in the same position for a long time using a computer mouse. Sometimes the mouse hand gets so cold your fingers get numb, right?

Don't laugh! Cold mouse hand is very annoying. What do you do when your mouse hand gets cold?

Mouse Hand Warmer Blog

A place to post thoughts about a cold mouse hand and things we do to keep our mouse hand warm. The Mouse Hand Warmer Blog was created to build awareness of an annoying "mouse hand" condition many computer users suffer from while working at their computer. We sit for hours with an exposed mouse hand in drafty rooms and late nigh drafts. It's time we had a place to post our thoughts and discuss remedies.

This blog is open for comments. Comments are moderated and will be posted when they are relevant to the article's topic. Let's build a very warm place to resolve a cool situation!

Welcome to the Mouse Hand Warmer Blog!