Being a User Interface designer in this particular section of history has been an exciting ride. In the past few years, we have seen an explosion of new ways of to interact with an increasing number of devices. Although we may never rid ourselves of the mouse and keyboard, like the one I’m using to write this article, there has been a fevered rush to look beyond their confines. Driven by emerging technology, this exploration has yielded a number of promising and useful interaction methods. With these new methods, it’s going to take a bit of time to sort the good and bad interactions and create a set of standards every user can learn.

The first, and undoubtedly most familiar, of these interaction methods is touch. Touch screen technology has existed since 1971 when Dr. Sam Hurst cooked it up in his lab at the University of Kentucky. However, I doubt the good doctor could have envisioned what is possible today. Since Dr. Hurst’s discovery, we have gone beyond the finicky single touch screen, (if you have ever worked in a restaurant and had to enter orders, you know the frustration) to elegant multi-touch solutions that will fit in your pocket. Multi-touch has provided a great number of opportunities for UI designers to create new interactions without the need for a mouse. It is truly the epitome of user-centric design, in that the user becomes the mouse, a mouse with ten cursors.

This new technology has become a sort of digital crack for designers. But, like nearly every keyboard you type on or every mouse you use, touch interfaces will require a degree of standardization. I know not all keyboards are the same, but chances are you do most of your typing on a QWERTY keyboard, unless you’re 14, then maybe you type more on a standard ABC keyboard.

Either way, these two standards, along with a scarce few others, dominate all the devices we use to type. Imagine if you were to pick up a new keyboard at the store and instead of a Q in the upper left corner of your device there was an R or a P. You would need to relearn your typing skills, in much the same way touch gestures will need to be standardized if they are to become an effective mental model.
It seems this process is already underway. Thanks to Apple’s multi-touch push, with other companies quickly following suit, we now recognize swipe as a page turn, tap as click, pinch as zoom out, spread as zoom in, and a turning gesture as rotate. I don’t see it ending with these few gestures; keep your mind open.

The second Interaction method that is quietly making a splash in interaction design requires little to no visual execution to work. Voice control is the sneakiest of interaction methods, but with recent advances in technology, it is making its way into the mainstream.

Do you recall the first time you heard of speech dictation for word processing. “Amazing,” we all thought, “now I can talk to my computer like Captain Kirk talks to the Enterprise.” We envisioned a future where we could tell our house to change the temperature or tell our robot servant to fetch us a fine craft beer from the fridge. That fantasy may still be a bit off, but we’re getting closer. With Microsoft Sync, I can now tell my car to call my sister, or play a song from my iPod. It will even run a vehicle health report and email it to my inbox. How cool is that?

The new Xbox Kinect will do similar things in your living room with your media. Voice control has been Microsoft’s child for a while, and this baby is finally growing up. As with touch interfaces, voice control will also require a bit of standardization. If I tell my car to “play artist Interpol,” it will grab an Interpol song from my iPod and fill my speakers with sounds of neo post-punk goodness. But if I say the same phrase into another device and it doesn’t do what I expect, the system is broken.

The newest, and seemingly mystical, UI technology is gesture-based input. This type of interaction has made its biggest splash in the video game arena. The Nintendo Wii was an excellent start, using a remote to replicate real life actions. Swinging the Wiimote swings a tennis racket on the screen or a myriad of other actions. The Wii started the conversation, but users and designers alike longed for the day when they could be rid of the controller forever.

Johnny Lee, a forward-looking UX designer, was able to take the Wii hardware and do a number of interesting experiments that are worth checking out (http://johnnylee.net/projects/wii/). When the Xbox Kinect made its debut at the 2010 E3 gamers’ conference, the dream was realized. Using a set of stereoscopic cameras, Microsoft has created multi-touch’s cousin – a 10ft experience. Without a remote, users can browse media, play games and video chat with friends from their couch. With this new advancement, however, comes another need for standardization. One interaction in particular needs to be sorted out, because it is the most basic interaction in any user interface. This interaction is selection.

With the traditional mouse and keyboard, selection is as easy as a click. On a touch interface, a selection is a touch. Pretty self-explanatory. As previously mentioned, a voice control interface creates a selection because you are telling it to.  A gesture interface runs into a new challenge, because there are no buttons to push or anything to touch. 

Microsoft has solved this problem in two ways. First, it has placed a timer on each button. If the cursor on your Xbox hovers over a button for a set time, the Kinect will assume a selection. They have also integrated voice control to make these selections more easily. Is the best selection method a timer, or would a push be a better method? The Kinect is equipped with stereoscopic cameras that can sense depth, so why not a point as a selection method? These are all things that need to be sorted out by designers moving forward.

The take away from this flood of new technology and user interactions is that no user interaction is perfect. Most of the time, a combination of two or more input methods seems to work the best.  It will be interesting to see what comes next, but in the mean time, it is the duty of UI designers to standardize actions in their interfaces to help make the experience for the user as delightful and stress free as possible.