The Event Marketing Summit brings together thought leaders in the industry to discuss the newest trends and best practices for events and trade shows. This is the first year that EMS will feature a UX track where attendees can learn about creating, designing and building impactful user experiences.

1. Science! This interesting study shows how the brain responds to e-books vs printed books and where e-reader interfaces are lacking.
2. The movie Oblivion features some intense GFX. Catch a sneak peek of the light table and helicopter UI with this GFX Montage.
3. Gestural interfaces have come a long way since the early sixties. This article highlights the milestones leading up to the creation of the Microsoft Kinect.
4. The Montréal en Lumière festival this year featured an interactive installation called Lift. Dynamic graphics were activated on a massive dome when people threw giant bouncy balls into the air.
5. Our favorite diversion of the month is Garmoshka, a Russian button accordion that you can play by expanding and contracting your browser.

Whenever Stimulant embarks on a new installation project, we survey the landscape of hardware solutions to determine what we can use to create a robust and immersive experience. Our offices are full of touchscreens, sensors, video cards, and gaffers tape and we’re constantly putting those pieces together in different ways to see what we can accomplish.

Our team has been spending a lot of time lately with depth-sensing cameras, which have gone from being a futuristic dream to a mature product category since the release of the Microsoft Kinect in 2010. We’ve worked with all of these devices in an array of different projects. How does each device fare? Read on.

The Original: Microsoft Kinect

Version: Microsoft Kinect for Windows SDK 1.7

It didn’t take long for the Microsoft Kinect accessory for XBox 360 to be reverse-engineered by the open-source community and repurposed for use in a greater context instead of just games. Fortunately, Microsoft decided to foster that community rather than fight against it by launching Kinect for Windows, a slightly-tweaked version of the hardware combined with a fully-featured set of development tools. The Kinect is actually a package of sensors, in addition to the depth-sensing camera there’s an infrared camera, an RGB camera, and an array of directional microphones.

Microsoft Kinect Advantages:

• Skeletal tracking: Applications can track the positions of user’s joints (head, shoulders, hips, hands, etc.) in space. Two people can be tracked simultaneously, in either standing or sitting poses.
• Face tracking: Various attributes of the user’s face can be tracked, including the relative positions of lips and eyebrows, which can be interpreted as facial expressions.
• Multiple sensors: Multiple Kinect sensors can be used together in order to track more users simultaneously, or to get a wider view of a space.
• Raw data: The Kinect SDK provides access to the raw depth data from the sensor, as well as images from the IR and RGB cameras. This can be processed by the application developer as they see fit.
• Voice control: The microphone array in the sensor can be pointed toward the user to better capture speech. Developers can pre-define a set of commands constituting a grammar for an application. Users are informed when one of a set of commands has been spoken.

Microsoft Kinect Issues:

• Compared to other sensors, the device is fairly large making it more difficult to conceal in an installation.
• A dedicated power cord is required because the sensor uses more power than can be provided by a USB bus. The amount of data generated by the sensor also tends to saturate a USB controller, so it’s recommended that each sensor be on its own controller.
• The sensing resolution is not as advanced as other sensors on the market. For example, the Kinect cannot easily distinguish individual fingers on a hand, which means gestures tend to involve more gross movements than simple pointing.
• Up to six users can be recognized in the field of view of the sensor but only two users can be tracked in detail.
• While there are open-source toolkits for working with the Kinect on non-Windows platforms, most of the features listed above require the Microsoft SDK, which is only supported for desktop applications on Windows 7 and 8.

Best for:

• Kiosks, installations, and digital signage projects where the user will be standing fairly far away from the display. 
• Windows development environments.

The New Challenger: Intel Perceptual Computing SDK 2013

Version: Intel® Perceptual Computing SDK 2013

Intel’s Perceptual Computing SDK 2013 aims to provide many of the same features as Kinect in a smaller package and a less Microsoft-centered toolset. Intel recently opened up the Perceptual Computing Challenge to encourage development of innovative multimodal interfaces with this SDK. The hardware is designed for tracking of a single seated user at close range and includes RGB and depth cameras as well as dual microphones.

Intel Perceptual Computing SDK 2013 Advantages:

• Smaller and less expensive: The Intel camera (produced by Creative) is smaller than a Microsoft Kinect for Windows sensor, is powered over USB, and is designed to sit on top of most computer displays.
• Close-range tracking: It is specifically built for close range tracking, with a range of 0.5ft to 3.25ft (and a diagonal frame of view of 73 degrees).
• Hand posture/gesture recognition: The SDK allows recognition of hand postures like thumbs up, thumbs down and peace, and gestures like waving, swiping and circling with a hand. Other gestures like grab and release or pan and zoom can be implemented by examining the openness of the palm and fingertip positions.
• Facial analysis: The Intel SDK provides capabilities for face tracking, recognition and detection as well as age and gender determination. Expressions like smiling and winking can also be detected.
• Speech: Developers can leverage speech recognition by specifying a predefined list of commands, or multiple lists constituting a grammar. The SDK also has built-in support for speech synthesis powered by Nuance.
• Raw data: Developers have access to raw color and depth data from the sensor along with a confidence map to account for distance and light conditions.
• Framework support: The Intel SDK supports frameworks like Processing, Unity and OpenFrameworks and ships with basic examples to make setting up and writing simple apps with these frameworks easier.

Intel Issues:

• Getting some of the deeper features (like age and gender detection) to work is a bit tricky.
• Due to the close range of the tracking system, hand gestures must be designed such that a user’s hand doesn’t occlude their own view of the display.

Best for:

• Desktop/laptop applications where the user will be seated in front of the PC.
• Close range applications where features, apart from hand tracking and recognition, are necessary without too much precision or accuracy.

The Small Wonder: Leap Motion

Version: Leap Motion SDK v 0.7.7

The Leap Motion controller is a tiny unit designed to be placed below and in front of a display, and uses an infrared technique to determine the position and orientation of fingers (or finger-like things, such as pencils) while they are positioned over it. Tracking is very accurate and fast, and the device can be calibrated to map fingertip positions to precise locations on screen. Unfortunately, the device’s range is quite limited at 8 cubic feet, but it seems like quite a bit less in our experience. There is also no access to depth data or a point cloud. If what you want is very accurate finger tracking, the Leap Motion is perfect, but it’s not very well-suited for much else.

Leap Motion Advantages:

• Gesture recognition: Finger tracking is fast and accurate.
• Even smaller and less expensive: The device is physically very small and is inexpensive.
• Developer friendly: The “Airspace” application store provides a way for developers to market and distribute Leap apps.
• Framework support: The Leap supports a number of frameworks, including .NET, Processing, Cinder, etc.
• Compatible: Works on both Mac OS and Windows.

Leap Motion Issues:

• Sensing range is fairly limited.
• Only fingers are tracked. There is no skeleton or face tracking.
• No access to the raw sensor data.

Best for:

• Controlled kiosk environments with a pointing-based UI.
• Generally best for general audience desktop apps which can be distributed in the Airspace store.

Summary

We’ve discovered that while there are a number of interesting camera-based sensors available, each tends to specialize in a few features, which are different from the others. For standing skeletons, Kinect is the only way to go. For seated voice and gesture, the Intel device may be a better choice. For precise pointing, the Leap is accurate and fast. It’s also possible to combine multiple devices. What could we do with two Kinects combined with a Leap, for example?

At Stimulant we always have a dual goal of creating amazing interactions beyond the typical mouse and keyboard, but we also require our installations to be bulletproof enough to withstand sustained public use (and abuse). These robust and affordable systems open up a huge range of possibilities for our work, and hopefully yours as well. Share your experiences with depth-sensing devices with us on Twitter @stimulant.

Date: 4/2013

Ray brings over twenty-five years of product development experience in creative, marketing, and management roles. He comes to Stimulant from Microsoft, where he spent twelve years as a Senior Director in Platform Evangelism and an Executive Producer in Microsoft Studios. During that time, he helped launch countless new products and worked with a diverse set of partners. Prior to Microsoft, Ray co-founded a new division within 3Com/Palm, worked as an Executive Producer at Viacom, and worked on the launch of the Sony Playstation.

Q&A with Ray

What appealed to you most about coming to work for Stimulant?
I had an opportunity to work with some of the Stimulant folks several years ago, and ever since I’ve admired their work from afar. Stimulant’s assembled a team of all-stars and I’m privileged to be a part of it. I’m particularly excited about working with a group of people who share my passion for taking technology in new and interesting directions.

Tell us about one of your favorite projects that you worked on prior to joining Stimulant.
I recently had a chance to work with the folks at Sesame Workshop, the people behind Sesame Street, on a series of educational interactive experiences for toddlers. During the course of that work, I realized just how important Sesame Street was to my own development and just how many fundamental lessons I learned from it. It was a real honor to have an opportunity to recast some of those lessons for a new generation.

Tell us a unique or interesting fact about yourself.
I’m the only person in Seattle who doesn’t eat seafood or drink coffee. Oh, and there’s no couch in my living room–just a pool table.

What do you do in your free time?
I’m a fanatical movie buff. There are more than four thousand films in my personal collection, and I generally catch four or five of the new releases each week.

1. The World Wide Maze is one of our favorite Chrome experiments yet. It’s a really playful idea and utilizes multiple devices to create a unique experience.
2. The Google Glass feature no one is talking about. This is a great, to-the-point article about the kind of impact Google Glasses could have on our privacy and social interactions and why we need to start thinking about etiquette for wearable computing before we go down that path.
3. AntiVJ’s Paleodictyon takes inspiration from something in nature and turns it into an amazing work of art.
4. The Chromatic Typewriter creates beautiful paintings with outdated technology.
5. Our favorite diversion of the month is Whilst in SF, a collection of hilariously accurate GIFs.

Students were challenged to explore the paradigm shift occurring with the ways we interact with technology. It’s no longer about using technology to simply satisfy a user need; it’s becoming further embedded into our daily lives and extends into ways we connect with one another. Students were also encouraged to explore a number of sub-themes identified by Intel Labs including; Social Sense Making and Play, Playful Data, and Open Platforms for Playful Making.

Five finalists were selected to present their concepts in a 5-10 minute presentation. It was interesting to observe that all concepts were tempered in realism. Students did not try to invent new technology but envisioned solutions using technology that already exists today. As one of our hiring managers, Nathan was excited to see a balance of big, outside of the box ideas that still retained a high degree of feasibility. Students were thinking about the broader social impacts of interactivity in their concepts.

Challenge winner Bethany Stolle exemplified this year’s theme by investigating the social impact technology can have as a medium for better understanding autism. Her concept was designed to encourage a child with autism to share and interact with their family through technology and in turn, this helped unlock the visual stimulants that make the child happy. Using a simple habit loop (Sense – Trigger – Capture – Share) and a camera synched to a tablet, the child captured the images that made them happy throughout the day. A specialized app encouraged and rewarded socialized interaction with a positive feedback loop each time the child chose to share snapshots of their day with their families.

The winning entries from the IxDA’s Student Design Challenge can been seen here. Congratulations to all of the finalists!

Maxim Integrated asked Stimulant to create a large scale interactive wall in the lobby of their new corporate headquarters. Our challenge was to create a fun, engaging and brand relevant way to introduce the new brand and inform guests about the company. Challenge accepted!

The resulting experience is designed to embody the three elements set forth in Maxim’s brand redesign; inventive, bold and grounded. The interface, presented on six 55-inch displays with a multitouch overlay, is a playful interactive brand extension married with a high-level, media-rich company overview. The wall is one of the largest physical Silverlight deployments to date.

Maxim Integrated touch wall located in the corporate lobby.

To stay on our tight schedule, we exposed dozens of variables that affected the visual system (speed, density, behaviors) in a custom control panel. This let us dial in the “feel” of the piece with the client in real time, expediting communication and saving many days of back and forth on revisions. This proved so successful that it has become a regular part of our development process.

The Stimulab will also showcase many of the concepts and technologies we explored in 2012, including:

• Multi-modal interfaces that merge speech, gesture, touch, and multiple devices
• Mobile and tablet experiences for Windows 8, Android, and iOS
• Massive, multi-touch touchwall installations
• Generative art and procedural graphics
• Digital interactive museum exhibits
• Embedded applications for small, portable devices
• Consultation on designing for devices

Continue to follow us on Twitter @stimulant. We’ll keep you up to date on the Stimulab’s progress as well as our latest ideas for events, museums, retail and beyond.

The release of Windows 8 in 2012 prompted a lot of PC manufacturers to start thinking about “touch-first” and natural user interface principles for the masses. Stimulant was lucky enough to be asked to create a fun, expressive, multi-user creativity application for Samsung’s Product Innovation Team (PIT) to be shown at the Samsung booth at the 2013 Consumer Electronics Show in Las Vegas.

Samsung PIT’s San Jose office and Stimulant collaborated to create Samsung SketchMotion, a touch-driven, single or multi-user creativity application that combines traditional drawing tools and animated procedural, algorithmic tools. One goal, which pervades much of our previous work, is to let software be a force multiplier for the user’s creativity: The tiniest amount of effort with no domain knowledge still yields unexpected, emotionally-compelling results.

With this in mind, we created a number of unique, always-evolving, ever-moving “painting” tools that let pictures move, breathe, and evolve. We created tools that leave continually-moving, animated particle systems as the user draws, as well as geometrically iterative tools that paint expanding fractal patterns to provide broad textural fills. This algorithmic visual sandbox all ran in a Modern-style UI on Windows 8, demonstrating a minimal, touch-centric interface in a rich, full-screen environment.

Samsung SketchMotion can be viewed at the 2013 CES show this week at the Las Vegas Convention Center, Center Hall, Booths 12000 and 12527.