If you’ve ever seen the movie The Sixth Sense, then you will understand what I’m about to tell you. There is a whole… other group of people walking around out there, and you need to be on the lookout for them. But if you know what signs to look for, you’ll be okay. Here’s a handy field guide.
Outboard Ears
It is a fairly common thing these days to see people wearing small bluetooth headsets on their ears, for use with cell phones. One common term for these people is ‘The Borg’, or if you don’t follow Star Trek, you could think of them as ‘dorks.’ The joke is that it’s getting harder to tell the truly crazy people apart from all the people walking around talking to themselves, until you realize that they’re the ones wearing the headsets.
My gut aesthetic sense (and popular culture’s opinion, it would seem) is that anyone wearing one of these earpieces is ‘different from us’, and something about these headsets makes people uncomfortable or irritated in some way. We make jokes about them, but it’s to ease pressure — perhaps to mask the annoyance of having them bring their half of a conversation into our public listening space — and now they can do so hands-free, meaning they can do more everyday things while chatting. A technology miracle! But more annoying. Even if you’re not actually being a bozo, the fact that you’re wearing one (or even that you own one) calls to mind a prior unpleasant experience that someone else has had with one of these Borg people yapping it up in their earspace.
Your new Third Eye
These days it’s also becoming more common to see people snapping pictures of objects with their cell phone camera. Maybe they think something is cool or funny, and want to make a note of it, or share it with their friends (on facebook, perhaps.) Or maybe they’re walking around looking at items in a store, using an image-recognition service (like SnapTell) to browse reviews or check the prices of items on the shelf against those online.
But there are subtle boundaries that people are encouraged to respect. For example, if someone was talking on their Borg headset in a library or during a play, you might feel justified in interrupting them to ask them to take their conversation outside. If you see someone taking your picture in a public place, they may have the right to do so, but it might still creep you out. If you saw a stranger taking a picture of a child, it might also make you even more uncomfortable. (Enough to encourage dumb legislation, even.)
And your Spare Brain
There’s an argument that having a PDA as an ‘offboard brain’ makes you dumber, because your own brain doesn’t have to remember as much stuff, and that losing it (or having the battery go dead) cuts you off from all of that information. While this is probably true to some degree, David Allen (of Getting Things Done fame) makes a pretty compelling counter-argument that if you can offload the important details into a trusted system, then you can blissfully Zen Out, knowing that you’ve got everything under control. Just don’t misplace your battery charger, I guess, and be sure to keep a backup of your spare brain.
Whose Rights are Right?
Whether you’re on the side of the techno -philes or -phobes, we have to balance the rights of the people empowering themselves with these technologies against the rights of everyone else… to reasonable expectations of peace and quiet (from their inane conversations), and privacy from their cameras.
Along these lines, I would argue that there are pros and cons to making the technology more or less conspicuous. Depending on your beliefs, you may choose to assert your right to take photos of items in a store. (I was once asked to stop photographing a set of wall hooks at The Container Store.) Or to take pictures of other people, if you’re in a public place. But there may sometimes be social or legal consequences, or at least some degree of conflict with others over your choices.
Your Outsourced Sixth Sense ($9.95 in the App Store)
I’ve been kicking ideas around about this TED video by Pattie Maes and Pranav Mistry from MIT’s Media Lab, entitled “Unveiling the “Sixth Sense”, aka ‘game-changing wearable tech.’ It’s basically a Wearable + Gestural + Cheap/off-the-shelf mashup with cell phone-as-brain; in its current implementation, people wear a small projector hanging downwards from their neck, with a 45-degree mirror so it can project out in front of them. There is also a web cam above this, which studies the scene looking for four small colored marker caps (worn on the pointer finger and thumbs of each hand). This is how the user interacts with the system.
Images copyright MIT Media Lab and whoever else, yadda.
The system has some pretty neat applications written for it. We see demos of various features: Pranav framing his hands to ‘snap a photo’, then reviewing and sorting his ‘snapshots’ (projected onto a wall), later on; drawing a watch on his wrist to see a clock projected onto it; ‘pinching’ in and out and scrolling around a map (again on the wall); dialing the phone (with dial pad projected onto his hand), etc. (It’s his actual phone number, too. I wonder how many people have called him as a result of seeing the video?)
Then we see Pranav in a grocery store, picking up a roll of Bounty paper towels. He is shown a ‘yellow light’ (think traffic light), projected onto the packaging. (This presumably refers to how eco-sensitive the manufacturer’s processes and products are.) He swaps this for a set of ’365′ (Whole Foods brand) paper towels and now sees a ‘green light’, and can touch the green circle to get more information. Pattie points out that these indicators could be tailored to the user’s personal consumption preferences; perhaps they just want the softest brand of tissues, and damn the other consequences.
Now he picks up a book, and we see an Amazon product rating. Turning to some random page offers up an annotation. He picks up a newspaper (with the photo cleverly pre-whited out) and a headline picture is projected onto it. An airline ticket reports (in the margin) that a flight is late. He even walks up to a person and sees a ‘tag cloud’ representing that person — right on their shirt. However, it is missing one key piece of information about this person:
- Thinks you are a huge dork.
Wearable Computing is Not New
As an undergrad student at the University of Michigan in the late nineties, I was a member of ‘entity’, a student group that exhibited digital art and was working towards the advancement of a cross-disciplinary multimedia degree program for students. During one of our conferences (Immedia 98), I helped Steve Mann (originally from MIT, then at University of Toronto) present slides about the evolution of wearable computing. This sidebar from the wikipedia article on wearable computing is a great example of how much the technology has changed, and how much more stealthy it has become. The glasses (which are now hip again, for what it’s worth!) contain a camera and a heads-up display.
Steve relayed an amusing story about how he would walk into a store with a (non-functional) Hi8 camera and pretend to be filming the ‘maybe-cams’ (black opaque or mirrored camera domes, some of which are just there to fool you into thinking there’s a camera behind them) until someone would ask him to stop. He would use this as an opportunity to have a dialogue with that person about why it’s okay for them to film him, but not vice versa? Usually the person would be very confrontational, at least until he’d put down the Hi8 camera, at which point they would relax a bit and explain the policy or whatever. Meanwhile, Steve is still filming and making an audio recording of the whole thing through the camera in his glasses. Too funny.
Steve also talked about an app that used head tracking to capture spherical images of the user’s environment, and featured the ability to make notes about certain locations or items in a room. So once again, this stuff is not new; it’s just being tied to an ever-expanding number of mash-up sites and data pipes which are getting easier and easier to access with EDGE, 3G, and so on.
Real-time, portable, Gestural Computing, however… is more new
And that’s why this demo is cool. Products like the iPhone and Microsoft Surface are becoming more mainstream; Apple’s trackpads are supporting gestures. The iPhone has an accelerometer (and some Mac programs use the one built into the MacBook, the Sudden Motion Sensor) which programs can take advantage of. Smaller mobile CPUs are getting powerful enough to decipher simple gestures, particularly those done on-camera; while this concept is certainly not new, the fact that it is possible on an inexpensive, hand-held device is. And that is opening up some new doors for applications like this.
Why this isn’t the Sixth Sense — yet. (5.5th Sense?)
This particular implementation doesn’t work well because it doesn’t track where your head is pointing; the camera just looks straight ahead. It is a chest-cam. (Please form an orderly line to present your jokes.)
A pair of camera/HUD glasses would more often be able to see (and display output) where the user was looking, though I wonder about one side effect — people move their head around a lot more than they move their torso, and I didn’t see too many examples of this system being used when the user was in motion. I wonder if a head-mounted system is too jerky for the current hardware platform?
There was a camera on the bill of Pranav’s cap in the outdoor ‘take a photo’ example, which is a little bit better — but being outdoors in the daylight raises another issue: Notice he’s not wearing his projector outside.
Sunlight washes out just about any projector’s light output, no matter how many ANSI lumens. A really good pair of HUD glasses might feature adaptive optics (like Smart Tinting to adjust the amount of outside light they let through, or the transparency of the HUD graphics) and a really good pair might even track the user’s pupils and determine not only what they were looking at but even the focal distance so the camera could change its focus to match. (I suspect that this would be outside of the contraints, the stated price range.)
Pattie points out that this version is by no means a finished product, and that one of their goals was to implement the system with inexpensive, off-the shelf-technology. But I cannot imagine that a pair of the camera/HUD glasses that Steve Mann was wearing in a photo that is now ten years old could be that much more expensive than a webcam/mini-projector, which she said would cost no more than $350 to put together today.
Finally, the fact that they’re using a projector means that they constantly have to find appropriate physical surfaces to project onto, and may further need to find sub-regions within those surfaces that are appropriate to project data onto.
I think it’s impressive enough if the system can analyze the video input, determine, “What type of object is this: a book? Okay, what’s the current page number?” etc, then go fetch someone’s annotation from the internet and just display it outside of the margins of that page of the book so it doesn’t clobber the text on the page.
But since you have a projector, you not only have to analyze that surface looking for context clues, but now that you have your annotations for the book, you have to say, “Okay: where’s the margin? How can I fit this long-winded user review into this tiny space at the bottom of the page? Hmm. Scroll bars?” etc).
If they’re really doing all of this additional thinking and processing just to be able to use a projector, and they’re doing it all on an affordable cell phone, then I’m certainly impressed by that. I just feel that the delivery of the I/O is a bit off.
My gut tells me that these are just demos with some very carefully programmed parameters, and a well-made video. But hey, I’d love it if this were much more real than I expect. (And downloadable open-source, for that matter.)
The Argument for Subtlety
‘Cole Sear’ (Haley Joel Osment’s character in The Sixth Sense) had an innate ability to see the spirits of dead people. But he seemed to just know about them; he didn’t come out wearing some ridiculous Ghostbusters rig on his back (like Steve Mann, ca ‘Early 1990s’) or it would have fundamentally changed the way we perceived and interacted with his character. Pair this with the phrase attributed to Arthur C. Clarke, “Any sufficiently advanced technology is indistinguishable from magic.”
People who have subtle, effective tech should ‘just know’ things and it should seem magic to someone who doesn’t know what’s going on. You show up looking like a Ghostbuster, and they’re not going to take you seriously. Well, they’re not going to take you out for a beer with the guys, anyway.
This goes back to my argument for inconspicuous technology. Don’t get me wrong; I am a dork, and these are my people. I would love to be able to use face recognition, image recognition, gesture recognition, and all of the other things shown in this video — but NOT with a big obvious camera hanging from my neck (or the bill of my cap) and NOT projecting that information out into the world for everyone to see! I am no stranger to the concept of projecting information onto objects; but I do it to make a point, and because I know that other people will see it. And quite frankly, the use of a projector for this application seems rather intrusive.
(Besides, how long before someone is running around making money by projecting G*ldenP*lace dot com onto everything? Mobile meme spammers with cheapo projectors, blinking subliminal messages… yuck. Sounds like a Charles Stross story.)
Once again, just as there can be consequences to using your bluetooth headset or your cell phone camera at the wrong time, there might be even more severe downsides to using this sort of HUD/cam glasses technology, especially if you abuse the freedom or knowledge that your ‘subtle tech’ gives you. You may just look really weird because people have no idea that you really are doing something by drawing circles on your arm… or you might get locked up for walking around while gesticulating in front of your face while wearing big aviator glasses and colored marker caps on your fingers, because people think you’re on LSD.
At least the projector makes it easier to convince other people that you’re not crazy, if they aren’t scared off by all that crap hanging from your neck. (Ooh! Ooh! I know; get the clip of Princess Leia saying, “Help me, Obi-Wan Kenobi, you’re my only hope!” and make beeping sounds while you project that onto people. Hot date: guaranteed.)
Pattie says, “Maybe we’ll see you back here in ten years with the direct-brain implant version!” but that opens up a whole different product aisle-full of many varieties of canned worms. You think it’s horrible when some Second Life griefer figures out how to hack flying phalluses onto your glasses’ heads-up display? Wait until you can’t take the glasses off anymore.