The biggest trackball

The Big Red Ball has become the world’s largest trackball! This is another 1-day project by Eric Gradman of SyynLabs. I transformed a 3′ walking globe into a trackball. I mounted it on a wooden assembly featuring four ball-transfer casters from McMaster-Carr. I spring mounted a wireless Logitech mouse such that it maintains contact with the underside of the ball at all times. To spin the ball on the casters is to essentially “move the ground beneath the mouse.”

The Big Red Ball keeps winding up in some weird places…

Speaking at IgniteLA, March 1

I’ll be speaking at Ignite LA on March 1, at Cinespace in Hollywood. Its free, so register early!

Monkeys & Robots Inc.

Monkeys & Robots is now officially a California S-Corp!

Privacy is Dead – Lessons Learned from the Cloud Mirror

I just returned from two weeks at Sundance Film Festival 2010. I was invited to install my interactive art piece “The Cloud Mirror” in New Frontier on Main, the venue devoted to art and interactivity rather than film. The Cloud Mirror is a large portrait LCD TV which displays live video from a camera pointed at the viewer. Casual visitors see themselves live on-screen. Superimposed on the image is a “comic book thought bubble” containing a snarky message. This thought bubble follows the visitor’s face around the screen as they move.

For the adventurous, there is another option. A visitor can pick up a badge from a nearby kiosk. Each badge has printed on it a unique fiducial glyph and a unique numeric id. The visitor then registers at a computer kiosk. They enter the badge’s unique id and log in with Facebook Connect, authorizing my Cloud Mirror application to access their data. Alternately, they can fill out a form with their name, email address, and other questions. Facebook is just “easier.”

Returning to the Cloud Mirror itself, visitors see themselves reflected in the mirror; but instead of a generic message in their thought bubble, they instead see information culled directly from their Facebook profile! Their “favorites” (movies, music, activities) are there; their relationship status; their last status update. But the Cloud Mirror doesn’t stop with Facebook: the visitor’s birthday is used to fetch their horoscope. Their name is used to search IMDB for their movie credits (this is Sundance after all!) and the Internet Sex Offender Registry. All the information gathered in this way is packaged up into little messages that fit in a thought bubble (“I didn’t know my name matches 4 registered sex offenders!”)

But there’s another surprise waiting for those wearing bages: When seen in the Cloud Mirror, the fiducial glyph on the badge itself is replaced with a photograph. The visitor is confronted with a “virtual photo album” hanging around their neck, and in that photo album are photos downloaded from Facebook and Flickr photos, those embarassing photos which their Facebook friends tagged them in, and image searches on their name.

What About Privacy?

Do you feel uneasy about this? Are you filled with terror at the prospect of your ephemeral online identity rendered in full color on your face in public? I promise that feeling will pass. People grew comfortable with the Cloud Mirror in stages. The first stage is a casual encounter: walking briskly past the screen they catch a glimpse of their face, and pause to read what the Cloud Mirror has written in their thought bubble. Groups of people gather around: sometimes groups of friends, but often aggregations of random people brought together by the thrill of performing on camera for others. They see how many faces they can fit in the frame, take pictures of one another on camera phones, and laugh together.

And then, someone realizes that this is only part of the story. There’s a registration kiosk, and a pile of badges, and instructions that hint at what the Cloud Mirror *really* does. The ice already broken, the group lines up excitedly to log in to Facebook.

Sometimes, there’s a hold-out. Someone who clings a little more tightly than the others to their illusions of privacy. But peer pressure always wins out in the end. You’ve got a Facebook page… what could you possibly have to hide?

The last stage is when they return a day later. They bring their friends, their family, their spouse. People in their network whose lives they already know. They’re giving their friends the opportunity to share on the big screen. And who on Facebook doesn’t love that?

The Next Generation Cares Less Than You Do

It may seem obvious that a generation that’s had access to social networks since they learned to type would have notions of privacy different from your own. But there’s no clearer illustration of that than the Cloud Mirror. Over the course of the week, I saw groups of junior- and high-schoolers register with the Cloud Mirror without a moment’s hesitation. What happened next was even more fascinating. They would proceed to literally *shove* one another out from in front of the camera, hungry for a moment on stage presenting the social network they’d so carefully cultivated .

There wasn’t a hint of hesitation in their actions. To them, the Internet is for sharing.

If Asked For Your Password

The Internet is for sharing interests and relationships, answering quizzes, and telling the world what you had for breakfast. If there’s a text-input box on a web form labelled “Boxers or briefs,” you can bet that a significant fraction of visitors will cheerfully answer the question. This doesn’t trouble me… I’m guilty of over-sharing too.

What troubles me greatly however is one box in particular that people were willing to fill in: “PASSWORD:”

I used what’s called Facebook Connect to link the Cloud Mirror to people’s online profiles. Facebook Connect is a well-conceived solution to the age-old problem of maintaining separate usernames and password on each website. With Facebook Connect, a participating website can compel its visitors to login instead to Facebook. Facebook responds with the user’s credentials. As a side-effect, Facebook authorizes the website to access the user’s Facebook profile for the duration of the login session. Its growing in popularity, and for many the Connect login screen is a familiar sight.

But entering your username and password into a familiar login screen is simply not safe on any computer other than your own! To be sure, public computers have *never* been safe. What shocked me was the thoughtlessness with which people cheerfully entered a username and password… for the online account that secures a growing amount of their information throughout the Internet!

Implementing Facebook Connect on the Cloud Mirror was an afterthought. I assumed from the start that asking people to enter a username and password was simply too bold. It was by far the most popular way to register.

The lesson here is that if you give someone a shiny trinket, they’ll give you their password in return.

Conclusion

The Cloud Mirror was an oddity at Sundance Film Festival, but an oddity that attracted many thousands of people over the course of the festival, including Robert Redford himself.

For many it was a source of joy which they returned to over and over. For others, it was profoundly awkward. The girl who’s IMDB profile credited her with porn after porn. The woman whose thought bubble said “Facebook says I’m single!” Both she and the man she had just met froze like statues, unwilling to look at one another (too soon!) In my week at the Festival, I heard people discussing the Cloud Mirror over dinner, and showing one another the pictures they’d snapped.

The Cloud Mirror reveals our online identities, but it also reveals that we’re easily persuaded to exchange personal data for a shiny trinket or a quick laugh.

DirtDB: A plugin-based network daemon to dig up Internet dirt

I’m currently working on DirtDB, an important component of the Cloud Mirror for installation at the 2010 Sundance Film Festival. DirtDB is a network daemon that answers requests via XML-RPC to “dig up dirt” for people on the Internet.

Its been up and working for two hours, and in that time I’ve developed a twitter plugin and an IMDb module. Next up is the National Sex Offender Registry and Flickr, with plenty more to come in the weeks ahead.

It expects that a small pool of available information about a victim is available in a MongoDB database. It then invokes a series of plugin modules, each of which use that corpus to mine the Internet (or local databases) for interesting information about the person. It augments the person’s MongoDB document with interesting finds. Because DirtDB is designed with the Cloud Mirror in mind, the plugins are biased to produce small snippets of text (for presentation in the victim’s thought bubble) and images (for augmentation of the victim’s badge).

But DirtDB is a general framework for human data-mining operations. It follows a blackboard pattern I’ve been fixated upon recently.

Developing DirtDB has exposed an interesting problem: plenty of people share a name. How then to distinguish you from all the other people who call themselves you? The solution lies in a technique that I have yet to relate to a formal model, but for which one certainly exists.

Assume that DirtDB knows some additional information about you, beyond merely your name. Say we know your email address. Even armed with this information, many Internet databases and APIs allow you to search by name, but not by email. Realizing, then that a name-only search is likely to bring up you and your dopplegangers, you might believe that knowing your email address is not useful in these cases. But in fact, using a name-only search, DirtDB can mine the internet for representations of many possible identities. Each name-only search yields a new proto-person with attributes that, though juicy, don’t conclusively belong to the victim.

Once a sufficient number of these proto-identities are built, DirtDB can try to collapse them by looking for correlations between their attributes. Did two name-only searches yield a similar physical address? Then those two identities are the same identity! Did one of those results include an email address that matches the one already on file? Then we have a chain of implication.

Now that I think about it, this sounds like “ABox reasoning” which I learned in semantic web research.

DMX Python module

I’ve released my Python module for communicating with DMX devices. Its been tested with the ENTTEC USB Pro module. You can find the module in the Cheese Shop.

Working OpenCV python bindings

I have long cursed the absence of working python bindings for OpenCV. However, with the recent release of OpenCV 2.0, there are new python bindings in the distribution that not only “just work,” but behave in an unexpected and wonderful way: OpenCV functions can manipulate numpy arrays in-place! It took an afternoon to get things properly compiled and seaworthy, but I can now use OpenCV’s Filter2D method to convolve a numpy array without any memory copies.

Compile OpenCV 2.0

1. Download OpenCV from WillowGarage. Resist the temptation to use autotools. Use cmake instead. The INSTALL file has the details. Read the INSTALL file. Learn from my mistakes.
2. Run make install. The python bindings will be installed… to the wrong place. If you’re using Ubuntu, you should move the installed module from /usr/local/python2.6/site-packages to /usr/local/python/dist-packages
3. run ipython and import cv. If this works, you’re good to proceed. If if fails, you’ll be tempted to import opencv.cv. Resist. This is the old binding… this road ends in tears.

Some sample code


import cv
import numpy

a = numpy.arange(256).reshape((16,16)).astype('u1')
ipl = cv.CreateImageHeader(a.shape, cv.IPL_DEPTH_8U, 1)
cv.SetData(ipl, a, a.shape[1])
cv.Flip(ipl)


Observe the contents of the a array before and after the call to Flip. You’ll see that the OpenCV operation has operated on the underlying numpy data.

Now all I have to do is develop some simple SWIG bindings for PointGrey FlyCap SDK, and I can do much of my computer vision stuff in Python.

Update

Much to my surprise, HighGUI works in a python thread. The following code should be run in ipython. I can test out OpenCV operations on im and see the results live in the HighGUI window. Take that, C++

import cv
import threading

orig = cv.LoadImage("test.png")
im = cv.CloneImage(orig)

def waitkey():
  cv.NamedWindow("debug")
  while True:
    cv.ShowImage("debug", im)
    cv.WaitKey(45)

threading.Thread(target=waitkey).start()

A formula for accurate freeway traffic visualization

Maps are a tool to represent the distance between any two points. Distances are subject to a distortion induced by map projection, but lets set that aside, because we’re going to make a map of Los Angeles, over which projection errors are negligible.

I want to make a different type of traffic map–one that represents the time required to travel between points at fluctuating speed. It does so by distorting or stretching the map along freeways (actually, along any route for which we have traffic data) so that that road and all the map areas connected to it are made farther away.

The formula for doing this is as follows:

• Get OpenStreetMap data for Los Angeles.
• Map each point on the map to a vertex on a 2d mesh
• Apply a surface deformation along each shortest path between points for which there exists traffic data (do any of the providers have a usable API for this stuff?). Note: we have to finely divide the mesh, so that high-traffic routes that are “contained” by low traffic routes are turned into zig-zags.
• Apply the deformation on the grid back to an image of the original map.
• Post the data live on the web, as a warning for those who dare to travel the infinite streets of Los Angeles in traffic on the evening of a long holiday weekend

Me… I’ll be biking it.

brainstormed with @kjkjerstin and @armedneutrality

Python code to decode SMS PDU

Internet, I was recently working on a project that required decoding incoming SMS messages from a T39 unlocked GSM phone over bluetooth. This was non-trivial. But for you, henceforth, it shall be trivial. For more information, see this place.


Copyright (c) 2009 Eric Gradman
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

import serial
import time
from cStringIO import StringIO
from math import ceil
from binascii import unhexlify, hexlify
from itertools import *

class T39(object):
  def __init__(self, file="/dev/tty.T39-SerialPort1-1"):
    self.file = file

    self.ser = serial.Serial(file, 115200)
    self.pdu = PDU()

    for s in ('ATZ', 'AT+CPMS="ME","ME","ME"', 'AT+CNMI=3,3,2,0,0'):
      self.ser.write("%s\r" % s)

      while True:
        d=self.ser.readline()
        print d
        if d.startswith("OK"):
          break

  def fetch(self):
    s = self.ser.readline()
    return self.pdu.decode(s)

class PDU(object):
  def decode(self, s):
    s = unhexlify(s)
    d = StringIO(s)

    # parse SMSC information
    p = {}
    p['smsc_len'] = d.read(1)
    p['type_of_address'] = d.read(1)
    p['sc_num'] = self.unsemi(d.read(ord(p['smsc_len'])-1))

    p['msg_type'] = d.read(1)
    p['address_len'] = d.read(1)
    p['type_of_address'] = d.read(1)

    p['sender_num'] = self.unsemi(d.read(int(ceil(ord(p['address_len'])/2.0))))
    p['pid'] = d.read(1)
    p['dcs'] = d.read(1)
    p['ts'] = d.read(7)
    p['udl'] = d.read(1)
    p['user_data'] = d.read(ord(p['udl']))
    p['user_data'] = self.decode_user_data(p['user_data'])

    return p

  def decode_user_data(self, s):
    """PDU user data is stored in a strange 7-bit packed format"""
    bytes = map(ord, s)
    strips = cycle(range(1,9))
    out = ""
    c = 0    # carry
    clen = 0 # carry length in bits
    while len(bytes):
      strip = strips.next()
      if strip == 8:
        byte = 0
        ms = 0
        ls = 0
      else:
        byte = bytes.pop(0)
        # take strip bytes off the top
        ms = byte >> (8-strip)
        ls = byte & (0xff >> strip)
      #print "%d byte %x ms %x ls %x" % (strip, byte, ms, ls)

      # append the previous
      byte = ((ls << clen) | c) & 0xff
      out += chr(byte)

      c = ms
      clen = strip % 8

    if strip == 7:  out += chr(ls) # changed 6/11/09 to incorporate Carl's suggestion in comments
    return out

  def unsemi(self, s):
    """turn PDU semi-octets into a string"""
    l = list(hexlify(s))
    out = ""
    while len(l):
      out += l.pop(1)
      out += l.pop(0)
    return out

Egometrics [build day 1]

I was inspired by Dorkbot this afternoon, so once I could extract no more sympathy for my hangover I decided to get some work done. There is no better cure.