The main thing that we’re excited about is the dynamic generation of screen elements, like the timeline at the top of the rehearsal workspace and the list of notes for each rehearsal. Each user has a newly generated color that appears on their timeline, on their notes, and in the key at the bottom of the screen. The video works (you can pause and play again!) and there is a slider that allows to you skip to new portions of the video. You can pull an individual note out of the stack, and it appears to the left of your rehearsal workspace for later review. And it looks good!

Screencaps:

Starting log-in screen (currently doesn’t have distinct user log-in codes, you just tap to start; we hope that in the future you can log in using the Surface as a fingerprint reader to automatically access your information.)

Overall look of all workspaces upon log-in, assuming four users are logged into the collaborative workspace.

Pulling an individual note out of the stack for review (The white highlight of the note marks it as “read” in the stack.)

Workspace for reviewing the most recent unread notes.

Panel to filter displayed notes by type or user; this button panel shows up when you tap on the timeline, allowing you to control what you’re reviewing.

Thumbnail spaces of past rehearsal. These are currently static, but in future implementations would have the ability to expand to the full size of the main rehearsal workspace we see to the right. This way, you’d be able to use all the real estate of the Surface size to review two rehearsal videos side by side for comparison.

Small key at the bottom of the Surface (can move to the opposite side depending on where you’re sitting) to show you who is logged in and what color refers to them.

Futhermore, there are other aspects of the project that are not clearly implemented at this stage; the project has a ubiquitous computing element consisting on a video camera that records rehearsals, RFID tags and readers to record when actors are onstage, and computerized pens that record notes being taken. Since there are no photos of the setup, here’s my small diagram:

We do HAVE each of the elements – a video camera, RFID tag readers, and the computerized pen, but none are linked together. The pen is the coolest part; it does indeed record the notes you’re taking! 

All in all, I’m pleased with the amount of work we’ve accomplished in only a few weeks; we have a working interface that is aesthetically presentable! I was originally very wary of C#, which the Surface is programmed in, and did most of my design work in XAML, which can control the look of the interface. In spending the last week integrating my XAML snippets with the existing C# code, however, I’ve become much more familiar with both languages. Hurrah! Our presentation is in 30 minutes and I hope to have some good feedback and discussion with attendees and other presentation groups; if we do, I’ll certainly discuss it here. There is also a video describing the whole project that has yet to be uploaded to the web; when it is uploaded, I will certainly link to it here.

At any rate, the central rehearsal review panel was redesigned to a vertical panel from a horizontal panel, which has pros and cons. The major problem with this is that the timeline across the top doesn’t have as much space to expand from side to side; a panoramic timeline is why we originally chose a horizontal interface. However, it turns out that resizing the timeline wasn’t as much of a problem as we thought it would be – it is slightly smaller, but still readable. The big plus side of moving to a vertical implementation is that we were able to get rid of the expanded timeline and move the notes under the video, which makes a lot of sense visually and frees up space elsewhere on the surface for other workspaces.

Those other workspaces are the previously discussed “Recent Notes” panel, which is actually smaller in our real implementation than pictured below, and the “Individual Note” panel, which is useful when a single note is pulled out of the stack for review. Additionally, I created a informational key to live at one edge of the Surface, providing data about the show itself and the list of current logged in users (and their colors).

The workspaces, as seen in Blend, not quite as implemented:

These elements are somewhat different in the actual implementation, which we’re working out now. It’s coming along slowly but is starting to actually LOOK good as well as function well. When we’ve finished the coding tonight and tomorrow, there will be more accurate screencaps to share!

In the end, we decided to simplify the interaction by synchronizing the responses to an interaction of every element on screen. There are three main points of interaction, and each one has the same result. A user can move a highlighted bracket across the main timeline, slide through the expanded timeline, or scroll through the panel of notes. At any of these interactions, the surface comes back with the same response: the highlight on the main bracket moves, the expanded timeline scrolls, the notes scroll, and the video clip moves to a new section. Hopefully, synchronizing these elements should simplify the program and the user experience.

Outside of those interactions, there are only a few other main workspace interactions to deal with: deleting and sorting notes. We’ll be working with these main workspace interactions this week as we move into our horizontal and vertical implementations in C#.

Our proposed interface will consist of four tangible objects (or sets of objects) and will be used in two phases. In the first phase (the recording phase) three objects will be used to record data about a rehearsal. An installed video camera will record any scenes being practiced. Location tags worn by actors will record who is onstage or offstage during said scenes. A computerized note-taking system (ie: tablet computers, computerized pens, etc) will be used by people watching the rehearsal to document critiques or congratulations. In the second phase (the review phase) a single object will be used to review data about past rehearsals. A Microsoft Surface (or other multi-touch surface) will display video footage of rehearsals tagged with notes at the moment the notes were taken during the rehearsal. Notes can also be reviewed separately or sorted by note-taker or note-receiver. Video footage will also be sorted by actor, so an actor can view a whole rehearsal or only their scenes. All of this data will have been processed and analyzed by the Surface.

Overall, the project should help theatrical teams to get more out of rehearsals and create a more collaborative process of review and critique, both for singular rehearsals and for the entire process over time. I’m specifically drawn to this project as a stage manager; I have been working in theater since I was 14 and have often been frustrated by the process of giving notes to actors and reviewing rehearsals. For example, being able to quickly take lines notes on a tablet that would be automatically tagged to video and sent to actors to review at a later date would be amazing. Though the project is extensive, we will hopefully be able to implement enough aspects to prove its worth as a tangible user interface.

ANDREAS LUTZ – Because clicking is so 90s! from andreaslutz on Vimeo.

I’m skeptical as to how well it works, so I’d like to try the site out; however, I’m in the library right now, and other students tend to frown upon using voice activated software in quiet spaces. I could probably get away with the giant hand gestures, but I don’t think I should jeopardize my relationships with the neighboring thesis carrels just yet.

From a TUI standpoint, this is quite different from what we’ve seen in class. It’s the first TUI that is a more full-body experience, and the first TUI that doesn’t require the user’s hands. It is also the first public installation of a TUI that we’ve looked at. And yet, I’m quite certain that corresponds to TUI principals we’ve discussed. For example, the stairs require an innate knowledge of naive physics and your physical body (stepping = pressure = sound played) as well as a knowledge of your environment (how to interact with stairs, for example). It draws upon a social knowledge, as users can play songs together.

And, it really looks like fun. People are drawn to the interface and really want to use it, as opposed to riding the escalator. It’s actually pulled off quite successfully, overall.

In the terms of Reality-Based Interaction: A Framework for Post-WIMP Interfaces, MouseHaus primarily utilizes interaction techniques already familiar to the user: the knowledge of naive physics and physical space. To add a building, park, or street to the workspace, a user only has to cut a shape out out of paper in the size of their new element’s footprint and place it on the table; they can then observe the effect their element has on traffic patterns. All of these interactions are actions familiar to users who understand relative scale, velocity, persistence of objects, etc.  Furthermore, users utilize their body awareness and skills to navigate around the table, viewing their urban scenario from many angles. Social interaction skills are also used when users collaborate around the MouseHaus interface; many users can interact and discuss possible plans, leading to a stronger and more collaborative final decision.

Tradeoffs were also made in the development of MouseHaus. In using paper cutouts as urban elements, MouseHaus’s developers traded expressive power for efficiency and practicality. Though their cutouts are not complex to realistically model lighting or wind, they are quick to create, easy to change, and very inexpensive.

In the terms of The TAC Paradigm: Specifying Tangible User Interfaces, the MouseHaus TUI is made up of a number of tokens and constraints. The urban elements, made of paper cutouts, are the most obvious tokens. Physically, these pyfos are combined with variables to metaphorically represent buildings, parks, and streets within the simulated environment. The material of the elements also operate as constraints; the chosen paper color and size of each cutout (that is, the main variables) represent its behavior and attributes. The table itself is also a constraint, for it limits the physical interaction space of the tokens. Again, the physical interaction space of the tokens is also limited by the tokens themselves: elements cannot be overlapped.

The prevalent TOC relationship within MouseHaus is that of the urban elements to the table surface; the placement of elements (tokens) on the table (constraint) creates the TUI environment by associating the tokens with their constraints. Within this relationship, the tokens themselves also operate as constraints on themselves and the other tokens around them. Depending on how you view the situation, the relative definition of each element can change significantly.

Within this TOC relationship, the urban elements have computational interpretation as they are manipulated. New elements may be added to the table (discrete manipulation) or existing elements may be shifted around on the table (continuous manipulation); both of these changes manipulate the state of the application and change the displayed traffic patterns.

Both Reality-Based Interaction and the TOC Paradigm create conceptual frameworks through which we can view the MouseHaus TUI. With theis analysis , we can better understand the role of this interface’s elements and better compare this interface to other projects.

Basically, it looks super fun, if only I was a music producer.

(photo from The Coolest)

Such a beautiful little GUI. It goes with this boarding pass:

Anyways, on to the real TUI marvels.

First of all, check out the phone. It splits into two pieces, one which can be an earpiece and one that can be a video-chat screen.

Apparently, one of the pieces can also be an augmented-reality-gps tool that shows you a floor plan of your immediate location with superimposed directions.

Even better, the two pieces connect back together to make a social media tool to set up meetings (with the help of the phone/GPS combo). When this flips back shut, it’s just a cell phone again.

Next: the most amazing keychain of all time. The phone (above) seems to also carry the qualities of the keychain. When they are set down on the interactive tabletop, every file necessary to your life appears on the table. The keychain is an insta-link to your entire online life, and everything you need appears on command. Here’s the tabletop with just the phone laid down:

Here’s what appears around the keychain:

And when the two are put together, the tabletop becomes an amazing collaborative workspace that knows what projects you’ve been working on, what your last correspondence was, and what files you’ll need.

This is cool too: an office computer station. All of the portals are just glass panes (until all of your information is displayed)… and these portals are everywhere.

What’s better, there is a super sweet mouse-esque tool that allows for multi-touch input, dragging, etc. It responds with different visualizations based on touch.

It is a little weird to me that they’re sticking so clearly to the mouse/keyboard system, but I suppose that’s to be expected. I think it’ll be a long time before everyday consumers change their dependence on a mouse and keyboard.

This is great too- a remote control for your entire life. You can control your personal computer-screen-on-the-wall (which, by the way can show you information about your house, or the weather, or your to-do list, or anything, really).

It, too, is made of glass.

Here’s what’s really cool- it can “photograph” (select) a project…

and MOVE IT to another interface location, where it will display for your use.

Speaking of that newspaper, I’ve been saving the best for last. It’s a practically single-page newspaper (which kind of seems like the Times Reader to me) that changes to display different stories. It’s like a giant… eInk paper interface.

You can slide stories off the page…

Or select them for more information.

Man, it’s just all SO impeccably beautiful. I suppose that the TUI elements aren’t exactly new to the scene, but they’re very well rendered and the ways that they interact is very well thought-out. I especially want that keychain and that newspaper. Oh, and this secretary-y space, for when I’m scheduling out my life:

And how cool is this, for a last thought? The room knows where you’re headed and projects an arrow going in the right direction on the floor in front of you. Amazingly useful, especially for the more directionally-challenged among us. (I’m good with a map, but I’ve traveled with quite a few people who aren’t… and they could definitely use that arrow.)

What I’m trying to say is (in true 30 Rock fashion): I want to go to there.