'Twas the Night Before the Alpha Review...

...And all through the lab, I am still working.

This past week has been me in a ball of stress because everything is happening this week. Projects are due, quizzes to be taken, and a review to go through. I think I finished all my slides for tomorrow, but I just need to make them into a movie and time all the slides.


I did two more sketches though.

 In this sketch, footprints are displayed along the path of motion (a thick line). This doesn't present the entire view of the motion, unfortunately.

This sketch is meant to be a proximity chart for joints (got the idea from E.J. Marey's graphic on train schedules). I'm not sure if this is the easiest to comprehend or if it's even a good visualization, but the idea is to display when joints get closer to each other. So in this example, I tried to depict a running cycle, so from from 0 to 10 (these numbers are arbitrary), the left arm and left leg move closer to each other, and the right arm and left leg move closer to each other. I think my graph isn't perfect (there are a few mistakes and some stuff can be clarified), but that's why this is just a sketch.

I will blog next week about comments from the review, and I'll try to revise the sketches and incorporate feedback, but it might be a bit hard for me to find time to do that since I'll be in Palm Springs attending TEDActive 2011. Commence pee-in-my-pants-excitement-dance. Geez, I'm so nervous.

Yippee, more sketches

This sketch shows a bird's eye view of the capture space, with the subject's starting and ending position identified through colors/icons/whatever. There's a line in varying widths tracing the subject's movements throughout the space: the thicker the line, the longer the person remained in that position.

This sketch is a combination of one of my previous sketches and this youtube movie (Autonomous Motion Synthesis through Machine Learning) that Joe showed me yesterday. Pretty much everything is indicated in the sketch. However, this might be too large of a view to include in search results - maybe it's more suitable for an advanced user? It also allows you to view certain joints or tun off certain joints.

This is just a brief sketch of a possible UI for search results: at the moment, it's really bulky. Hopefully I can condense it down. Right now, the pieces of information I'm including in each result are:

• image of one frame
• framerate
• description
• tags
• available files for download
• visualization
• view more visualizations link (for advanced users) - this may slide down an additional panel, or it might redirect to a new page for just that search result?

Since the alpha review is next Friday (!!!), the things I have to accomplish before then are:

• more sketches!!
• more UI sketches
• make my alpha review video/presentation
• CRAM-type evaluation thingy

Sketches

So this past week, I did some sketches for possible visualizations. Some of them might be a little out there, and some others might be more feasible.

Sketch 1:

The idea for this visualization is that you have a skeleton shown at Frame 1, and its major joints are labeled with dots. So in order to prevent showing many skeletons at various frames, you instead plot the joint position at a frame and connect it to the joint position at the previous frame to create a kind of continuous line of motion. This hopefully makes it easier to see which body parts move the most, and which stay stagnant. Of course, my drawing is super bad and I did it by random, so we'll actually have to load a motion into it to see how it looks. Another idea was that if you were to hover over any part of the group of lines, you could see what the skeleton looks like at that particular frame. Perhaps, if it scales well, you can also choose to view it from the front/back, which could present a very different view.

Sketch 2:

This sketch is mainly for the skeleton (ie, what to display at a given frame). The idea was to place arrows indicating direction of movement at each of the major joints, and the direction would be determined by the difference in position between this frame and the next frame. Problem: the arrows might appear visually cluttered though, so they would have to be thin and lightly colored. There are various ways to display the arrows (length, shape, opacity, width) so that they don't appear as intrusive. Maybe the skeleton could take a back seat and let the arrows be the focus.

Sketch 3:

This is just your typical frame-by-frame display of motion. Standard.

Sketch 4:

This concerns the skeleton as well. Maybe for each frame, we can highlight the parts of the body that see the most motion, or the part of the body whose position changes from the previous frame to this frame. The highlights could also be on a spectrum: lighter color = less movement, darker color = more movement. These skeletons can be displayed side by side (like in Sketch 3).

Sketch 5:

Now this one is more of an overlay of skeletons at each frame (or intervals of frames). Each skeleton would be translucent so you can see which body parts stay in the same place (indicated by a darker color), or which body parts move around more (indicated by lighter colors). The idea is to make this look like a slowed down motion clip, where you see traces of the frames before. Onion skinning, is that the term?

Sketch 6:

This is another way of displaying the skeleton per frame. Instead of seeing the skeleton as a whole, maybe someone would like to focus on a specific body part, say the legs. For each frame displayed, we could either show just the legs (resulting in a really weird torso-less body), or we can show the whole skeleton with with the rest of the body in a light color, and the legs in a dark color. This allows you to focus on the body part that concerns you. For example, if you're searching for motion capture data of an Irish dancer, then you probably would want to see movements of the legs and not the upper body, since the upper body stays still most of the time anyways.

So for next week, my plans are to:

• Iterate on these sketches
• Start sketching the website (results page) layout
• Look at the asf/amc parser code that Joe will hopefully send me!

More Ideas

So unfortunately, this week has been a little slow for me due to various reasons, and I haven't worked on my project as much as I had hoped. One interesting idea (among several) that surfaced during a meeting with Orkan Telhan, David, and Joe on Tuesday was the idea of stenography, and how it may apply to visualizations. Stenography, if you're not aware, is the process of writing in shorthand. So the question is, is there a shorthand way of displaying motion? I suppose this is the question I'm trying to answer, but phrased in a different way.

Joe was also kind enough to email several major animation/effects studios to ask them how they organize their motion capture data, and how they pull out specific shots. So far, only Sony has responded, and they pretty much said nothing of relevance to my project :/

Basically, next week I plan on doing the following:

• Doing more research
• Sketching more motion visualization ideas (including things that may be "out there")
• Sketching a preliminary search results UI
• Deciding on what technology to use for coding the visualizations
• figure out how to use the code that the lab already has for parsing asf/amc files.

I also set up webspace on my own domain for temporarily hosting everything. The URL is movis.yiyizhou.com, so you can go there if you want, but all it says is "hello" at the moment. Not very exciting.