iOS

Recently, a few developers were having a discussion on ADN about a promo video for a new iPad app. Uli Kusterer raised the point that the video was missing something: fingers. It was hard for him to tell what was a tap, and what was something that was merely happening as normal in the app. Since the app in question is a game with a lot of animation and things popping in and out, it’s easy to see how things might get confusing.

Fingers are a really important part of an iOS app demonstration video. There are definitely a few ways to make that happen (and there was some speculation about how it might be done better in that ADN conversation). I’ll get to some techniques you might use later, but first, perhaps we should talk about why it’s so important.

Why Fingers?

When you take a screencast of a Mac, unless you’re doing something really fancy, your cursor appears in the video. The cursor is also known in certain circles at the pointer because, well, it points to things. You can even wave it around and gesture and point to things that you’re not even going to click on. It’s the digital extension of that digit on your hand that you point to things with all day, every day. Always pointing at things, you are.

The cool thing about pointing and gesturing at stuff is that the viewer’s eyes find it really useful, so they can anticipate. The human eye and brain are a pretty good team when it comes to following the motion of object, and when they have that, it’s a lot easier to follow your cursor from one action to the next, than if your cursor, say, suddenly jumped from one side of the screen to the other, and you had to search around to figure out where it had gone.

However, when you take a straight screencast of an iPhone or iPad app, your finger doesn’t show up on the screen. You can’t gesture to things. The viewer’s eye can’t track where you’re going next, so it’s really easy to get lost and frustrated, trying to figure out what’s going on.

Giving Your App the Finger

Now, when I talk about fingers in a video, I don’t necessarily mean a literal human finger. It could be a simple circle representing a finger pad. The important thing is that there’s some way for your viewer to see where you’re going, and be able to follow along.

Let’s take a look at some options for getting that in there. I’ll try to go from least to most complicated, and give you an idea of the pros and cons of each technique. I don’t think any one technique will be right for every app, so it’s good to know your options.

Run your app in the simulator and use SimFinger

• Maybe you think SimFinger is creaky and old, but you know what? It still does what it needs to do, and does it well.
• SimFinger replaces your Mac’s cursor with a simulated finger pad that changes from convex to concave when you click, representing a tap with your virtual finger.
• It also gives you iPhone and iPad frames to put around your simulator screen, if you’re into that kind of thing.
• Con: It’s the simulator, so it may not look and act exactly the same as the device, or may not be viable if you have features that don’t work well in the simulator. It will also be hard to use this for games with crazy fast multi-tapping action. Or any multi-tapping action at all, really. Your mouse is really only equivalent to one finger, or maybe two if you use the pinch mirroring features in the simulator.

Run your app in the simulator and use ScreenFlow

• You can also do this kind of cursor replacement with ScreenFlow, and maybe you’re already using ScreenFlow to record anyway. In fact there’s a setting for the mouse pointer called Circle – Light that looks remarkably like Ye Olde SimFinger.
• You can adjust the transparency of the finger cursor, so that you can always see what’s underneath.
• Con: Still the simulator.

Shoot a video of a human finger actually using the live app on a device.

• You capture the video all in one go, so the shoot itself seems pretty easy.
• Con: The shoot is actually terribly difficult because the color temperature in the environment is very hard to match to the color temperature of the display. And even if you have a fancy bank of LED lights where you can dial up different light temps, it’s still really difficult to make the live app on the device look decent because the contrast and brightness of your live scene is never going to match the screen very well.
• Also, there’s a big LED light bank shining on your device, so: GLARE. No, you can’t keep the light off the screen. Your finger needs it.
• Also also, the video you shoot of a screen is never going to look as clean and sharp as a raw screen recording.
• Tip: You can maybe get away with it, if you want to go for the silhouette effect. Very dim lighting or just accent lights on the real world, and adjust your camera for best exposure of the screen.
• Con: Your finger, and maybe the rest of your hand, obscures the screen, so it might still be hard to see what’s going on.

AirPlay to your Mac and record the screen with TouchPosé

• TouchPosé is a set of classes you can add to your app to make semi-transparent circles appear when you touch the screen.
• Con: It only registers touches, so I’m not sure I can recommend it. There is nothing for the eye to track from one tap to the next, so I don’t think it’s much of an improvement. Perhaps if you’re doing a live demo or a presentation somewhere on a projector, and you can explain to people what you’re doing in more detail, but in a fast-paced demo video, it’s easy to lose track of the little tap circles, especially if it’s a game or other app where random circles popping up could well be part of the app’s animation.
• Caveat: I haven’t used this myself — only seen it in other folks’ videos. Perhaps there’s an option to customize it in such a way that would make it more clear?

Use TouchPosé to create a skeleton for the video, and animate your own finger.

• You could add an animation on top of the TouchPosé style video, so that you can help the viewer anticipate. The circles that appear let you know when a tap happened, and give you timings for when you need to animate your pointer overlay into that position.
• The animated pointer could be anything — matching the TouchPosé circle exactly, just a simple SimFinger-style dot, or any other element you could move around. It could even be a cut-out photograph of a hand with a pointing finger, and you could make it semi-transparent so it doesn’t obscure the rest of the screen.
• Con: A lot of work. Probably requires learning After Effects or Motion if you’re not already a video guy like me.

Shoot with green screen on the device and record the app separately; sync them up in editing and composite

• Looks great if you do it right. You can balance the colors and luminance between the real shoot and the screencast, and create something that, while it never really exists that perfectly in nature, makes for a beautiful video.
• Cons: Toil. It’s a royal pain in the ass to pull off. You need to figure out where your taps will be, and record them so that they sync up with the video. Not too bad if you’re just cutting away for a “real world” demo moment or two, but if you’re doing a full-blown app demo, it can be deadly.
• A green screen on the device is hard to balance when you’re shooting, to get a good, clean key. You’ve got to get it bright enough that you can pull a good key from the green, but not so bright that the green spills onto the demonstration finger and half of your finger disappears along with the green.
• You might consider just tapping on a black screen, and rotoscoping your hand to create a matte, but that can take a long time and look unnatural if you’re not skilled at it.
• We did both of these techniques when I worked on the videos for Briefs recently. I’ll talk about the process in more detail in my next post.

Shoot with an opaque green screen overlay on the device running the actual app

• Peter Hosey and Mike Lee came up with this idea during the aforementioned ADN discussion. I haven’t tried it, but I think it’s pretty darn clever.
• The real trick is, what would you make the overlay out of? It needs to be opaque enough to give you a nice clean green, but thin enough that you can transmit touches to the device through the overlay.
• Jose Vazquez suggested including a touch-transmitting green UIView on top of the real views of the app, when viewed on the device, but removing that green for the AirPlay output that you would record, so then you wouldn’t need anything physical.
• If you’re using a physical green material, it also might need to be thin enough, if you’re filming the live action off-axis, to not show any significant thickness, or edge, that would need to be removed in compositing. However, I think Mike and Peter were thinking it would be shot dead-on, and the fingers simply composited over the screencast — not composited back into the live action, like we did for Briefs.
• You would then record both the camera shot of the live fingers and the app’s screen video (via AirPlay) simultaneously, and composite them together. You could even make the fingers semi-transparent to show the full app screen while you demo the app.

All interesting ideas, with varying degrees of preparation, difficulty, and results. What method will you use on your next app video? I look forward to hearing your thoughts.