Distant freehand pointing and clicking on very large, high resolution displays

COMMENT
1. Comment on Drew's blog


SUMMARY
The paper discusses system for pointing and clicking interaction with very high resolution displays from a distance. The pointing interaction enabled by this system is absolute (as if the user is pointing with a laser beam emitted from his hand) as opposed to pointing using a mouse where position of the mouse and the position of the cursor on the screen are not direct mapped. Clicking and grabbing are performed by using hand gestures. Prototype:- User wears reflective markers on the hand and fingers. The large display used is rendered using a cluster of 18 PCs. The paper explores two clicking gestures - AirTap (tapping with the index finger in air) and Thumb trigger (moving the thumb in and then outwards from the index finger). However users found the thumb trigger tiring and the authors adopted a style similar to air tap as the style of interaction. The authors also explored 3 pointing techniques:-
1. Ray casting. The cursor is placed at a point where the ray coming out from the tip of the intersects with the plane of the screen. Air tap method of clicking could not be used with this method due to the induced jitter.
2. Relative pointing and clutching: In this method the open hand is used for relative cursor control. Clenched hand is used to disengage the hand from the pointer and recalibrate it.
3. Hybrid technique:- Hybrid method uses ray casting for recalibration, whereas cursor movement is done via relative hand movement.
In their study, there was no significant difference between air tab and thumb trigger. Ray casting as a method of pointing resulted in high error rates and uncomfortable interaction. The evaluation was conducted with 12 participants. Users were asked to perform two tasks - Transition task and sequence task. Ray casting was found to be generally faster but also very error prone. There was no significant difference between Ray and Ray to relative. participants however, preferred 'relative' because the clutching action was similar to lifting a mouse (on which they were already trained).


DISCUSSION
Overall the paper provides an in depth study of different interaction techniques with large displays. Regarding the jitter induced by air tap and the its claimed incompatibility with Ray casting, I feel a data glove would be able to solve this problem. I feel that when users are pointing using ray casting the movements involved are in the forearm and the shoulder, while the joints in the hand remain fixed. However, during clicking the shoulder and the forearm remain fixed whereas the joints on the index finger move. So side effect Air tap clicking could be removed by ignoring any changes to cursor position when only the joints in the hand are moved.