Deutsch 
English (UK) Chapter 7: Evaluating display properties
The previous two chapters investigated how multiple users collaborate with each other when connected from distributes sites. They demonstrated that interfaces and displays can hamper closely-coupled collaboration. This raised the question which factors in particular are responsible and this chapter aims to answer this by studying how a single user in the Virtual Gazebo benchmark is affected when using different interfaces and displays.
Hypothesis-3: display properties determine effectiveness of collaboration
The study presented in this chapter analyses a similar task (see Chapter 6) carried out by a single user, so that factors affecting collaboration can be isolated. The aim is to understand the impact of using a CAVE-like display on user-to-object interaction, so that it is possible to isolate this from previous results that showed an improvement in multi-user cooperation through shared objects. This will tell us if the advantage comes from more natural interaction with objects or more natural interaction with other participants through and around objects.
Discussion
This section discusses why perceived and measured performance was different, what the FOV has to do with user locomotion & navigation and why the interaction technique influences the user collaboration and performance.
One clear observation was the difference between the perceived performance and the time needed to complete the task. The contradiction of these results may be explained through the relationship of the perception of being there, immersiveness and interaction technique (Figure 7-1).
The results of this study show significant differences in perceived presence for all displays. The same tendency can be seen for performance, FOV, missing touch and interface problems. Although those tendencies are not as strong as for presence, they show that the more one is immersed and engaged the higher is the feeling of being there. Presence is not something that can be clearly measured, but is a feeling created by a number of factors (Chapter 2 and 3). Those factors, like immersiveness, naturalness of interface and ease of interaction, all appear to contribute to a feeling of being there. Small differences of perception (between displays) for all those factors have a profound influence on the perceived presence. This also explains the difference between perceived and measured performance. If one feels more engaged and present, time will seem to pass quicker and the user’s own activity will enhance the feeling of performance. This can also be seen in the reaction of users, who consistently mentioned that the use of the immersive display was much more enjoyable (“Great experience and fun!”) than the desktop.
One objective in this study was to determine how much the FOV would influence task performance. The hypothesis was that with a wider view frustum the task would become easier and increase performance as the scene is more visibly accessible and therefore objects can be spotted quicker. In contrast to the desktop, both immersive displays are similar in the way the user interacts, however the FOV is their main difference. Differences can be seen in the data and observations gathered during this trial. At the workbench, motion paths show clearly longer paths for locomotion in comparison to the CAVE. In addition, the observation during the trial was that on the workbench the joystick was used more often to attain an object as compared to the CAVE, where physical walking toward an object was easier and only longer distances needed the use of the joystick.
Orientation through rotation is a behaviour people do all the time and in virtual environments joysticks (keyboard) are used to achieve a full rotation. Exceptions are fully immersive displays like HMD or 5-sided CAVE, which have a natural rotation of 360º, independent of the FOV. This means that with an HMD the user may not need to use a joystick to rotate, but rather uses his own body [Bowman et al., 2002a]. In contrast, the desktop has the smallest FOV of all the tested displays, yet the locomotion recorded during the trial was very low. The reason for this appears to be based upon the ray-casting manipulation of objects. The user did not need to get close to the object, but could do everything from a remote place, from which the whole scene could be observed. However, in previous trials this behaviour was reason for complaint as other collaborating users could not see the correlation between a user and the object they were interacting with ([Hindmarsh et al., 2000; Roberts et al., 2004b], Chapter 5). In addition, working from a remote place is only possible if the given environment supports such behaviour, for example, a world without walls or very large rooms.
Therefore, in a subsequent trial to this study users were asked to repeat the task on the desktop, first from a several metres away in the virtual environment (using ray-casting) and then from a location adjacent to the object (virtual-hand). The result was that the time taken to perform the task doubled for the close-up trial (mean of 9.4min). Therefore it can be hypothesised that if we try to improve the collaboration between users by allowing only close-object interaction, performance time for desktop user will drop due to their limitation in FOV hence resulting in extended locomotion time to orientate (see Figure 7-1). In addition, a study from Steed et al. [2005], that compared ray casting and virtual-hand interaction on HMD and CAVE displays, found that virtual-hand is superior for selection and manipulation of objects.
This study looked into influences on a single user task. Those influences sustain in a co-presence situation and may even enhance. For example, problems with interface and manipulation of objects can interrupt the workflow in a closely-coupled situation [Hindmarsh et al., 2001b]. The previous studies showed that people have a higher perception of the performance of an immersed user, independent of the assessment of themselves or others (Chapter 5 and 6). They also show a significant difference between two immersed users and a desktop user, which was related to the ease of manipulation and navigation.
Form the above discussion it is clear that highly immersive displays have a positive effect on users comfort und usability of the VR environment. It also indicates that immersive displays support the workflow and thereby create a performance increase. A frequently asked question is how this can be applied to other fully immersive displays like HMD’s. The comparison of IPT and HMD has surprisingly not been widely studied. A lack of user studies is reported by [Manek, 2004] and a complete absence of direct comparisons reported by [Steed et al., 2005]. The few studies available are restricted to selection, manipulation or locomotion tasks.
A related study by Sander was using a novel approach of placing a HMD in the CAVE-like display [Sander, 2005], it allowed participants perceiving the environment through either, and to be observed moving within the IPT holograph. Combined with sharing the same tracking and camera systems, this provided a direct comparison of tracking measurements, interaction behaviour, perception, video and other observational data. The experiment studied participants moving objects around a living room setting initially on a level surface and then whilst varying the height and shape of the walking surface through raised boards. Performance in the synthetic environment, using both display types, was compared to that in a physical mock up of the living room.
Using the same tracking system and space for both immersive displays has the advantage that tracking data can be matched. Figure 7-2 shows a series of such trajectory graphs where the CAVE-like user had no real problems to balance along the board, this was much harder using a HMD.
 |
 |
Observations showed that people behave quite naturally in the CAVE as opposed to HMD usage. This can also be seen by looking at the tracking data, as Figure 7‑11a shows for the CAVE a smother paths verses erratic paths at the HMD (Figure 7‑11b). It was argued that this is due to the different properties of both displays with a lower field of view and only a partial visible virtual body in the HMD VE. Unlike the CAVE, participants using the HMD noted that they perceived a difference of the virtual and real hand position. This error in preproiception [Mine et al., 1997b] was due to a limited accuracy of the used magnetic tracking system. Such problems are less notable using a CAVE as a user has his own body as a reference. Higher precision in tracking with a higher quality display may reduce such problems when using a HMD.
Summary
The measurement of performance is always difficult to achieve, as it depends on the way measures are taken and how quantifiable a task is. This applies as well for performance in a virtual environment. One may be able to measure the time it takes to finish a task, but as this study shows this is not necessarily reconciled with perceived performance.
Results from combining this chapter with the last two suggests that people always think they are performing better in a “step-into” CAVE-like display than on a “look-into” desktop or “reach-into” displays. However, objective measurements of task performance indicate such improvements in object focussed collaboration but not for single user object interaction.
The results of this chapter also clearly underline that immersive displays have a significant positive impact on the perception of presence and realism. Although chapter 2 discussed this in detail and made references to previous work with similar results, this work is contributing to the discussion from the approach of close object interaction. A frequently asked question is how this can be applied to fully immersive displays like HMD’s. Placing a HMD in the CAVE-like display [Sander, 2005] allows participants to perceive the environment through either, and to be observed while moving within. It is likely that the outcome of those test is determine by the quality of the HMD used. In this particular case a non state of the art HMD seemed to have a clear effect on people’s behaviour.
The studies discussed have shown that different factors lead to an increasing perception of presence and performance. Factors such as FOV, manipulation technique and navigation, may also influence a user’s interaction and its effect on other participants in a collaborative task (e.g. no fragmented workflow). In addition, different displays influence the way users interact and behave to a degree that makes working in the environment very difficult or even causes sickness. This means that a display has to be suited to a task and that the design of such a task ought to incorporate the display and interface properties. A more detailed discussion about various influences on a task concentrating on closely-coupled collaboration will be given in the next chapter.