Spatial learning, image maintenance, and virtual environment

When acquiring knowledge about an environment, spatial information is often accompanied by visual stimuli properties. Some studies focus on precise location of stimuli (^{Kelly & McNamara, 2010}), while others consider the examination of visual stimulus properties (^{Hahm et al., 2007}; ^{Roupé et al., 2014}; Dong et al., 2021).

The process of spatial cognition is closely related to spatial thinking, an important skill that enables individuals to comprehend their surroundings, engage in mental modeling, and structure problems to reach solutions (^{Metoyer et al., 2015}); it is also the cognitive process of manipulating and analyzing spatial information (^{Ishikawa, 2021}). Route estimations are linked to the term 'image maintenance,' a cognitive process involving the completion or prediction of spatial properties during mental imagery. Despite the absence of a full view, individuals utilize visible cues to navigate, emphasizing spatial visibility. The process of 'information completion' and the estimation of spatial information at invisible points are accomplished through the mechanism of 'image maintenance' (^{Kosslyn & Shin, 1991}). When the desired or intended image is inconsistent during the mental imagery process, an attempt can be made to transform it by changing or restructuring its features, then, through image maintenance, the imager endeavors to align the transformed image with the intended one by adjusting its content or characteristics, ensuring continuity in the imagery process until its conclusion or cessation (^{Cumming & Eaves, 2018}).

The extraction of metric information and spatial configuration during navigation, such as the line's direction, distances between landmarks, and turns, is closely associated with encoding in working memory (^{Hegarty et al., 2006}). Cognitive representations of space formed during the learning process may extend beyond the initial learning standpoint, even with information from various viewpoints (^{Van der Kuil et al., 2021}). The preservation of visual mental image equates to its recurrent recreation due to the inherent difficulty in sustaining such images over time (^{Kosslyn & Shin, 1991}).

The acquisition of spatial knowledge can be obtained through direct experience within the physical environment (^{Qiu et al., 2020}), and indirect experience with the help of various media and virtual environments. The most accurate way for acquiring spatial information remains direct experiences within the physical environment with virtual environments serving as the second-best alternative for learning (^{Richardson et al., 1999}).

The acquisition of spatial knowledge from virtual environments (VE) aims to understand their space-related characteristics and serves as a substitute for actual environments. Experimental studies, including urban design simulations, analyses of legibility and wayfinding analyses, emergency response planning, and investigation into spatial usability are frequently conducted in virtual environments. These environments serve as valuable testbeds closely resembling real-world situations enabling participants to collect both spatial and visual data.

3D volumetric isovist

The term ‘isovist’ was first coined by ^{Tandy (1967}), although it appears to be an extension of an older concept used by geographers to define the area visible from a specific position in the landscape, typically from a high point (^{Dalton et al., 2022}). The concept closely tied to isovist’s psychological and spatial features may have originated with ^{Gibson’s (2014}) ‘ambient optic array,’ but it was ^{Benedikt (Benedikt, 1979}; ^{Davis & Benedikt, 1979}; ^{Benedikt & Burnham, 1985}; ^{Benedikt & McElhinney, 2019}) who drew attention to isovist’s architectural importance for the first time.

An isovist is the entire volume generated by a hypothetical point with the ability to ‘see’ 360°. Elements within a space that obstruct visual capability and therefore affect accessibility and wayfinding ability, such as exterior walls, partition walls, and atriums, alter the boundaries of the area defined by the isovist (^{Benedikt, 1979}).

While isovists provide accurate measurements of the geometrical attributes of an individual’s point of view and viewshed, their planar representations are used for navigation behavior and wayfinding research (^{Wiener et al., 2007}; ^{Hölscher et al., 2012}; ^{Meilinger et al., 2009}; ^{Emo et al., 2012}). However, isovists are volumetric entities, and although they are mostly reduced to 2D projections on orthographic planes (e.g., plans, sections, etc.), the actual experience of ‘seeing’ can be more precisely represented if the isovist is modeled conically rather than triangular.

With advancements in computational technologies, the application of 3D isovists has accelerated (^{Krukar et al., 2021}). Various methods for calculating volumetric isovists have been introduced by ^{Derix et al. (2008}), exploring the potential effects of open and constrained spaces on navigation. ^{Kim et al. (2019}) proposed a novel metric called the ‘Openness Index’, capable of capturing the structures of 3D space around an observer. Similarly, ^{Fisher-Gewirtzman (2016)} presented a 3D visibility model that considers elements of the urban environment such as roads and trees. As ^{Lu and Lu (2019}) demonstrated, 3D models, especially for multi-floor interiors, offer significant advantages over 2D isovist equivalents. 3D isovists have also been used for the analysis of orienting information during rotational locomotion (^{Kondyli & Bhatt, 2018}) and wayfinding (^{Kondyli et al., 2018}).

For this study, isovist volume is used as a rational instrument for measuring the spatial visibility of participants in the virtual environment, as virtual environments resemble real environment spatial learning conditions. Analyzing the size of the isovist field throughout the route, where the size of the isovist is correlated with visual capability (i.e., isovist volume is low when visibility is limited and vice versa), allows to reveal the relationship between ‘image maintenance’ and ‘spatial visibility’.

Development of the model

The subsection details the preparation phases for a 3D virtual town covering a blueprint area of 24,000 m². Initially, 2D simple map base layouts were created using AutoCAD software. Previous studies have shown that using 3D models instead of 2D drawings not only positively impacts wayfinding performance but also increases success rates for individuals with low spatial cognition (^{Verghote et al., 2019}). In the second phase of model development, the 2D layout was transferred to the Rhino/Grasshopper environment for 3D modeling and 3D isovist analysis.

For individuals navigating unfamiliar destinations who need more prior information beyond their immediate surroundings, reliance on internal representations for wayfinding support is needed; moreover, they must still make navigational decisions based on the limited available information to reach their destination (^{Gath-Morad et al., 2021}), Thus, a virtual environment was designed instead of using a model of an existing location to ensure that all participants had no previous experience with the place. The virtual town comprised rectangular buildings, each with a random floor height of one to five floors set at three meters each. Research conducted in the fields of environmental psychology, urban planning, and design demonstrates that the physical characteristics of the environment shape individuals' understanding of spatial features (^{Ahmadpoor & Smith, 2020}). To provide a realistic environment within the model, some elements, such as diverse façades, trees, scaled human figures, and a dynamic sky effect, were integrated, while further detail was enhanced through variations in six building blocks and public furniture (Figure 2).

Source: Authors (2024)

Figure 2: Various viewsheds from the virtual town along the main route  

A high-resolution model was provided to participants for the experiment, which facilitated easier recollection of spatial features due to increased detail. However, certain details such as door codes were intentionally omitted to prevent participants from becoming distracted during the experiment. To maintain stimulus control, most facades, except for a few buildings, were kept white without added textures aligning with the same logic.

For the interface of the navigation task (further detailed in Section 2.3), the Unreal Engine was chosen allowing users to navigate smoothly in the VR environment. Camera settings and angles were configured in the model file sent to Unreal Engine ensuring a realistic first-person view aligned with that of a person with a height of 1.70 meters. A collider was integrated into the ground within the program granting users control over the first-person perspective as they moved through the model. The VR environment was displayed on an 18-inch laptop screen throughout the navigation task.

Development of the 3D volumetric isovist

The 3D isovist volumes, based on the first-person human viewpoint and the environment, were created in the Grasshopper environment. Initially, a sphere was generated to completely encompass the entire virtual town, and then subtracted from the town’s ground to create a hemisphere with maximum visual capacity. A point was positioned at a height of 1.60 meters, representing the user’s experimental viewpoint at the beginning of the path, illustrating the eye level of a person with a height of 1.70 meters.

To create a 3D isovist volume, 360° rays were projected from the point corresponding to eye level. Building blocks were introduced as two rows of obstacles. The first row consisted of buildings around the user's path and were in the primary field of vision while walking, while the second row represented the areas around the walking path visible with head movements. In accordance with the obstacles created, two separate 3D isovist volumes were modeled. The purple isovist volume described the user's main visual field, while the pink one described the areas the user could see while looking around (Figure 3).

Source: Authors (2024)

Figure 3: Development principle of the 3D isovist volume  

Changes in the two isovist volumes could be observed simultaneously as the point defining the eye level moved (Figure 4a). All volumes were overlapped to reveal the most visible areas for both the main and secondary areas generated by the 3D isovist (Figure 4b).

Source: Authors (2024)

Figure 4: Main and secondary areas along with overlapped isovist volumes  

The 3D isovist is used as an indicator of image maintenance capabilities in this research. As previously mentioned, image maintenance is a cognitive process that involves completing semi-visible areas and visual data acquired from the environment, making it directly related to the dynamic viewshed of the subject. The 3D isovist allows us to represent and, more importantly, calculate the viewsheds, which are a crucial component of the image maintenance process.

Case study

Participation in the experiment was entirely voluntary, with all participants either having normal vision or wearing appropriate medical glasses. Ten Ph.D. students took part in the experiment, comprising five females and five males, aged between 27 and 34 years old (with a mean age of 30.4). They all had a background in design education and were experts in various specific fields, including urban design, architecture, and industrial design.

Each participant was tested individually, with the experiments conducted in the same order for all participants. The experiment consisted of three main steps: the ‘Santa Barbara Sense of Direction Test (SBSOD Test) task’, the ‘navigation task’, and the ‘sketch map task with think aloud protocol’. The tools provided on the table included a laptop and a mouse for the navigation task, as well as a tablet and a tablet pencil for the SBSOD Test task and sketch map task.

Before the experiment commenced, participants were briefed on the tasks they would be performing. They were informed that they would be drawing a map and that their voices would be recorded for analysis during the think aloud protocol. Additionally, they were told that they could explore the virtual environment freely and speak aloud for as long as they wished.

(First step) SBSOD Test Task: The SBSOD questionnaire was employed to assess the spatial ability of the participants. Developed by Hegarty et al. (2002), this self-report scale is utilized for evaluating spatial ability. The test outcome offers a preliminary understanding of the participants' accuracy in perceiving the geometrical attributes of the environment (^{Burte & Hegarty, 2013}).

(Second step) Navigation Task: This step of the experiment was conducted within a virtual environment. 3D virtual environments typically feature an egocentric viewpoint, providing users with multiple perspectives on the visual information display (^{Hughes & Lewis, 2005}). However, in this study, only one viewpoint, namely the egocentric perspective, was utilized to prevent participants from viewing the entire layout. Participants navigated through a predetermined route, marked by a well-defined entrance and exit, highlighted with a red line, using the Unreal Engine (Figure 5a). Prior to commencing the experiment, participants were informed that they could walk and stop at any point to look around and perceive the spatial features of the overall space. No time limit was imposed for the navigation task.

(Third step) Sketch Map Task & Think Aloud Protocol: After completing the navigation task, participants were instructed to draw a 2D map of the 3D environment they had experienced. They were asked to create a cognitive map incorporating both spatial and visual details (Figure 5b). To gain deeper insights into the participants’ personal perspectives, the sketch map task was accompanied by a ‘think-aloud protocol’. The think-aloud protocol involves the concurrent verbalization of thoughts during a task (^{Ericsson & Simon, 1993}).

Source: Authors (2024)

Figure 5: The setting and hardware used during the tasks  

In alignment with ^{Goldschmidt's (1991}) methodology, the participants’ verbal thoughts are recorded and subsequently transcribed into written form. The outputs of the think-aloud protocol process were analyzed based on the phrases that the participants mostly used. Since the think-aloud protocol accompanied the sketch map task, it was possible to ascertain the structure of the participants’ thought process during the experiment. As participants were instructed to articulate their thoughts without any filtering during the protocol, the connection between the experiment and their thoughts is conveyed without being influenced by participant reinterpretation.

Results of the SBSOD test task

The outcome of the SBSOD Test is evaluated as recommended by the authors. Affirmative questions (1, 3, 4, 5, 7, 9, 14) are scored reversely, while the rest of the questions are scored correlatively. Then, the average score is calculated. The final score falls between 1 and 7, where 1 indicates a poor sense of direction and 7 indicates a good sense of direction. It's important to note that this score is only an indicator of the participant’s self-assessment and expectations of their navigation abilities.

The highest score among all 10 participants was 5.7 (Participant 7) and the lowest was 2.7 (Participant 3). The average score of the sample was 4.2: two participants (3 and 8) presented a poor sense of direction, six participants (1, 4, 5, 6, 7, and 10) showed a good sense of direction, and the last two revealed an average capacity (Table 1).

Source: Authors (2024)

Table 1: SBSOD test task results 

Results of the navigation task

The overall experience of the navigation task is assessed, focusing on the points where participants chose to stop. This analysis aims to identify significant points where participants gathered extra information by enlarging their field of view. Several parameters are considered, including ‘stops and gathering information (NAV_Stops, NAV_Looking_Around, NAV_Recurrent_Path_Estimation)’, 'viewshed character (NAV_Narrow Angle, NAV_Wide Angle)’, and ‘overall time spent (NAV_Overall_Time)’. These parameters help provide insights into participants’ behavior and decision-making processes during the navigation task.

The number of stops is meaningful data for understanding the experience of the participant in gathering visual information about the surroundings. Participants sometimes stop without looking around and continue on the path; however, sometimes they deliberately stop to look around. These two actions are numerically analyzed separately. Recurrent path experiences also give characteristics to each participant’s experience because some participants choose to walk the path more than once. These data are also shared in the relevant column. If the participant experiences the route more times than the number written but less than the consecutive number, it is marked with a “+”. For example, if the participant walked the whole route three times but did not finish the last round, “3+” is written.

The viewshed character is assessed based on the angles participants choose to create while looking around. Voluminous, thus wider angles, suggest that participants are gathering as much information as possible at that point, as the viewpoint allows for a broader perspective. This interpretation indicates the participant’s active engagement in exploring and gathering information from their surroundings.

Additionally, the overall time spent throughout the navigation task - as well as the sketch map task (further detailed in Section 3.3)- was an important factor in assessing the amount of information gathered during the navigation task.

The path that participants experienced during the navigation task consists of seven segments from Segment 1 (S1) to Segment 7; and eight nodes from Node 1 (N1) to Node 8; including the starting and ending points N1 and N8 (Figure 6a).

Overall, P_02 spent most of the time (09:40 minutes) on the navigation task, whereas P_033 completed the whole task in 03:30 minutes, marking the least time spent among all participants. Although P_09 spent a lot of time on the navigation task, wide viewsheds were not created, since the participant was not comfortable with the virtual interface. Therefore, P_09 was exposed to the least amount of data. Nevertheless, P_09 was mostly interested around S3, creating the widest field of view around N3. On the other hand, P_07 had the most recurrent experience of the path, stopping 35 times and looking around at 31 of them. P_07 was exposed to the most information along the path. Although P_07 created narrow angles of viewsheds more than wide angles, by moving and turning at the same time, P_07 gathered more information about the vicinity compared to other participants. (Table 2).

Source: Authors (2024)

Table 2: Quantitative data of the navigation task 

When considering the paths and viewsheds, it can be observed that participants focused more on segments between N3 and N8 compared to segments 1 and 2. In other words, wider fields of view were created throughout S3, S4, S5, S6, and S7, whereas S1 and S2 were analyzed with both fewer and narrower viewsheds. Since N3 defines a significant turning point of the whole path, participants created the widest fields of view around it (Figure 6b).

While P_01, P_02, P_06, and P_07 created wider viewsheds, P_04, P_05, and P_09 chose to experience the vicinity more with a narrower perspective. However, there were no significant differences between the narrow and wide angles created by P_03, P_08, and P_10 (Figure 6).

Source: Authors (2024)

Figure 6: Outcome of the navigation task analysis 

Results of the sketch map task

For the evaluation of the Sketch Map Task, the factors “path segment estimation (Path_Segment_Estimation)”, “path segment estimation rate (Path_Segment_Rate)”, “number of estimation path errors (Path_Estimation_Error)”, “number of landmarks-visual expression (Landmark_Count_Visual)”, “landmark visualization and properties”, “number of landmarks-written expression (Landmark_Count_Written)”, “written landmark properties”, and “the overall sketch map drawing time (Sketch_Drawing_Time)” were considered. “Landmark visualization and properties” and “written landmark properties” were considered during the general evaluation. However, it was anticipated that these properties would not yield meaningful relationships when converted to numerical values, so they were not included in the Pearson correlation analysis.

The examination revealed the inclusion of predictable routes within the drawings, with proportions ranging from a minimum of 25% to a maximum of 90%. P_01 was identified with the highest score in route estimation (%90) also, as the second with the most visual representations of landmarks on her map (Number:14). Conversely, P_09, who generated the fewest number of route predictions, drew the map with the most landmark visualization (Number:15). However, the landmarks depicted by this participant were predominantly concentrated along the red-lined path traversed, failing to align with the route estimates. Additionally, P_03, characterized by the highest incidence of incorrect route predictions, employed a grid-based approach to map definition, resulting in route estimations extending beyond designed pathways and between adjacent structures. P_07, distinguished by the most prevalent written landmark presentation, exhibited a 30% accuracy to predictable route depiction. Despite the absence of erroneous route predictions, the participant’s limited success in route estimation is attributable to an excessive focus on landmarks. As Rand et al. indicated, the information regarding the distance traveled within a space plays a supportive role in spatial learning of new environments (^{Rand et al., 2019}). Besides, understanding the environment is facilitated by the sequencing and integration of a series of landmarks and their locations, thereby forming the pathways constituting the surroundings (^{Van Der Kuil et al., 2021}). Within the scope of this study, as a general interpretation of the sketch map task result data, it was observed that the participants adopted divergent approaches in map creation, with some commencing by outlining routes while others prioritized landmark depiction, subsequently integrating routes in accordance with the depicted landmarks. Mental images can fade as soon as they are formed, so the memory representation must be continually reactivated (^{Kosslyn et al., 2006}). During the Sketch Map Task process, some participants maintained very intense visual representations of their sketch maps. This may be because they wanted to transfer and concretize their mental images on paper before disappearing immediately. The examples of sketch- map drawings are illustrated in Figure 7; Quantitative and qualitative data in Table 3.

Source: Authors (2024)

Figure 7: Examples of sketch-map drawings  

Source: Authors (2024)

Table 3: Quantitative and qualitative data of the sketch map task 

Results of the think aloud protocol

During the think aloud protocol, participants tended to give more environmental information at the point where they first began navigating, followed by details at intersection points along the route they walked, and lastly, they gave information about the completed area. Participants stated that they remembered the route and environmental features less during the route-drawing process. The most commonly used phrases were ‘building block’, ‘tree’, ‘multi-story building’, ‘woman’, and ‘canopy’ which matched the written expressions stated on the sketch map drawings. Expressions such as "the road turns right, it turns left", egocentric expressions such as "to my right, to my left" are also used when describing roads. Participants were more likely to provide details to which they were directed, expressing it as "it directed me, I walked here, the block here was very important to me, there is urban furniture here" were observed. The participants participated in the think-aloud protocol stage with the sketch map they drew, and some of them utilized their maps to aid their recall. Some participants placed question marks on the map for routes or landmarks they could not remember, then returned to update the drawing and the route progressed. Additionally, some participants expressed those differences in floor heights and variations in facades guided them. Participants also tended to adopted a systematic approach to street nomenclature, and label the streets as first street, second street, etc., and provided themselves with distance estimation data with expressions such as “as if this route was longer”. In the mental imagery process, the generation, inspection, and transformation of images take time, thus necessitating the preservation of images to facilitate the execution of these processes and the attainment of desired outcomes through the image maintenance (^{Cumming & Eaves, 2018}). The participants were observed to use the think-aloud protocol to help preserve images during the mental imagery process and remind themselves of the environment they experienced.

Results of correlation analysis

Analyzing the correlation matrix reveals significant relationships both within and across the tasks. These insights can help understand how different cognitive and perceptual skills interrelate, providing a deeper understanding of spatial perception. To identify meaningful correlations and derive significant insight between the variables in the three tasks: “Sense of Direction Task,” “Navigation Task,” and “Sketch Map Task,” Pearson correlation matrix (Table 4) was prepared.

Source: Authors (2024)

Table 4: Pearson correlation analysis 

According to the Pearson correlation matrix, significant correlations within and across tasks are explained below:

Within tasks:

Navigation task (NAV) correlations interpretation: There is a very strong positive correlation between NAV_Stops and NAV_Looking_Around (0.94), indicating that participants who stop more frequently during navigation are likely to spend more time looking around their environment. Also, another strong positive correlation exists between NAV_Stops and NAV_ Recurrent_Path_Experience (0.88), suggesting that participants who stop more frequently are more likely to experience recurrent paths during their navigation. ​Between NAV_Looking_Around and NAV_Recurrent_Path_Experience (0.84), there is a strong positive correlation, illustrating that as the amount of looking around during the navigation task increases, the likelihood of experiencing recurrent paths also increases. Participants who spend more time looking around are more likely to follow paths they have already traversed. The correlation between NAV_Time_Seconds and NAV_Stops (0.16) is a weak positive correlation, sugesting that there is only a slight tendency for the navigation time (in seconds) to increase as the number of stops increases.

Sketch map task correlations interpretation: Within Sketch Map results, there is an almost perfect positive correlation between Path_Segments_Estimation and Path_Segments_Rate (0.99), showing that participants who estimate more path segments also tend to have a higher rate of path segments, indicating consistency in their sketch mapping. The correlation between Path_Segments_Estimation and Landmark_Count_Written is approximately -0.195, indicating a weak negative correlation. It suggests that as the estimation of path segments increases, the count of written landmarks tends to decrease slightly. However, the relationship is not strong, implying that these two variables are not closely related and other factors might influence them. There is a moderate to strong positive correlation between Landmark_Count_Written and Sketch_Drawing_Time__Seconds (0.68), indicating that as the count of written landmarks increases, the time taken to complete the sketch drawing also tends to increase. This relationship implies that participants who write down more landmarks tend to spend more time on the sketch-drawing task.

Across tasks:

Sense of direction task (SOD) and navigation task (NAV) correlations interpretation: A moderate positive relationship exists between the participant's SOD_Average" and "Landmark_Count_Written, and the correlation coefficient is 0.613. This means that participants with a better sense of direction tend to write down more landmarks during the Sketch Map task. A higher Sense of Direction Task score is associated with a higher count of written landmarks. Participants who are better at orienting themselves and understanding spatial layouts are also more likely to identify and record a greater number of landmarks.

Navigation task (NAV) and sketch map task (sketch) correlations interpretation: The correlation coefficient 0.714 between "NAV_Looking_Around" and "Landmark_Count_Written" indicates a strong positive correlation. This means that as time spent looking around during the navigation task increases, the number of written landmarks also tends to increase. Participants who spend more time looking around during the navigation task tend to document more landmarks in written form, suggesting that looking around more frequently may be associated with a heightened awareness and recognition of landmarks, leading to a greater number of landmarks being noted down. The correlation coefficient 0.764 between "NAV_Time_Seconds" and "Sketch_Drawing_Time__Seconds" illustrates a strong positive correlation. Participants who take longer to navigate through the environment also tend to spend more time drawing their sketches. This could imply that the time invested in navigation is related to the time invested in accurately representing the environment through sketches. Longer navigation times might lead to more detailed mental maps, requiring more time to translate these maps into sketches. There is a strong positive correlation between NAV_Recurrent_Path_Experience and Landmark_Count_Written (0.741). Participants who experience recurrent paths more frequently during the navigation task tend to document more landmarks in written form. This suggests that the repetition of paths helps reinforce the memory and recognition of landmarks, leading to more landmarks being noted down. The correlation coefficient -0.429 between "NAV_Recurrent_Path_Experience" and "Path_Segments_Estimation" indicates a moderate negative correlation. This means that as the experience of following recurrent paths during navigation increases, the estimation of path segments tends to decrease. Participants who experience recurrent paths more frequently during the navigation task tend to estimate fewer path segments. This could imply that familiarity with the environment gained through recurrent paths simplifies their mental representation of the path structure. Both accuracy and speed of perception in wayfinding and spatial orientation tasks are enhanced by increased familiarity with the environment (Kim et al.,2021). The repeated traversal of the same paths may reduce the perceived complexity of the environment, resulting in lower estimates of path segments.