How to design user interfaces for searching and retrieving 3D models in a digital humanities context? This was the question investigated in our short paper entitled “More than just a Picture” – The Importance of Context in Search User Interfaces for Three-Dimensional Content [1] (PDF), published in the proceedings of the CHIIR conference 2020. Since the conference couldn’t physically take place due to the COVID-19 situation, we briefly present our work in this blogpost.

Background

This paper was inspired by the growing number of historical environments which are reconstructed in three dimensions, for instance the 17th century houses in Amsterdam within the Virtual Interiors project. However, search and retrieval of three-dimensional assets are so far underrepresented in research and practice. Thus, it is unclear how to support complex searches for these types of content, for instance in the context of digital humanities (DH) research.

To investigate interfaces for searching and retrieving 3D models, we looked at three questions: 

1. How do digital humanists conceptualize search in the context of historical reconstructions? 

2. Which spatial support by 2D, enhanced 2D and 3D maps do they prefer for exploratory searches?

3. Which dimensions of spatial context do digital humanists refer to when searching for 3D objects?

The questions were researched via an exploratory user study with six Digital Humanities researchers, using a so-called “think-aloud” protocol, in which participants verbalize their thoughts and actions. We evaluated interactive mockups of a prospective search engine, contextualized by 2D, enhanced 2D and 3D maps.

Figure 1: Basic (‘baseline’) search user interface (first 2 results). Result item: (A) 3D model thumbnail. From archival info.: (B) Title; (C) – (D) descriptions; (E) house address, room; (F) – (H) various metadata. (I) – (J) Degree of uncertainty in proposed 3D reconstruction.

Findings

Our findings are fully described in the paper referenced below [1], but here we summarize the main points:

Conceptualizing a 3D search engine (1)

During our user study, DH researchers imagined how a search engine for 3D objects would work, and what it would look like. They emphasized textual aspects, such as “a list of paintings that were in the house”, item descriptions, links to sources and various enrichments. Also, visual aspects played a key role. For instance, the 3D search engine should be “visually attractive”, one “should know what things looked like” and “not just get the image”, but also the overall spatial and historical context was emphasized. This means not just knowing “where something is”, but also how an object “relates to other objects and environments”, according to the participants.

Comparing 2D & 3D map variants (2)

In a subsequent part of our study, we looked further into which spatial support should ideally be provided in a 3D search engine. Participants tried out interactive mockups containing static 2D, and interactive 2D and 3D maps of a prospective search engine in a randomized order (see Figure 1 and 2). The responses of post-task surveys can best be summarized by participants’ satisfaction ratings: this was clearly highest for the 3D maps (4.2 out of 5) as opposed to enhanced 2D (3.8) and static 2D maps (3). As one participant indicated, the 3D maps “give a better perception of space”. At the same time, a need for combining map modalities came up: the usefulness of the 2D and 3D maps depends on the type of task a researcher is conducting.

  • (1) Static 2D maps
  • (2) Interactive 2D map
  • (3) Interactive 3D map
Figure 2: Evaluated variant map interfaces (click to enlarge)

Dimensions of spatial context (3)     

Our third question focused specifically at the role of spatial context, via a thematic analysis of the transcribed think-aloud sessions, which derives the main themes and subthemes that occurred for the combined set of participants. The main identified themes were:

  • Object properties ❍, referring to location and size of objects
  • Object relationships ⬥, referring to relations between objects with other objects, or the environment 
  • Perception of space ▲, including proportions, structural elements, space functions and lighting

These categories are visualized in Figure 3, which depicts the subcategories of the three main themes, and how many study participants referred to them. The findings represented in this diagram can help to inform design of future search interfaces for 3D objects, and to decide which kind of spatial context is needed.

Figure 3: Spatial context themes & sub-themes [# of participants]

Conclusion

Summarizing, our exploratory study showed the prime role of context in retrieving historical 3D objects, such as object properties, object relationships and cues in the environment. Participants expressed a preference for 3D contextual maps when presented with different types of maps in a prospective search interface. Our findings suggest a need for hybrid visualization modalities depending on the task at hand, which can be informed by the identified spatial context dimensions. Please refer to our published paper for full details and references.

[1] Hugo Huurdeman & Chiara Piccoli (2020). “More than just a Picture” – The Importance of Context in Search User Interfaces for Three-Dimensional Content. In Proceedings CHIIR 2020. ACM. (PDF).