The first phase of the research to identify the location of the house of Gillis van Coninxloo and propose a reconstruction hypothesis has been charted in this previous blog post. Further research has confirmed its location and added details to the appearance of this house and the evolution of this part of the Oude Turfmarkt over time. This blog post summarizes some of the main findings.

Frans Badens’ house

Due to the temporal proximity between Coninxloo’s death and the purchase of a house on the Turfmarkt by painter Frans Badens (Fig. 1) from Arent ten Grootenhuys, Badens’ house has been identified as the most likely candidate for Coninxloo’s.¹ His house was just north of the narrow alley that is visible on the map of Amsterdam made by Balthasar Florisz. van Berckenrode in 1625 (see the previous blog post). The likelihood that Badens took over the house where previously Coninxloo lived is increased by the fact that in the tax registrations (‘verponding’) of 1647-49 and 1650-52 Baden’s house is the only one that is taxed for 200 guldens, the same amount that Coninxloo had to pay to Ten Grootenhuys for a month of rent (as recorded in Coninxloo’s inventory).² Other archival sources, most notably transport acts, contain additional pieces of information on some of the house’s characteristics and informs us about the people who lived in this and neighbouring houses over time.

The house of Gillis van Coninxloo at the Oude Turfmarkt: New insights

The first phase of the research to identify the location of the house of Gillis van Coninxloo and propose a reconstruction hypothesis has been charted in this previous blog post. Further research has confirmed its location and added details to the appearance of this house and the evolution of this part of the Oude Turfmarkt over time. This blog post summarizes some of the main findings.

Frans Badens’ house

Due to the temporal proximity between Coninxloo’s death and the purchase of a house on the Turfmarkt by painter Frans Badens (Fig. 1) from Arent ten Grootenhuys, Badens’ house has been identified as the most likely candidate for Coninxloo’s.¹ His house was just north of the narrow alley that is visible on the map of Amsterdam made by Balthasar Florisz. van Berckenrode in 1625 (see the previous blog post). The likelihood that Badens took over the house where previously Coninxloo lived is increased by the fact that in the tax registrations (‘verponding’) of 1647-49 and 1650-52 Baden’s house is the only one that is taxed for 200 guldens, the same amount that Coninxloo had to pay to Ten Grootenhuys for a month of rent (as recorded in Coninxloo’s inventory).² Other archival sources, most notably transport acts, contain additional pieces of information on some of the house’s characteristics and informs us about the people who lived in this and neighbouring houses over time.

Fig. 1: Frans Badens in the Pictorum Aliquot Celebrium Praecipuae Germaniae Inferioris Effigies (source)

‘Het huijs van den Iteliaen’ and its ‘gemene gangh’

For example, we know that Coninxloo’s house had a wall and a lead gutter shared with its northern neighbour and that the narrow alley was shared (‘gemene gangh’) with the southern neighbouring house.³ As we learn from the sources, this alley was primarily used to remove rainwater (‘osendrop’) and allowed access to the small courtyard of the southern neighbour situated at the back of the house.⁴ The archaeological excavations carried out by the Amsterdam Municipality and cited in the previous blogpost shed light on the measurements of this narrow space, about 1 meter wide. From another archival document drawn up at the end of the century we also know of the presence of windows and a door on the side of the house which opened onto the shared alley.⁵ Although this document refers to the situation in place decades after Coninxloo lived in this house, we can reasonably expect that a door and a few windows would have existed also in Coninxloo’s time on the side of the house bordering the shared alley.

A particularly interesting source is a schematic plan of this house’s southern neighbour made in 1648 and kept in the archive of the Gasthuis (Fig. 2).⁶ The plan maps out the inner division of its southern neighbour in detail and shows that the shared alley allowed access to the back of this house. It also shows how the kitchen was built against the slanting wall of the Nieuwe Nonnenklooster. Although the house where Coninxloo once lived is rendered only with its perimeter, the representation of space of its southern neighbour (especially the shape of the courtyard and the kitchen in the back) offers us a visual source to hypothesize a similar situation at the house where Coninxloo’s once lived. As the note written within the perimeter of the house tells us, at this time this was ‘the house of the Italian’ (‘het huijs van den Iteliaen’). The existence of the tax registration for this year allows us to identify this Italian man with Filippo Biancucci, a merchant from Livorno.⁷

Fig. 2: The plan made in 1648 of the house neigbouring the shared alley to the south (right in the image). The house to the left of the shared alley is the house where Coninxloo once lived and is now defined as ‘het huijs van den Iteliaen’, the merchant Filippo Biancucci (Amsterdam City Archives, 342 Inventaris van het Archief van de Gasthuizen, inv. 1435).

Oude Turfmarkt over time

One important element to consider in order to situate the houses that Hudde built on the Oude Turfmarkt – and Coninxloo’s house among them – is the location of the entrance to the Gasthuis (which nowadays serves as the main entrance to the Special Collections of the University of Amsterdam). As is clearly visible by comparing the maps of the Gasthuis made in 1628 and in 1680, the location of this entrance has changed (see fig. 3). This is a consequence of the big renovations that were carried out on the Turfmarkt in the early 1640s: The architect Philips Vingboons worked on both northern and southern ends of the street and built a series of houses with a neck-gable facade for the Pietersgasthuis, and two houses (nrs. 145-147) with a twin-facade for Pieter Jansz Sweelick. To make room for the houses built by Vingboons, part of the northern inner courtyard (nr. 93, called ‘‘t bleeck velt’ in the map captions) was removed and the entrance to the Gasthuis was moved toward the south. Being acquired in 1656, the southern neighbour of ‘Coninxloo’s house’ and the shared alley appear on the map of the Gasthuis made in 1680. Numbered 23, this house is defined as the soldiers house (‘soldaaten huys’) in the map captions. An axonometric view made in the same year shows this house viewed from its southern side with two chimneys in the main building and one in the smaller annex on the back which served as the kitchen (Fig. 4, right).

Fig. 3: Comparison between the maps made in 1628 and 1680. In the middle section, the perimeter of Vingboons’ houses is roughly superimposed on the map of 1628 to give an idea about the portion of the inner courtyard that was incorporated into the new design and the shifted location of the access to the Gasthuis terrain. (Amsterdam City Archives, 342 Inventaris van het Archief van de Gasthuizen, inv. nrs. 1468 and 1469).

Fig. 4: Axonometric maps made in 1628 (left) and 1680 (right) displaying a more detailed rendition of the Gasthuis (Amsterdam City Archives, 342 Inventaris van het Archief van de Gasthuizen, inv. nrs. 1468 and 1469).

Historical photos display the development of this part of the street in more recent times. In one of the earliest, dated 1865, the entrance to the Gasthuis is visible and the remnant of the shared alley can be glimpsed through the tree in front of the house (Fig. 5, top image, highlighted in blue). The house on the right side of the alley in this photo therefore occupied the space where Coninxloo’s house used to lie. In 1870, this house and its northern neighbour were dismantled and the Amsterdam ‘Spaarbank’ was built on their two plots. As can be seen from a picture of the Spaarbank taken around that year, the shared alley, walled up and closed by a door, is still on its place (Fig. 5, second image, highlighted in blue). The alley will remain there until 1883, when it was dismantled to build the Sint Bernardus Gesticht (Fig. 5, third and forth images).

Fig. 5: Historical images showing the evolution of this part of the street over time. Highlighted in yellow, the entrance to the Gasthuis (now the main entrance to the Allard Pierson Special Collections); in blue, what remained of the shared alley mentioned in the archival sources and uncovered by the archaeological excavations in this area. The alley was then dismantled in the late 19th century with the construction of the Sint Bernardus Gesticht (images from the Amsterdam City Archives beeldbank).

As this Coninxloo case has demonstrated, archives are a treasure trove to reconstruct the history of a house, its inhabitants, and its neighbours. Pieces of information to be cross referenced can come from disparate sources, even produced much later than the period in question. This time consuming but rewarding task of researching archival materials over a long period of time helps to shed light on the diachronic urban development and on the people that inhabited these buildings. The schematic, grey-scale 3D reconstruction of Coninxloo’s and neighbouring houses (fig. 6) based on the sources discussed in these blog posts and a chronological list of these sources are available to download from the Amsterdam Time Machine zenodo community (DOI: 10.5281/zenodo.7274567).

Fig. 6: The schematic 3D reconstruction of Coninxloo’s and neighbouring houses resulting from this research and available to download from the Amsterdam Time Machine zenodo community.

Acknowledgments

I thank Frans Grijzenhout and Bart Reuvekamp for their help with the archival research on this case study.

Footnotes

1. Amsterdam City Archives, Archief van de Schepenen, Register van Schepenkennissen (5063), inv. nr. 11, fol. 285 (15 June 1607). See the previous blogpost with references to Dudok van Heel 1987 and Bok 2005.
2. Amsterdam City Archives, Inventaris van het Archief van de Thesaurieren Extraordinaris (5044), 254-293 Verpondings-quohieren van den 8sten penning. 41 perk. (Oud nr. 33), 1647-1733, inv. nr. 254, fols. 195r-200v and inv. nr. 255, fols. 195-1v – 200v; Amsterdam City Archives, Inventaris van het Archief van de Notarissen ter Standplaats Amsterdam (5075), notary Frederik van Banchem (nr. 13), inv. nr. 262, fol. 86 (11 January 1611).
3. Kwijtscheldingen, archive nr. 5066, inv. nr. 36 and Kwijtscheldingen, archive nr. 5067, inv. nr. 25 (18-06-1680).
4. Ibidem.
5. Kwijtscheldingen, archive nr. 5062, inv. nr. 73, 26-06-1698.
6. I thank Frans Grijzenhout for finding this plan and share it with us.
7. Amsterdam City Archives, Inventaris van het Archief van de Thesaurieren Extraordinaris (5044), 254-293 Verpondings-quohieren van den 8sten penning. 41 perk. (Oud nr. 33), 1647-1733, inv. nr. 254, fol. 195 and inv. nr. 255, fols. 194-2 and 195-2.

Chiara Piccoli and Weixuan Li

A crucial question in our project is how to identify the location of a house in the urban space. As we know, archival sources record addresses at the street level, sometimes with additional information regarding their relative location (e.g., at a corner with another street or in front of a recognizable building). This way of spatially “mapping” the city in early modern sources poses challenges to our possibilities to securely pinpoint a house on the map. On the 2D level, uncertainties in locations are expressed through fuzziness and various degrees of transparency of a possible area so that the range of uncertainties involved in the 2D representations can be communicated explicitly.^[1]

When it comes to the 3D level, however, the additional dimension increases the uncertainties exponentially as all details need to abide by the stringencies of the exact house location and its spatial boundaries. This problem has not manifested itself in the case study of the house of Pieter de Graeff at Herengracht 573. There, the house is still standing and the memory of its owners has been preserved. But what to do when the early modern house is long gone and the owners have been forgotten with the passing of time? Can digital mapping tools and 3D reconstructions help us to gain some insights in how the domestic space was used and experienced also in this case? Often, the evidence is so scarce that there is no ground to even start trying to create a reconstruction hypothesis; but what about in other situations, for example when the street is known, but the precise location of the house can only be hypothesized? We are going to explore these questions with the house of the painter Gillis van Coninxloo.

Coninxloo: Life and Work

Gillis van Coninxloo (Antwerp, 1544 – Amsterdam, 1607) was one of the most important and celebrated painters in Amsterdam at the turn of the 17th century and had a profound influence on the stylistic development in landscape paintings in the Dutch Republic (Fig. 1). Born into a family of painters, Coninxloo apprenticed in Antwerp and established his painting career there for over a decade until the religious strife in the Southern Netherlands under the Spanish rule forced the Reformed artist to flee north in 1585. Coninxloo left first for Middelburg and then in 1587 for Frankenthal in Germany before permanently settling down in Amsterdam in 1595, where he died in 1607.^[2]

As one of the few established painters emigrating from the South who resumed their career as ‘kunstschilders’ (artist painters) in Amsterdam, Gillis van Coninxloo introduced a new approach to Flemish landscape painting.^[3] His up-close landscapes turned forests into the main subject instead of the backdrop by crafting elaborate compositions of trees and fine shades of the leaves on scale (Fig. 2). His dense, airless forest scenes influenced and inspired the younger generation of landscape painters of the 1600s in Amsterdam, such as Roeland Savery and David Vinckboons, in addition to his numerous pupils. In 1604, art theorist Karel van Mander wrote in his Schilder-boeck that ‘[Coninxloo] is, as far as I know, the best landscape painter of his time; his style is now frequently imitated in Holland’.^[4] Besides his influence in the stylistic development in landscapes, Coninxloo was, too, active in the art trade.^[5]

After his death in 1607, the auction of his estate gathered almost the entire artists’ community in Amsterdam. A probate inventory of Coninxloo’s possessions in his house prior to the auction survives and offers a valuable and rare example of how this renowned painter (who was also active in the art market) arranged his workshop and operated his business in the domestic space in the first decade of the 17th century.

Coninxloo’s inventory (Fig. 3) is preserved at the Amsterdam City Archives^[6] and it is part of the sample that Weixuan Li is analysing for her study on the creative communities in Amsterdam. A large number of paintings including many unfinished ones and a significant amount of art supplies probably suggested that the artist not only produced paintings but also ran a shop at home. Interestingly, the inventory speaks of both ‘Coninxloos winckel’ and an ‘achter winckel’ on the first floor alluding to the possibility that the studio and painting shop were located in the same space. This aspect has made this house an interesting case study to see what kind of contribution could 3D modelling give to our understanding of its internal room organization, notwithstanding the uncertainty about its location as will be further discussed in the next section.

---

Sources for locating Coninxloo’s house

The presence of the painter Gillis van Coninxloo in Amsterdam is registered for the first time in the ‘Poorterboeken’ on 22 April 1597.^[7] On 23 August 1603, Coninxloo posted banns with Geertje van Eede.^[8] In this document, he appears for the first time as living on the Turfmarkt, and the addition of ‘8 ans’ in smaller caps on top of ‘wonende’ indicates that he had been living in Amsterdam for eight years. As usual, no additional indication of a more precise location is given to pinpoint the house where Coninxloo lived.

Traces in the archive

The starting point to identify the range of possible candidates on the Turfmarkt is the reference contained in his probate inventory to a debt of 200 gulden he had with Arent ten Grootenhuys, one of the founders of the VOC, for one-year rent of a house. Ten Grootenhuys was the grandchild of the merchant Arent Hudde, who we find in the Kohier of 1562^[9] as the owner of three houses on the Turfmarkt, and again in the Kohier of 1568/69,^[10] this time mentioned as the owner of two more, thus owning a row of five adjacent houses. The Kohier of 1578^[11] provides us with the picture of the situation ten years later: after the list of houses belonging to, and rented out by, the Nieuwe Nonnen convent, we find Hudde clearly stated again as the owner of five houses, all rented out. The names of the tenants are given and the fifth one is the painter Dirck Barentsz. Barentsz appears therefore to have rented the last (most south) of Hudde’s five houses.

Additional pieces of information are contained in the execution of will of Cathrijn Jacobs, Hudde’s widow, dated 1 September 1600.^[12] There, we find out that she left her possessions, among which the houses on the Turfmarkt, to her grandchildren. Specifically, the “children of Reijmerich Hudde” (among them, Arent ten Grootenhuys) inherited the house where the painter Pieter Isaacsz used to live. No relative location of this house in relation to the others is explicitly mentioned in this document, but its higher value could be taken as an indication of a bigger house size. Another key to the puzzle comes from an archival document indicating that in 1607 Arent ten Grootenhuys sold a house on the Turfmarkt to the painter Frans Badens.^[13] Some of the characteristics of this house are described in this source, such as that it had a shared alley with its southern neighbor.

The years in which Coninxloo lived on the Turfmarkt (1603-1607) fall in between the periods covered by the archival sources and therefore Coninxloo is not mentioned in any of them. Based on the previously discussed evidence, Bas Dudok van Heel and Marten Jan Bok made the logical step to assume that the four painters (Barentsz, Isaacsz, Coninxloo and Badens) inhabited in sequence in the same house – the fifth of the houses originally built by Arent Hudde.^[14] It seems indeed straightforward to make a one-to-one connection between the house inherited by the children of Reijmerich Hudde, and the house for which Coninxloo still needed to pay Ten Grootenhuys one year of rent. The archival research conducted in the context of our project shows however a less clear situation:^[15] By 1615 the children of Reijmerich Hudde seem to own more houses on the Turfmarkt, instead of only the one that is indicated in Cathrijn Jacobs’ will. The fact that Badens bought the house from Ten Grootenhuys only a few months after the auction of Coninxloo’s belongings can be taken as an indicator that he indeed acquired the house where Coninxloo lived, which had become vacant. The likelihood that Badens and Coninxloo inhabited the same house is therefore high, but the possibility that they lived in different houses cannot be excluded with certainty at the moment. More research is needed to follow through the questions opened up by the archival documents and to understand whether they have any implications for locating Coninxloo’s house.

Archaeological evidence

The archaeological excavations at Oude Turfmarkt 135-139 conducted by the Amsterdam Monuments and Archaeology Office in 2005 helped to shed more light on the location of Badens’ house.^[16] Their documentation allows us to see the foundations of three parcels built in the first quarter of the 16th century and of a 1m-wide alley, in between parcels II and III (Fig. 4). As already suggested by Bok, for its characteristics, parcel II corresponds most certainly to the house that Badens bought in 1607 from Ten Grootenhuys.^[17] This house was located on a parcel in the area now occupied by the Special Collections and the Allard Pierson Museum Cafè (n. 139) (Fig. 5).

The heavy foundations on which the houses were built are exceptional for residential buildings. Their sturdiness was given by the necessity to prevent them from sinking into the unstable terrain. The archaeological investigations showed that they were in use for a long period of time, thanks indeed to their solid construction. We are therefore looking at the same perimeter that the houses had in the early 17th century. The excavated remains clearly show that the parcels become longer towards the south-east (as it is also visible in the 1625 map by Balthasar Florisz van Berckenrode, see Fig. 6). This difference in length originates from the necessity to negotiate the available space between the river’s bank on one side and the arch-shaped row of houses of the earlier Nieuwe Nonnen convent which followed the river’s course on the other.

---

3D modelling as a drafting and exploratory tool

By inference, as discussed above, Badens’ house is at present also our most plausible candidate for the house where Coninxloo lived. As we showed, however, the evidence at our disposal is insufficient to exclude other possible candidates. This is the first dimension of uncertainty that we are dealing with when we are trying to position Coninxloo’s house on the map. An additional dimension of uncertainty comes from the internal arrangement of space. In the case of De Graeff, although his house has been modified over the century, it still offered some anchor points to compare and relate which rooms were mentioned in the inventory.^[18] There, the sources of uncertainty were related to the relative position of objects within the rooms and their appearance. In Coninxloo’s case, the inventory is organized by room but we do not have any extant remains that help us with this task. The only information about its dimension is the (incomplete) perimeter of the house supposedly owned by Badens that emerged from the archaeological excavations displayed in Fig. 5.

Given these circumstances, we use 3D modelling as an exploratory and drafting tool that allows us to visualize our hypotheses and interpretations. The 3D reconstruction acts as a platform for discussion and is presented in a visual language that conveys the highly hypothetical and exploratory nature of this work. The rendering method is therefore not photorealistic but schematic and wireframe (Fig. 7). What 3D modelling can contribute to in dealing with multidimensional uncertainty, for example, is the possibility to visualize what the house could look like in the various possible scenarios. We can explore questions such as to what extent the changes in the available interior space matter when the same room partition is applied to the progressively smaller parcels.

One of the possible outcomes could be the observation that even with different parcels, the changes in the available inner volume would not be significant after all. If a smaller size would have little impact on how space could be used inside, then the problem of the secure identification of the house location would become less important in view of what we are interested in investigating. Taking it further, this exploratory use of 3D modelling could also help us to support the hypothesis that all the above-mentioned painters lived in the same house in case we would be able to demonstrate that it offered particularly favorable conditions, for example in term of lighting and access, for the installation of an artist workshop.

---

^[1] See Weixuan Li’s forthcoming dissertation: Artists and the Creative Urban Space: Deep-Mapping Painters’ Locations in Golden Age Amsterdam (University of Amsterdam).

^[2] J. Briels (1997). Vlaamse schilders en de dageraad van Hollands Gouden Eeuw, 1585-1630: met biografieën als bijlage, Antwerpen: Mercatorfonds.

^[3] Many immigrants from the Southern Netherlands who were known as painters did not have any known work and therefore were questioned for their artistic qualifications. Only Coninxloo, Hans Bol, Jacob Savery as established Flemish landscape painters continued to paint in Amsterdam. Most landscape painters with roots in the South emigrated with their parents at a young age and were trained in the northern cities like Amsterdam. D. Van der Linden (2015). ‘Coping with Crisis. Career Strategies of Antwerp Painters after 1585.’ De Zeventiende Eeuw. Cultuur in de Nederlanden in Interdisciplinair Perspectief 31 (1), pp. 18–54. E.J. Sluijter (2015). ‘Career Choices of Migrant Artists between Amsterdam and Antwerp. The Van Nieulandt Brothers.’ De Zeventiende Eeuw. Cultuur in de Nederlanden in Interdisciplinair Perspectief 31 (1), pp. 101–37.

^[4] ‘[…] soo weet ick dees tijt geen beter Landtschap-maker: en sie, dat in Hollandt zijn handelinghe seer begint naeghevolght te worden […]’. Van Mander, Karel, Het Schilder-boeck, 1604, fol. 268r.

^[5] J.M. Montias (2002). Art at Auction in 17th Century Amsterdam. Amsterdam: Amsterdam University Press, pp. 115-119.

^[6] SAA, Inventaris van het Archief van de Notarissen ter Standplaats Amsterdam (5075), notary Frederik van Banchem (nr. 12), inv. nr. 262, fols. 68-88 (11 January 1611).

^[7] SAA, Inventaris van het Archief van de Burgemeesters: poorterboeken (5033), inv. nr. 1, fol. 183v (p. 354).

^[8] SAA, Inventaris van het Archief van de Burgerlijke Stand (5001), Huwelijksregisters, inv. nr. 411 (23 August 1603), p. 8.

^[9] NA, Archief van de Staten van Holland voor 1572, inv. nr. 1206, kohier van de tiende penning te Amsterdam 1562, fol. 313r.

^[10] SAA, Inventaris van het Archief van Burgemeesters: stukken betreffende verscheidene onderwerpen (5028), inv. nr. 550: Kohier van de verhuring van de huizen van fugitieven, geconfisceert in Amsterdam, 1568-1569, 1 deel, S III, p. 39, nr. 17, fol. 26.

^[11] SAA, Inventaris van het Archief van Burgemeesters: stukken betreffende verscheidene onderwerpen (5028), inv. nr. 661: Register van de 100ste penning van de onroerende goederen, gelegen aan de Oude Zijde in Amsterdam, 1578, 1 deel, S III p. 39, nr. 18, fol. 189.

^[12] SAA, Inventaris van het Archief van de Notarissen ter Standplaats Amsterdam (5075), notary Salomon Henrix (nr. 4), inv. nr. 11 (Band 3), 2v ff. (1 September 1600).

^[13] SAA, Archief van de Schepenen, Register van Schepenkennissen (5063), inv. nr. 11, fol. 285 (15 June 1607). This document is cited by Dudok van Heel as a ‘Rentebrief’ (B. Dudok van Heel (1987). ‘Enkele adressen van zestiende-eeuwse kunstschilders’, Amstelodamum 74, pp. 1-7 (7)). We thank Marten Jan Bok for having retrieved the document and shared the correct reference with us.

^[14] Dudok van Heel 1987 and M.J. Bok (2005). ‘Een Ganymedes van Francois Badens en de werkplaats voor schilderijen in Italiaanse stijl aan de Oude Turfmarkt’, Amstelodamum 92 (4), pp. 3-14.

^[15] We thank Bart Reuvekamp for his help with this archival research and for the transcription of Coninxloo’s inventory, and Judith Brouwer for checking it.

^[16] J. Gawronski, R. Jayasena, J. Veerkamp (2010). Van Amstelbocht tot Binnengasthuis Archeologische opgraving Oude Turfmarkt, Amsterdam (2005)’, AAR (Amsterdamse Archeologische Rapporten) 31, 2010. I thank R. Jayasena for having shared the original AutoCAD drawings of the excavations with me (C.P.).

^[17] Bok 2005, p. 5.

^[18] C. Piccoli (forthcoming). ‘Home-making in 17th century Amsterdam: A 3D reconstruction to investigate visual cues in the entrance hall of Pieter de Graeff (1638-1707)’, in G. Landeschi and E. Betts (eds.), Capturing the senses: digital methods for sensory archaeologies, Springer.

---

How to design & evaluate user interfaces as well as interaction paradigms for accessing 2D & 3D datasets involving uncertainty?

This was the question which I addressed in my project stream within Virtual Interiors. My work involved designing, creating and evaluating a demonstrator for a 3D research environment, in a co-design setting with Chiara Piccoli and during embedded research at project partners Netherlands Institute for Sound & Vision and Brill Publishers.

This post summarizes the work pertaining the 3D research environment, points to additional resources and also looks ahead to future opportunities.

1. The challenging task of 3D interface design

It is not straightforward to create user interfaces for accessing 3D reconstructions in a historical setting. User interfaces can be seen as an “in-between device” between the historically interested user and the actual 3D reconstructions and underlying data (see e.g. [1]).

The interaction with 3D environment and assets is challenging in itself and navigation issues and disorientation are commonplace [2]. This affects the usability of a prospective interface [3]. In addition, the usability is influenced by the number and complexity of interface features: the paradox is that more features might make an interface more useful to a researcher, but also harder to use. Striking a balance between functionality and usability is therefore of key importance.

Achieving this balance is also more arduous because there is a lack of documentation of best practices for interface design in a 3D context. Hence, it is hard to determine which “standard” set of features is necessary for interacting with 3D historical content [4].

2. Exploring requirements

To shed more light on these issues, I spent the first months of my work as a postdoc in the Virtual Interiors project (2019) in exploring the requirements for a 3D research environment. This was done via an analysis of previous use cases for 3D, design thinking [5] sessions with the other project members. Some requirements for a 3D research environment that came up were the ability to view the 3D space from different perspectives, to interact with objects, to add analytical layers (uncertainty & lighting), to annotate and to view context information. Furthermore, a user study with digital humanists was conducted [6].

From this user study, it became apparent that historical and spatial context are of key importance for a 3D research environment. From a thematic analysis of session transcripts, we identified the spatial context dimensions object properties, object relationships, and perception of space, pictured in Figure 1.

More information:
📖 Hugo Huurdeman & Chiara Piccoli (2020), The Importance of Context in Search User Interfaces for 3D Content (Proceedings CHIIR 2020)

3. The Virtual Interiors 3D research environment

Based on the initial phase of analyzing requirements, a prototype research environment was designed and developed in an iterative way. The prototype addressed the requirements mentioned above. While a full discussion of its functionality, the rationale behind it, the technical structure and the reconstruction process is available elsewhere [16], I discuss some key features here.

Analytical & Experiential user interfaces

Historical 3D environments can be utilized by widely different audiences, ranging from the casually interested to topical experts. For these audiences, different user interface features can be worthwhile. Instead of creating one interface to cater for all of their needs, which may lead to to the aforementioned tensions in the balance between functionality and usability, I decided for a different approach.

To cater for these different audiences and their goals, we constructed a “multi-layer interface” [7], in which two distinct interfaces use the same underlying framework and data. On the one hand, there is an analytical interface layer, aimed at researchers, and on the other hand a more experiential interface layer, which can be used by both researchers and other historically interested individuals:

• The Analytical interface allows researchers to take a deep-dive into underlying research data about domestic interiors, view uncertainty overlays, explore associated archive documents and to follow leads to connected Linked Data for instance examining artist biographies, related artworks and various historical data sources. Moreover, it is possible to explore hypotheses related to object arrangement and lighting. These analytical interfaces are best experienced via desktop computers or tablets (see Figure 2A & 2B).
• The Experiential interface makes it possible to experience historical spaces intuitively, by e.g. utilizing Virtual Reality and the motion sensors of mobile phones (see figure 2C & 2D). Casual users and researchers can “jump into” a 3D historical space, look around and interact with the objects around them, in more immersive [8]. Experiential views have been explored in the context of Chiara Piccoli’s 3D reconstructions of domestic interiors, but also during my embedded research at the Netherlands Institute for Sound & Vision.

• 

• 

• 

• 

Read more about analytical interfaces:
📖 Hugo Huurdeman, Chiara Piccoli (2021), 3D Reconstructions as Research Hubs (Open Archaeology journal)
Read more about experiential interfaces:
📖 Hugo Huurdeman, Jesse de Vos (2020). Visualizing a Radio of the Past using Technology of the Future (blogpost, Neth. Inst. for Sound & Vision, R&D blog, with link to early 3D demo)

Integrating Linked Data

Another key differentiation point between the Virtual Interiors research environment and other 3D frameworks [9] is the pivotal role of Linked Data [10] in enriching our content. Linked Data provides a common approach to more easily interconnect data, and has also been used in the context of (digital) history [11]. More scarcely, however, it has been applied to the context of 3D reconstructions [12].

For the Virtual Interiors 3D reconstructions, we use Linked Data to dynamically integrate biographical information from Ecartico, but also additional (and visual) resources pertaining the reconstructions at hand. For instance, selecting Quellinus’ portraits in the entrance hall of the reconstructed house of Pieter de Graeff and Jacoba Bicker reveals a gallery of works by the same creator, and works of a similar type (from Adamlink and Wikidata). Linked Data can have added value for these kinds of interfaces, although the involved SPARQL [13] queries need to be designed carefully, and it is very important to clearly display provenance.

The connection between Linked Data and both experiential and analytical interfaces is also illustrated by Figure 5 and 6. The analytical interface shows biographies, links and galleries based on Linked Data in the sidebar, while the experiential interface displays image galleries via a spatial projection surrounding and immersing the user.

Read more:
📖 Hugo Huurdeman, Chiara Piccoli, Leon van Wissen (2021), Linked Data in a 3D Context (DH Benelux 2021).

Analytical searching

Another important element of our research environment demonstrator is the ability to do anlaytical searching. As indicated in [6], 3D scenes and reconstructions can potentially consists of a vast number of objects and details, which may be hard to explore due to issues in navigating 3D spaces. Using the demonstrator’s approach, it is possible to browse a textual list of objects in the scene, but also to highlight singular as well as categories of objects — while keeping the context of the objects surrounding it. For instance, a user might want to see all furniture in the the entrance hall of Pieter de Graeff’s and Jacoba Bicker’s house (Figure 5). These elements are highlighted, while the surrounding objects are still (dimly) kept visible.

• 

4. Evaluation

As mentioned, the design and creation of the research environment took place in a co-design setting with Chiara Piccoli, but also involved frequent feedback from the other team members. Besides this internal feedback, also external feedback is of key importance to resolve usability and system issues. A more formal summative study of the analytical desktop prototype, combined with Chiara’s 3D reconstructions and research about Pieter de Graeff’s entrance hall, was conducted in June 2021 (6 participants).

Participants in the study filled out the System Usability Scale (SUS) questionnaire, which measures usability. The resulting SUS score was 77,9. This means that the usability is clearly above average, with only a need for minor improvements.

The majority of minor issues coming to light in the study could be fixed in the subsequent weeks, which included:

• Navigation issues. The familiarity with 3D environments of the participants influenced how easy it was to navigate the scene. We resolved this by providing more explicit instructions, and giving a higher prominence of a guided tour which informs about the possibilities of the 3D environment and viewer.
• Usability issues — these issues could be resolved rather easily, and were for instance related to the selection and deselection of objects in the scene, the textual labels of features and buttons.
• Technical issues and bugs — the majority of these smaller issues could also be fixed. More substantial issues, such as a lower performance on old computers could be partially alleviated by adding settings for simplified display and custom framerates.

5. Preservation strategy

Generally, cutting-edge 3D technologies are of an unstable nature. Software and platforms continuously evolve, often leaving behind support for prior technologies. This might render apps from the past non-functional [14, 15].

To mitigate this issues, the following approach was taken in Virtual Interiors:

1. to use only open tools for creating the 3D models (e.g. Blender),
2. to utilize open file formats (e.g. CSV, GLTF), and
3. to use open technologies for creating our research environment (e.g. BabylonJS).

A key component is the open-source 3D game engine for the web named BabylonJS. We chose a web technology to provide us with the flexibility to support various platforms (ranging from mobile to desktop) and to be able to use more immersive modalities such as VR via the emerging WebXR standard.

In our turn, we are also publishing the underlying data and tools via an open-source tools license as well, and make these accessible in Zenodo and DANS Easy by the end of the project. Documentation of process, demos and code is also a key focus, augmented by screencasts and screenshots of the demonstrator at different points in time.

6. Conclusion

Despite the end of my tenure at Virtual Interiors, the developments do not end here. Additional 3D reconstructions will be integrated into the research environment. Moreover, the 3D research environment will be refactored and modularized by Saan Rashid, who recently started as a software engineer in Virtual Interiors.

This blogpost discussed the creation of a demonstrator for a 3D research environment. I have outlined the challenges, our approach to tackle them, but also the key importance of evaluation when creating these kinds of novel interfaces. I highlighted some of the numerous exciting opportunities to use 3D reconstructions in a research context, ultimately providing a form of “analytic immersion”.

▶ Try out the experimental prototype (desktop version) with the entrance hall of the 17th-C house of Pieter de Graeff & Jacoba Bicker, reconstructed in 3D by Chiara Piccoli (prototype version: August 2021).

Footnotes*

1. Gane & Beer (2008)
2. see e.g. Chittaro & Ieronutti (2004); LaViola et al (2017); Jerald (2016)
3. See e.g. Jacko (2012); Lewis (2012); Mayhew (1999)
4. See for instance Champion (2019); Wood et al (2019)
5. See e.g. https://www.interaction-design.org/literature/topics/design-thinking
6. See Huurdeman & Piccoli (2020) and this summary blogpost.
7. See Shneiderman (2002).
8. E.g. Dede (2009), Immersive Interfaces for Engagement and Learning.
9. See e.g. the survey of features in current 3D frameworks by Champion & Ramahan (2020).
10. See e.g. Antoniou & Van Harmelen (2008)
11. see e.g. De Boer et al. (2016), Linked Data for Digital History; Meroño-Peñuela et al. (2020), Ontologies in CLARIAH; Zamborlini et al. (2017), Toward a Core Conceptual Model for (Im)material Cultural Heritage.
12. As e.g. done by Kuroczyński et al. (2016)
13. SPARQL is a query language commonly used for Linked Data repositories
14. See e.g. Delve et al., 2012, The Preservation of Complex Objects—Volume 1; Lischer-Katz et al., 2017, Supporting Virtual Reality and 3D in Academic Libraries. Also, projects such as Pure3D are looking at infrastructures for 3D visualizations.
15. More about this topic will be discussed in Huurdeman, Posthumus, Van den Heuvel, Beyond Dynamic Drawings: Restoring and Re-using Interactive 3D Visualizations (in preparation).
16. Huurdeman & Piccoli (2021), 3D Reconstructions as Research Hubs

Abbreviated citations correspond with Huurdeman & Piccoli (2021), 3D Reconstructions as Research Hubs, full details in paper’s reference list.

• 

How to design user interfaces for searching and retrieving 3D models in a digital humanities context? This was the question investigated in our short paper entitled “More than just a Picture” – The Importance of Context in Search User Interfaces for Three-Dimensional Content [1] (PDF), published in the proceedings of the CHIIR conference 2020. Since the conference couldn’t physically take place due to the COVID-19 situation, we briefly present our work in this blogpost.

Background

This paper was inspired by the growing number of historical environments which are reconstructed in three dimensions, for instance the 17th century houses in Amsterdam within the Virtual Interiors project. However, search and retrieval of three-dimensional assets are so far underrepresented in research and practice. Thus, it is unclear how to support complex searches for these types of content, for instance in the context of digital humanities (DH) research.

To investigate interfaces for searching and retrieving 3D models, we looked at three questions: 

1. How do digital humanists conceptualize search in the context of historical reconstructions? 

2. Which spatial support by 2D, enhanced 2D and 3D maps do they prefer for exploratory searches?

3. Which dimensions of spatial context do digital humanists refer to when searching for 3D objects?

The questions were researched via an exploratory user study with six Digital Humanities researchers, using a so-called “think-aloud” protocol, in which participants verbalize their thoughts and actions. We evaluated interactive mockups of a prospective search engine, contextualized by 2D, enhanced 2D and 3D maps.

Findings

Our findings are fully described in the paper referenced below [1], but here we summarize the main points:

Conceptualizing a 3D search engine (1)

During our user study, DH researchers imagined how a search engine for 3D objects would work, and what it would look like. They emphasized textual aspects, such as “a list of paintings that were in the house”, item descriptions, links to sources and various enrichments. Also, visual aspects played a key role. For instance, the 3D search engine should be “visually attractive”, one “should know what things looked like” and “not just get the image”, but also the overall spatial and historical context was emphasized. This means not just knowing “where something is”, but also how an object “relates to other objects and environments”, according to the participants.

Comparing 2D & 3D map variants (2)

In a subsequent part of our study, we looked further into which spatial support should ideally be provided in a 3D search engine. Participants tried out interactive mockups containing static 2D, and interactive 2D and 3D maps of a prospective search engine in a randomized order (see Figure 1 and 2). The responses of post-task surveys can best be summarized by participants’ satisfaction ratings: this was clearly highest for the 3D maps (4.2 out of 5) as opposed to enhanced 2D (3.8) and static 2D maps (3). As one participant indicated, the 3D maps “give a better perception of space”. At the same time, a need for combining map modalities came up: the usefulness of the 2D and 3D maps depends on the type of task a researcher is conducting.

• 

• 

• 

Dimensions of spatial context (3)     

Our third question focused specifically at the role of spatial context, via a thematic analysis of the transcribed think-aloud sessions, which derives the main themes and subthemes that occurred for the combined set of participants. The main identified themes were:

• Object properties ❍, referring to location and size of objects
• Object relationships ⬥, referring to relations between objects with other objects, or the environment 
• Perception of space ▲, including proportions, structural elements, space functions and lighting

These categories are visualized in Figure 3, which depicts the subcategories of the three main themes, and how many study participants referred to them. The findings represented in this diagram can help to inform design of future search interfaces for 3D objects, and to decide which kind of spatial context is needed.

• 

Conclusion

Summarizing, our exploratory study showed the prime role of context in retrieving historical 3D objects, such as object properties, object relationships and cues in the environment. Participants expressed a preference for 3D contextual maps when presented with different types of maps in a prospective search interface. Our findings suggest a need for hybrid visualization modalities depending on the task at hand, which can be informed by the identified spatial context dimensions. Please refer to our published paper for full details and references.

[1] Hugo Huurdeman & Chiara Piccoli (2020). “More than just a Picture” – The Importance of Context in Search User Interfaces for Three-Dimensional Content. In Proceedings CHIIR 2020. ACM. (PDF).

This project in a nutshell:

My postdoc project investigates how individuals created, used, displayed and experienced cultural goods in their homes during the Dutch Golden Age by developing virtual reconstructions of a selection of domestic interiors. These three-dimensional (3D) models aim to act as data integration and hypothesis visualizing platforms to spatially connect, manage and research the rich and heterogeneous data sources on this period. This project aims also to deal with the modelling process in a transparent way and, when possible, to visualize changes through time (4D).

Read More