Arvest prototype and the COESO Project

Devleopment of Arvest began in 2023. Our initial goal was too create a new version of our tool MemoRekall that would be compatible with the International Image Interoperability Framework (IIIF), hence the working name MemoRekall-IIIF. To anchor development in a real semantic case study, we created a number of examples around the documentation of a collaboration between a choreographer and a philosopher in Dancing Philosophy, one of the Pilots of the H2020 COESO Project.

Semantic Analysis Case Study: the COESO Project

MemoRekall was an integral part of Dancing Philosophy, Pilot 2 of the COESO Project. To begin, MemoRekall was used to document the collaboration between the dancer and the philosopher in the form of annotated video capsules. You can learn more about the project here. In this second iteration of MemoRekall, we considered that video is not always the principal document around which one may need to articulate one’s annotations in order to analyze a collaboration. We decided to propose a new development of MemoRekall and take as a basis a IIIF-driven environment. We called this prototype MemoRekall-IIIF and refered to the first version of MemoRekall as MemoRekall ‘legacy’. With this case study, we wished to show how MemoRekall-IIIF allows for the annotation of all types of documents, including video, and also that it could support a semantic analysis.

Demo Content

This work produced a IIIF manifest network. In this live demo, you can browse a collection of IIIF manifests of varying natures. We included a diverse number of documents from Dancing Philosophy: video recordings of performances and workshops, audio recordings, written documents, Laban Kinetography, photos etc. From a technical point of view, we built upon the Mirador IIIF viewer interface that allows the user to create annotations for image documents. We added the possibility to annotate all types of documents, including video. In addition to this, we also shaped composite and bespoke manifests that offer themselves as powerful tools for semiotic and fluid navigation of multimodal collections.

In manifests 01 Manual Network Configuration and 02 Automatic Network Configuration, you will find network visualizations of a same collection of documents. The first was made manually, the second using a simple algorithm that created links by processing the documents’ metadata. These visualizations are interactive, and you can open each document straight in the Mirador viewer by clicking on it.

In manifests 03-06 we demonstrate an example of a ‘legacy’ MemoRekall capsule that has been transposed into the new environment. It is important for us to ensure retro compatibility with the previous version of MemoRekall, and these manifest networks demonstrate that it can easily be done through simple parsing of the XML file generated by MemoRekall ‘legacy’. We also explored how these translations can be augmented in the new environment with network visualizations and different types of annotations.

In the manifest 07 Composite Collaboration Analytics Compass another bespoke data-driven visualization of the corpus is derived from our semantic analysis on the collaboration between the dancer and philosopher, based on the work of Dominique Boullier. Each document is positioned on a compass that indicates the typology of their collaborative nature (see the article).

This work allowed us to understand how important it will be to offer simple yet powerful tools to all users to create their own data-driven interfaces from a multimodal documentary corpus, accompanying their research or artistic projects. Rather that proposing a fixed set of interface possibilities, we wish to make this feature completely customizable.

Sandbox

You can also experiment in the live sandbox and create your own annotations. This is an empty version of the tool with a few example documents loaded. The possibilities werelimited to simple annotations, and the sandbox was open to all meaning that you will see annotations made by other users and other users will see yours (these regularly get cleaned-up). This live demo allowed the user to get familiar with the MemoRekall-IIIF interface, and get a feel for the tools that will become available.

Source Code and Documentation

You can consult the source code here. You can also consult a wiki here that took you through the various functionalities of the interface, and answers and questions you may have around usage.