Research Creations & Digital Scholarship

Taiko Composition and Performance at SIGGRAPH featuring Uzume Taiko

Research in the context of my activities is polysemic (has two or more meanings). The term research creation is best to use in order to describe “research in and through the arts”; a practice-based approach of conducting research through the act of creation (Borgdorff, 2010). One example is the investigation of how sound is designed in productions from its inception to its public presentation as both artefact and later as the theoretical ‘object’ of study.

Research in the co-development of xR or AI integrated applications exist on a spectrum of digital scholarship that can become increasingly layered. These, may undergo additional experimental and usability studies that lead to multiple types of published outputs such as a lived mixed reality event, journal, book chapter, conference, public exhibit, and/or online shareable media. When software development projects are facilitated in project-based learning environments with real clients, research can also mean the process of investigating teaching and learning interactions.

How the research that I conduct is defined, published, and consumed is influenced by the expectations of the communities of practice who appraise those results. These include peers from performative, medical, gaming, educational and technological disciplines.

An Augmented Reality Brain in Collaboration with The HIVE lab's Dr. Claudia Krebs

An example of the intersection of different types of research I engage in was demonstrated with the Fun Palace: Carnival of Mixed Reality, an in-person public event that integrated two MDM Program courses, consisted of 11 curated Mixed Reality prototype installations, some that I supervised and others which I co-created. It was also an event which I designed sound and composed music for, and where usability research was conducted with all participants. The event featured live actors, dancers and musicians, and consisted of Mixed Virtual Reality, Augmented Reality marker-based 3D objects and games, A Curiosity Booth where audiences were permitted to choose the curiosity they wished to experience through a digital interface, an interactive Dance-like juke-box within a more traditional proscenium-based performance space by long time collaborator Small Stage and more.

Beyond the event itself as a ‘container’ of publically disseminated research creations, a formal invited presentation also occurred at the ASC Cybernetics Conference : Acting Cybernetically. The Fun Palace was also featured as part of my keynote on Mixed Reality and the Arts, at the Arts, Culture and Digital Transformation Summit in Banff. Additionally, I was invited to submit a paper to the following journal: A Mixed Reality Carnival through the Looking Glass of Cybernetics. Cybernetics and Human Knowing27(2), and co-wrote another peer-reviewed article based on one of the mixed reality installation entitled Body Remixer: Extending Bodies to Simulate Social Connection in an Immersive Installation. Body Remixer was accepted at the most established computer graphics conference internationally as part of SIGGRAPH’s 2020 Art Papers sig, and became part of the Proceedings of the ACM on Computer Graphics and Interactive Techniques.

The paper was published in a special edition of MIT’s Leonardo Journal, known as one of the premiere art and technology journals internationally. Finally, Body Remixer, whose soundtrack I composed, enjoyed three further remounts touring across the Lower Mainland of Vancouver and was a featured installation as part of RectoVerso during last year’s pandemic; a satellite event linked to the largest and most established VR festival in the world (23 years): Laval Virtual. Finally, a journal article written in collaboration with John Desnoyers-Stewart will be part of the proceedings at ISEA 2022 in Barcelona. The article depicts how we mapped interactions at the Carnival within a typology of in-person proximity and virtual relationship, that we then applied to a live event that occurred during the pandemic.