We live in a society with increasingly pervasive digital interactions. The confinement resulting from the current pandemic has drawn attention to the increasing mediation of digital technologies in the way we relate to each other and to the importance of physical presence. In this context, human relationships lose a sense of embodiment that derives from touch, a shared experience of space, and a vivid perception of movement. As Susan Kozel argues [1], sharing the body through digital devices promotes a collaborative construction of new physical states, levels of consciousness, and even ethics. Many artists and scholars have approached this issue from different perspectives, however, as this mode of relationship becomes more present, we believe that, far from being an exhausted subject, further analytical and critical studies are urgently needed. One of these technologies is Virtual Reality, where, in an immersive environment, our physical body can interact with virtual bodies. The term Virtual Reality (VR) refers to a three-dimensional environment in which, using headsets, it is possible to remove the visual stimuli of the outside world by replacing them with computer-generated images, thus creating a sensation of being present in another reality. Although it has been present on the market for about four decades now, it is only more recently that VR has achieved greater maturity as a performance technology. VR explores new modes of bodily interaction, generating countless possibilities yet to be explored, however, interaction with a virtual body loses the material and vibrant quality intrinsic to a physical body, which is associated with fundamental embodied perceptions such as weight, temperature, touch, and breath. This suggests a new relational paradigm, in which our material body interacts with the movement of immaterial entities, and concomitantly requires a new way of looking at and analyzing movement. Thus, a central question to explore is: what happens to our perception of embodiment when we become immersed in a VR environment? What implications will this other mode of embodiment of self and others have on the way we relate to our surroundings? Contemporary dance explores an unlimited variety of movement, not only traditional dance movement but also movement derived from everyday gestures, sports, moods, animals, etc. The analysis of such unlimited potentialities of dance movement has always been a challenging work. Laban Movement Analysis (LMA), created by Rudolf Laban and later developed by Irmgard Bartenieff, is considered the most recognized and most widely used system for movement analysis in dance. However, this system does not consider a virtual relationship where one of the dancers is an immaterial body. How is the perception of weight, effort, time, and space when a physical dancer interacts with a virtual body? How does the dancer perceive his own kinesphere, his sense of touch, and his perception of bodily materiality when his own body can be invaded by an immaterial entity in motion? The interactive relationship of a physical body with a VR body challenges the analysis of movement as Laban conceived it. GhostDance aims to explore the embodied perception of the dancer when in interaction with a virtual peer and to contribute to the production of critical thinking within this new paradigm of non-physical relationship. More specifically, we will develop a phenomenological and computational study of movement analysis based on the LMA system that contrasts the movement relationship involving duets with flesh and blood dancers, with the movement relationship in which one of the elements of the duet is a virtual body. The results of the project are conceived in four main lines: a) A performance-conference that brings together physical reality with VR, to be presented on artistic and academic platforms; b) A repository in website format, including i) videos of the process, ii) explicitness interviews, ii) texts produced, iii) documentation of the movement virtualisation process, iv) biomechanical data, and v) a library of digital animations (dance movements). c) A minimum of four papers in recognised journals and conferences under three different approaches: i) a critical perspective based on the themes of the project's exploratory process; ii) the perspective on how to adapt the LMA to a VR context, and iii) technical perspectives on movement mapping and analysis involving the virtualisation of a real body. d) Development of a software prototype of movement analysis in VR able to integrate LMA. Such a prototype will result further into the definition of a protocol establishing guidelines for future implementation. By investigating a critical insight into the experience of (des)embodied relations, we believe that this project will have a significant contribution not only within the fields of performing arts and VR but also within the debate on artificial intelligence.

From May 1st, 2022 until October 31st, 2023

Team

Members

Rui Filipe Antunes - Principal Researcher(INET-md)

Cecília de Lima - Co-Principal Researcher (INET-md, Faculdade de Motricidade Humana, Universidade de Lisboa)

Ana Paula Cláudio (FCiencias.ID)

Bruno Pedro (INET-md, Faculdade de Motricidade Humana, Universidade de Lisboa)

Daniel Tércio (INET-md)

Hugo Magalhães (INET-md, Faculdade de Motricidade Humana, Universidade de Lisboa)

Jonas Runa (Universidade Lusófona de Lisboa)

Maria Beatriz Carmo (FCiencias.ID)

Paula Varanda (IHA, Faculdade de Ciências Sociais e Humanas, Universidade Nova de Lisboa)

Sérgio Bordalo e Sá (INET-md, Faculdade de Motricidade Humana, Universidade de Lisboa)

Sílvia Pinto Coelho (ICNOVA, Faculdade de Ciências Sociais e Humanas, Universidade Nova de Lisboa)

Advisors

António Cabrita

Ivani Santana

Luís Santos

Towards a Computational implementation of Laban Movement Analysis

T.1.1 Workshop on Laban Movement Analysis.

T.1.2 Calculation of LMA on motion capture data from XSENSE.

T.1.3 Implementation of prototype software model integrating LMA in VR.

T.1.4 Definition of a protocol for the development of a new software tool producing LMA analysis

Creation of Dance Duets and their motion capture

T.2.1 Creation of physical dance duets

T.2.2 Duet movement recording

T.2.3 Motion capture of one element of each duet

Virtualisation of the body from one of the elements of the duet

T.3.1 Mapping vertices and nodes from raw data;

T.3.2 Integration with a static 3D model (rigging and skinning);

T.3.3 Virtual scene with 3D character dancing.

Dance with virtual graphic representation (authentic body vs. virtual body - simulacrum)

T4.1 Virtual environment with a virtual dancing body;

T4.2 Dance experience between (authentic) dancer and (virtual) body.

Analysis of the movement of the performances

T5.1 Biomechanical analysis of dance with real partners

T5.2 Intersubjective analysis of the movement produced by dancers with real partners

T5.3 Biomechanical analysis of dancer in interaction with VR partner

T5.4 Intersubjective analysis of the movement produced by the dancers in interaction with VR partner

Performance-conference and informal presentations

T.6.1 Creation of a responsive environment

T.6.2 Creation of sound experience

T6.3 Public presentation of Performance-conference

Critical approach

T.7.1 - Group sessions of observation of the dance creations and discussion with the dancers on their sensorial experience and on the analysis of the movement.

Group session of literature review and discussion.

Organization of group and individual writing of articles