A pioneering and bold attempt to apply artificial intelligence (AI) technologies in the performing arts was carried out by Hong Kong Baptist University (HKBU) in a public concert held on 14 July in Hong Kong.

 The innovative performance was the first human-AI collaboration of its kind in the world, and it showcased how AI can be a creative force that can perform music, create cross-media art, and dance.

The performance was powered by the AI technologies developed by HKBU scientists under the “Building Platform Technologies for Symbiotic Creativity in Hong Kong” research project. The concert, dubbed “A Lovers’ Reunion”, was the Annual Gala Concert of the HKBU Symphony Orchestra, and it was conducted by its music director, Professor Johnny M Poon.

The research project is led by Professor Guo Yike, Vice-President (Research and Development) of HKBU, and supported by HK$52.84 million (US$6.77 million) in funding from the Theme-based Research Scheme under the Research Grants Council. Its deputy project coordinator, Professor Johnny M Poon, is the Associate Vice-President (Interdisciplinary Research) and the Founding Dean of HKBU’s School of Creative Arts.

The project team also comprises computer scientists from HKBU’s Augmented Creativity Lab. The unique mix of scientists and artists in the project team enables them to merge their cross-disciplinary knowledge and come up with sparkling ideas on how to unleash the endless possibilities of art creation with the infusion of cutting-edge technologies.

At the Concert, the project team presented a performance that marked the first time in the world that an AI choir had combined with an AI-generated visual storyteller to perform interactively with a conductor and an orchestra. The HKBU Symphony Orchestra shared the stage with an AI virtual choir to perform a newly arranged version of the song Pearl of the Orient with the voices of 320 virtual performers.

The AI virtual choir was “trained” by HKBU researchers using the generative models of singing that they built by extracting and disentangling key contributing features of vocal singing from a collection of songs recorded by professional singers.

An AI media artist learned from the lyrics of the Pearl of the Orient, and it created a stunning cross-media visual narrative of the song according to its interpretation of the underlying meaning of the lyrics. Unlike the conventional AI machines that import images as a reference for the algorithms to mimic, it used textual lyrics as the sole input so that it can associate the underlying meaning of the lyrics with an appreciation of the beauty of Hong Kong.

Another highlight of the Concert was a ballet performance featuring AI virtual dancers in Ravel’s Daphnis et Chloé, accompanied live by the HKBU Symphony Orchestra. With the help of professional dancers from the Hong Kong Dance Company, the AI virtual dancers interpreted the underlying emotional and aesthetic connections between the music and the dance. The dance movements, inspired by a newly discovered species of box jellyfish in Hong Kong, were then choreographed.

The AI-driven performance is one of the many milestone deliverables of the “Building Platform Technologies for Symbiotic Creativity in Hong Kong” research project, and others include an art data repository, a generative AI algorithms system, a research theatre, a digital art and policy network, and some unique and creative application projects, to name but a few.

e AI-driven performance is one of the many milestone deliverables of the “Building Platform Technologies for Symbiotic Creativity in Hong Kong” research project, and others include an art data repository, a generative AI algorithms system, a research theatre, a digital art, and policy network, and some unique and creative application projects, to name but a few.

Another important initiative HKBU is organizing is the “Human, Machine, Art, Creativity: International Symposium” in the Hong Kong Convention and Exhibition Centre on 9 August.

“AI is revolutionizing numerous fields of knowledge, and it is playing a more and more significant role in many creative processes like music, visual arts, and architecture. The Symposium will gather world-renowned researchers from academia and industry, crossing different expertise areas, to jointly discuss emerging topics in human-AI symbiotic art creativity,” said Professor Guo.

The Symposium will cover topics including the enabling technologies of symbiotic art creation and manifestation, the aesthetics and cognitive values of symbiotic art creativity, the new trend of the art economy with Web 3.0, new issues raised by the new forms of art in society, and the impact on art community stakeholders, among others.