Art – It was only five years ago that the electronic punk band YACHT entered the recording studio with a daunting task: they would train an artificial intelligence on 14 years of music, then synthesize the results in their album “Chain Tripping”.

“I’m not interested in being reactionary,” YACHT member and tech writer Claire L. Evans said in a documentary about the album. “I don’t want to go back to my roots and play acoustic guitar because I’m so scared of the coming robot apocalypse, but I also don’t want to jump into the trenches and welcome our new robot overlords.”



But our new robot overlords are making huge strides in AI music production. Although the Grammy-nominated “Chain Tripping” was released in 2019, the technology behind it is already obsolete. Now, the venture behind open-source AI image generator Stable Diffusion takes us on again with its next move: making music.

Image To Music: Ai Generates Musical Composition Inspired By Your Pic

Harmonai is funded by Stability AI, the London-based startup behind Stable Diffusion. In late September, Harmony released Dance Diffusion, an algorithm and set of tools that can create music clips by training hundreds of hours of existing songs.


“I started my work on audio propagation around the same time I started working with Stability AI,” said Jack Evans, head of development at Dance Diffusion, in an email interview. “My development work with [the image rendering algorithm] Disco Diffusion drew me in and quickly decided to return to audio research. I started Harmonai to facilitate my own learning and research and to build a community focused on audio AI.”

Dance Diffusion is in testing – currently the system can only produce clips that are a few seconds long. While the early results offer a promising glimpse into the future of music creation, they also raise questions about the potential impact on artists.


Music Abstract Oil Painting Canvas Hand Painted Home Room Decor Modern Art Wall

Dance Diffusion emerges a few years after the San Francisco-based OpenAI lab behind DALL-E 2 described its larger experiment in music production called Jukebox. Given a genre, artist, and piece of lyrics, a jukebox can play a relatively static set of music complete with vocals. However, songs produced by the jukebox lacked large musical structures, such as choruses with repetitive and often meaningless lyrics.

See also  Music Artist Houston

Google’s AudioLM, detailed earlier this week, shows more promise with its uncanny ability to produce piano music when given a short playback track. But it is not open source.


Dance Diffusion aims to overcome the limitations of previous open source tools by borrowing technology from renderers such as Stable Diffusion. The system is called a diffusion model, which generates new data (e.g. songs) by learning how to destroy and recover many existing data samples. Because the model feeds into existing samples — say the entire Smashing Pumpkins discography — it’s better at retrieving all previously destroyed data to create new works.

Newsies Broadway Theatre Musical Theatre, Disney Art, Png

Kyle Worrall, Ph.D. A student studying musical applications of machine learning at York University in the United Kingdom explained the nuances of diffusion systems in an interview:


“In training a diffusion model, training data such as the MAESTRO dataset of piano performances is ‘destroyed’ and ‘recovered’ and the model evolves in doing these tasks as it progresses through the training data,” he said. By email. “Finally, the trained model can take sound and convert it to music similar to the training data (ie piano performances in MAESTRO’s case). Users can use the trained model to do one of three things: create a new sound, recreate an existing user-selected sound, or switch between two input tracks. . interpolate.”

It’s not the most obvious idea. But as DALL-E 2, Stable Diffusion and other systems have shown, the results are very realistic.


Artists Use Ai To Compose Music

“Our first reaction was, ‘Well, it’s a raw tone forward from where we were before,'” Bechtolt said.

Unlike popular rendering systems, Dance Diffusion is somewhat limited in what it can create – at least for now. Although it is fine-tuned to a particular artist, genre or instrument, the system is not as generic as a jukebox. Some dance diffusion models are at their limits, including a hodgepodge from Harmonai and early adopters on the official Discord server, including models finely tuned with clips from Billy Joel, The Beatles, Daft Punk and musician Jonathan Mann’s Song A Day project. remains. strips. So, the Jonathan Mann model always produces songs in the style of Mann’s music.

See also  Music Artist Generator


And the music produced by Dance Diffusion isn’t fooling anyone today. The system can “style” songs by applying one artist’s style to another, essentially creating covers, but it doesn’t produce clips and takes no more than a few seconds (see below). clip). Nicolas Martel, a self-taught game developer and Harmonai Discord member, says this is the result of technical hurdles Harmonai has yet to overcome.

Overview Of 2014 Music Standards

“Because the model is trained on small samples 1.5 seconds at a time, it doesn’t learn or account for long-term structure,” Martell said. “The authors seem to say it’s not a problem, but in my experience — and logically at least — that’s not true.”


YACHT’s Evans and Bechtolt worry about the ethical implications of AI — the artists they work with — but they note that these “style transfers” are already part of the natural creative process.

“It’s something that artists in the studio are already doing very informally and sloppily,” says Evans. “You sit down to write a song and I want a fall bass string and a B-52 tune and I want it to sound like it’s from London in 1977,” you say.


Free Black And White Music Art, Download Free Black And White Music Art Png Images, Free Cliparts On Clipart Library

But Evans wasn’t interested in writing a dark, post-punk rendition of “Love Shack.” Instead, he thinks interesting music comes from experimentation in the studio — even if you’re inspired by the B-52s, your final product may not have the characteristics of these influences.

“When you try to achieve that, you fail,” Evans said. “One of the things that drew us to machine learning tools and the art of AI is the way they fail, because these models aren’t perfect. They predict what we want.”


Evans describes artists as the “ultimate beta testers” who use tools outside their intended means to create something new.

Art Music Stock Illustrations

“Often, the output can be really unnatural, damaged and upsetting, or it can feel really weird and new, and that failure is great,” Evans says.


Assuming that dance diffusion ever reaches the point of forming a coherent whole of songs, it seems inevitable that major ethical and legal issues will come to the fore. Even simpler AI systems already exist. In 2020, Jay-Z’s record label issued a copyright strike to YouTube channel Vocal Synthesis for using artificial intelligence to create Jay-Z’s covers of songs such as Billy Joel’s “We Didn’t Start the Fire”. After initially removing the videos, YouTube noted that the takedown requests were “incomplete” and reinstated the videos. But deep-fake music is still on murky legal ground.

See also  Art Music Academy

Perhaps anticipating legal challenges, OpenAI has open-sourced Jukebox under a non-commercial license, prohibiting users from selling any music created with the system.


Album Art Concepts On Behance

“There are few studies that determine how original the outputs of production algorithms are, so using production music in ads and other projects can lead to unintended copyright infringement and property damage,” Worrall said. “This area deserves further investigation.”

An academic paper by Eric Sunray, now a legal intern at the Music Publishers Association, claims that artificial intelligence music producers such as Dance Diffusion are infringing music copyright in the United States by creating “sustainable tapestries of sound from the work they’ve acquired during training”. Copyright.. He argued that he had violated the right to reproduce law. After Jukebox’s release, critics also questioned whether training AI models on copyrighted music material was fair use. Similar concerns have been raised about training data used in AI systems that generate images, code and text, often retrieved from the web without the creators’ knowledge.


Technologists such as Matt Dryhurst and Holly Herndon have built Spawning AI, a set of artificial intelligence tools designed by artists for artists. One of their projects, “Am I Trained?” Allows users to search for artwork and see if it has been included in an AI training kit without their permission.

Neo Pop Art

“We show people what’s happening on popular datasets used to train AI vision systems and initially give them tools to opt out or join training,” Herndon said via email. “We are also interviewing many of the largest research institutions to convince them that consensus-based data is beneficial to everyone.”


However, these criteria are optional and probably remain so. It has not been said whether Harmonai will adopt them.

“Should be


Violin And Music Note Waving Clip Art Free Png Image|illustoon

Leave a Reply

Your email address will not be published. Required fields are marked *