MusicLM, a new AI system developed by Google, can generate music in any genre from a text description. However, due to potential risks, the company has no plans to release it.
While other AI systems for music composition, such as Riffusion, Dance Diffusion, Google’s AudioML, and OpenAI’s Jukebox, have been developed, MusicLM is unique in its ability to create complex and high-fidelity songs due to its advanced technology and extensive training data.
According to an academic paper, MusicLM was trained using a dataset of 280,000 hours of music. This training enables it to generate coherent songs based on “significant complexity” descriptions like “enchanting jazz song with a memorable saxophone solo and a solo singer” or “Berlin ’90s techno with a low bass and strong kick.”
The resulting songs are similar to those created by human artists, though they may lack creativity and musical cohesion. MusicLM’s samples are extremely impressive, especially given that it does not use human musicians or instrumentalists.
Here is a sample.
Yesterday, Google published a paper on a new AI model called MusicLM.
The model generates 24 kHz music from rich captions like “A fusion of reggaeton and electronic dance music, with a spacey, otherworldly sound. Induces the experience of being lost in space.” pic.twitter.com/XPv0PEQbUh
— Product Hunt (@ProductHunt) January 27, 2023
MusicLM can capture nuances such as instrumental riffs, melodies, and moods even with detailed and complex descriptions.
MusicLM’s capabilities go beyond simply generating song clips. Google researchers have shown that the system can build on existing melodies that are hummed, sung, whistled, or played on an instrument.
Furthermore, MusicLM can take a series of descriptions, such as “time to meditate,” “time to wake up,” “time to run,” or “time to give 100%,” and create a melodic “story” or narrative that can last several minutes, making it ideal for movie soundtracks.
Here are a few more samples.
今日もまた世界が変わってた ！Googleの研究チームがテキストから音楽を生成するMusicLMを公開。似たようなサービスは前もあったけど、それとは次元が違う。さすがGoogle 。公式ページ（ https://t.co/3GQW3DaaWa）で5500個のサンプルを無料で公開していて、5月のGoogle I/Oが待ち遠しい 。 https://t.co/XVk5eIWjKU pic.twitter.com/MnJqX65BIL
— sangmin.eth @ChoimiraiSchool (@gijigae) January 27, 2023
MusicLM can also be guided by a combination of images and captions, or it can generate audio that is “played” by a specific type of instrument in a specific genre. The AI “musician’s” experience level can also be set, and the system can create music inspired by locations, epochs, or requirements, such as workout music.
Nonetheless, as previously stated, Google does not intend to make MusicLM available to the public due to the risks and challenges that such a system poses. For example, it may be able to create copyrighted material using the training data it received.
The creators wrote:
We are aware of the possibility of creative content associated with the use case being misappropriated. We emphasise the importance of additional future work in addressing these risks associated with music generation.