Spotify holds a patent for technology that could generate personalized song mashups.
The filing, unearthed by MBW, describes a system that can analyze songs to determine which tracks are musically compatible, then automatically combine them into new mashup versions for individual users.
According to the document, which you can read in full here, the technology would work by first separating tracks into their component parts – vocals and instrumentals – then “adding the vocal component” of “at least one music track to at least a select segment of the base music track.”
In other words, Spotify’s algorithm could automatically layer the singing from one song over the backing track of a completely different song, though users would still control the creative choices.
The filing explains: “Some commercially available websites enable users to listen to playlists suited to the users’ tastes, based on state-of-the-art machine learning techniques. However, the art of personalizing musical tracks themselves to users’ tastes has not been perfected.”
Our discovery arrives hot on the heels of the news that Spotify plans to develop “artist-first” AI music products in partnership with Sony Music Entertainment, Universal Music Group, Warner Music Group, Merlin, and Believe.
One of the big questions following that announcement was what exactly those products might be.
A Spotify spokesperson told us on Tuesday (October 21) that the company’s song mashup patent is “unrelated to last week’s announcement”.
The patent describes a system that would evaluate musical characteristics to ensure compatibility, including what the filing calls “acoustic feature vector distances,” tempo matching, and harmonic progression. It would analyze whether two songs are in compatible keys, have similar tempos, and whether their beats align properly.
“A mashup is a fusion or mixture of disparate elements,” the patent filing states, noting that creating successful combinations requires careful musical alignment.
The technology would perform sophisticated audio processing, from pitch shifting to time-stretching, and what the patent calls ‘transition refinement’ to ensure smooth musical transitions.
A deep dive into the US patent archives reveals the invention detailed in the filing from June 5, pictured above, isn’t entirely new.
The most recent filing is a continuation of a patent application originally filed in the US in December 2019 and granted in October 2022.
Spotify also filed an application for the patent in Europe in 2020. It was granted by the European Patent Office on June 18 of this year.
In response to a request for comment about the song mashup patent, a Spotify spokesperson said: “We don’t have any news to share about this patent that was first filed in 2019 and made public in 2021.”
“As we have made clear, we respect copyright and any future innovations will be enabled with upfront agreements, artists’ and rightsholders’ choice, and fair compensation.”
Spotify spokesperson
They added: “It is unrelated to last week’s announcement. Spotify has filed patent protections for hundreds of inventions, and we file new patent applications every month. Some of these patents may become part of a future offering, while others may provide valuable insights.”
“As we have made clear, we respect copyright and any future innovations will be enabled with upfront agreements, artists’ and rightsholders’ choice, and fair compensation.”
The patent describes tailoring mashup creation to individual users’ listening habits.
According to the filing, Spotify could identify “music tracks for which a plurality of users have an affinity” and then determine which of those popular tracks a specific user actually likes, using at least one of those tracks as the base for personalized mashups.
The filing also references generating custom titles for these mashup creations by combining words from the original song titles, and even creating “personalized album art” by blending images associated with the combined tracks.
User control and interface
The patent also describes user interface elements that would give listeners control over the mashup creation process. Users would select a “query track” to serve as the base, and the system would then suggest compatible candidate tracks that could be mixed with it.
A volume control interface would allow users to adjust the balance between vocal and instrumental components, controlling how prominently each element appears in the final mix. According to the patent diagrams, users could manipulate a slider to control the balance between vocal and instrumental components in the mashup.
This suggests the technology would be interactive rather than fully automated, with users making creative decisions about which tracks to combine and how to balance them.
The patent reveals considerable technical sophistication in how the system would create transitions. Beyond basic tempo and key matching, it would evaluate factors including “beat stability,” “loudness compatibility,” “vocal activity detection,” and “harmonic change balance.”
The technology would perform what the patent calls “boundary and transition position refinement” – analyzing vocal content to avoid cutting off lyrics mid-phrase and aligning segment boundaries with downbeats. It would apply fade-ins, fade-outs, and filtering to smooth transitions between combined segments.
For each potential mashup, the system would calculate multiple compatibility scores – including tempo, harmony, loudness, vocal activity, and beat stability – then combine these into an overall “vertical mashability” score and “horizontal mashability” score to determine which segments work best together.
Spotify has been a prolific generator of patents in recent years.
Some of these have been quite ambitious – such as, for example, its A&R technology for predicting breaking artists. It also secured a patent to identify music tastes by tracking the personality traits of its users.
In 2023, we revealed that it had filed a patent for a system to create digital mixtapes that enable users to integrate their own audio with a selected list of songs.Music Business Worldwide
