qzdl / Samplify

A sample retrieval engine, powered by whosampled.
GNU General Public License v3.0
29 stars 5 forks source link

Discovery Rate Improvements: Retry on spotify search, minimum search terms first #7

Open qzdl opened 5 years ago

qzdl commented 5 years ago

With the following usage

[samuel@qzdl spotify-samples]$ python samplify.py --search 'resurrection common' --album
Resurrection by Common
I Used to Love H.E.R. by Common
Watermelon by Common
Book Of Life by Common
In My Own World (Check The Method) by Common
Another Wasted Night With... by Common
Nuthin' To Do by Common
Communism by Common
WMOE by Common
Thisisme by Common
Orange Pineapple Juice by Common
Chapter 13 (Rich Man vs. Poor Man) by Common
Maintaining by Common
Sum Shit I Wrote by Common
Pop's Rap by Common

# SAMPLIFY: Resurrection

This playlist was generated using Samplify.
Github: https://github.com/qzdl/samplify

The album, Resurrection, has been broken down into:
"['contains samples of ']"

Percentage Matched: 75.9%

Songs with no match:
Resurrection -> The Sorcerer of Isis (The Ritual of the Mole), by Power of Zeus
Watermelon -> Get Outta My Life Woman, by The New Apocalypse
Book Of Life -> Stoop Rap, by from Wild Style
In My Own World (Check The Method) -> The Mixed Up Cup, by Clyde McPhatter
In My Own World (Check The Method) -> Keep It Rollin', by A Tribe Called Quest feat. Large Professor
Nuthin' To Do -> Walk on By, by Living Jazz
Communism -> Knocking 'Round the Zoo (1971 Version), by James Taylor and The Flying Machine
Thisisme -> The Power of Love, by Alton McClain & Destiny
Thisisme -> I'm Also a Client Commercial, by Sy Sperling
Maintaining -> Scenario (Remix), by A Tribe Called Quest feat. Leaders of the New School and Kid Hood
Sum Shit I Wrote -> Outside Love, by Brethren
Sum Shit I Wrote -> Lisbon Antigua, by The Jonah Jones Quartet
Sum Shit I Wrote -> We Write the Songs, by Marley Marl, Heavy D and Biz Markie

==> search terms:
The Sorcerer of Isis (The Ritual of the Mole) power of zeus
Get Outta My Life Woman the new apocalypse
Stoop Rap from wild style
The Mixed Up Cup clyde mcphatter
Keep It Rollin' a tribe called quest large professor
Walk on By living jazz
Knocking 'Round the Zoo (1971 Version) james taylor flying machine
The Power of Love alton mcclain   destiny
I'm Also a Client Commercial sy sperling
Scenario (Remix) a tribe called quest leaders of new school kid hood
Outside Love brethren
Lisbon Antigua the jonah jones quartet
We Write the Songs marley marl, heavy d biz markie

Sample Info:
Resurrection -> Dolphin Dance, by Ahmad Jamal Trio
Resurrection -> No Delayin', by Nice & Smooth
Resurrection -> I Got It Made, by Special Ed
I Used to Love H.E.R. -> The Changing World, by George Benson
I Used to Love H.E.R. -> I'm Gonna Do You, by Jungle Brothers
I Used to Love H.E.R. -> Still Talkin', by Eazy-E
I Used to Love H.E.R. -> Fly, Fly, the Route, Shoot, by If
Watermelon -> Sweet Inspiration, by King Curtis
Watermelon -> Watermelon Man, by Johnnie Taylor
Watermelon -> Just to Get a Rep, by Gang Starr
Watermelon -> A Who Seh Me Dun, by Cutty Ranks
Book Of Life -> Everybody Loves the Sunshine, by Roy Ayers Ubiquity
Book Of Life -> Get Up, Stand Up, by Bob Marley and The Wailers
Book Of Life -> The Message, by Grandmaster Flash and The Furious Five feat. Grandmaster Melle Mel and Duke Bootee
Book Of Life -> Welcome to the Terrordome, by Public Enemy
In My Own World (Check The Method) -> Las Vegas Tango, by Gary Burton
In My Own World (Check The Method) -> But Not for Me, by The Modern Jazz Quartet
Nuthin' To Do -> Leroy the Magician, by Gary Burton
Nuthin' To Do -> Protect Ya Neck, by Wu-Tang Clan
Nuthin' To Do -> Heidi Hoe, by Common
Communism -> The Surest Things Can Change, by Freddie Hubbard
Communism -> Leaves, by Pete Jolly
WMOE -> Capricorn, by The Cannonball Adderley Quintet
Thisisme -> Something's Got a Hold on Me, by Etta James
Thisisme -> Momma Miss America, by Paul McCartney
Thisisme -> Build and Destroy, by Boogie Down Productions
Orange Pineapple Juice -> Hydra, by Grover Washington, Jr.
Orange Pineapple Juice -> Maybe Tomorrow, by Grant Green
Orange Pineapple Juice -> Hostile, by Erick Sermon feat. Keith Murray
Orange Pineapple Juice -> Thisisme, by Common
Orange Pineapple Juice -> Going Back to Cali, by LL Cool J
Chapter 13 (Rich Man vs. Poor Man) -> Cross Country, by Archie Whitewater
Chapter 13 (Rich Man vs. Poor Man) -> You're Getting a Little Too Smart, by Detroit Emeralds
Chapter 13 (Rich Man vs. Poor Man) -> Buggin Out, by A Tribe Called Quest
Chapter 13 (Rich Man vs. Poor Man) -> The Message, by Grandmaster Flash and The Furious Five feat. Grandmaster Melle Mel and Duke Bootee
Chapter 13 (Rich Man vs. Poor Man) -> I Know You Got Soul, by Eric B. & Rakim
Maintaining -> But Not for Me, by The Modern Jazz Quartet
Maintaining -> Why Can't People Be Colors Too?, by The Whatnauts
Sum Shit I Wrote -> I Need Love, by LL Cool J
Pop's Rap -> Water No Get Enemy, by Fela Kuti
Pop's Rap -> Next Lifetime, by Erykah Badu

==> search terms:
Dolphin Dance ahmad jamal trio
No Delayin' nice   smooth
I Got It Made special ed
The Changing World george benson
I'm Gonna Do You jungle brothers
Still Talkin' eazy-e
Fly, Fly, the Route, Shoot if
Sweet Inspiration king curtis
Watermelon Man johnnie taylor
Just to Get a Rep gang starr
A Who Seh Me Dun cutty ranks
Everybody Loves the Sunshine roy ayers ubiquity
Get Up, Stand Up bob marley wailers
The Message grandmaster flash furious five grandmaster melle mel duke bootee
Welcome to the Terrordome public enemy
Las Vegas Tango gary burton
But Not for Me the modern jazz quartet
Leroy the Magician gary burton
Protect Ya Neck wu-tang clan
Heidi Hoe common
The Surest Things Can Change freddie hubbard
Leaves pete jolly
Capricorn the cannonball adderley quintet
Something's Got a Hold on Me etta james
Momma Miss America paul mccartney
Build and Destroy boogie down productions
Hydra grover washington, jr.
Maybe Tomorrow grant green
Hostile erick sermon keith murray
Thisisme common
Going Back to Cali ll cool j
Cross Country archie whitewater
You're Getting a Little Too Smart detroit emeralds
Buggin Out a tribe called quest
The Message grandmaster flash furious five grandmaster melle mel duke bootee
I Know You Got Soul eric b.   rakim
But Not for Me the modern jazz quartet
Why Can't People Be Colors Too? the whatnauts
I Need Love ll cool j
Water No Get Enemy fela kuti
Next Lifetime erykah badu

Created playlist SAMPLIFY: Resurrection

Take note of Percentage Matched: 75.9%.
This is due to some songs not existing on Spotify, but some songs aren't returned
as there is too much detail being entered to the search (e.g producer credits considered as 'artists' on whosampled, but not on spotify.

# NOT ON SPOTIFY
Resurrection -> The Sorcerer of Isis (The Ritual of the Mole), by Power of Zeus
Watermelon -> Get Outta My Life Woman, by The New Apocalypse
# EXISTS ON SPOTIFY, BUT IMPEDANCE MISMATCH
In My Own World (Check The Method) -> Keep It Rollin', by A Tribe Called Quest feat. Large Professor

The current search strategy is "{SONG_TITLE} {ARTIST}", which is mostly okay, but it can be improved to reduce the amount of false negatives in searching Spotify for sampled tracks:

parse_sample_items in whosampled.py will need to be updated to grab each artist entity from the text component of links on the line item in each section.

The culprit here is header.find_next_sibling('div', attrs={'class': 'list bordered-list'}).text, which is split out into the detail section by parse_sample_items

# with summary page as soup of https://www.whosampled.com/Common/In-My-Own-World-(Check-the-Method)/
headings = soup.findAll('header', attrs={'class': 'sectionHeader'})
    for header in headings:
        for direction in self.directions:
            if direction not in header.span.text:
                continue
            match.append(self.parse_sample_items(
                song_title=song_title,
                sample_data=header.find_next_sibling(
                    'div', attrs={'class': 'list bordered-list'}).text,
                link=link,
                direction=direction
            ))

The value of artist in the sample detail will need be changed to an array, instead of just the unstructured text of 'artist': 'A Tribe Called Quest feat. Large Professor', and everywhere that indexes the detail by artist will require an update to handle list > string.

qzdl commented 4 years ago

Glad I left this pretty comprehensive description. This will be used in the clojure re-implementation @ https://github.com/qzdl/clj-samplify