search

SwagLyrics / SwagLyrics

Main repo to unify projects.
MIT License
6 stars 1 forks source link

Potentially switch from Genius to Google #10

Open aadibajpai opened 5 years ago

aadibajpai commented 5 years ago

When SwagLyrics was first created, Google did not actually have lyrics hence Genius was an appropriate choice.

However, Google obviously has far more processing power and indexing storage to have lyrics from other lyrics provider than Genius and hence supports more songs especially for non Hollywood songs too.

Additionally, with Google you don't need to be exact either as it is actually searching for lyrics so terms like remix wouldn't affect the outcome.

Hence, it might be a better choice now to use Google.

But there needs to be a potential test to see how beneficial Google would be over Genius as well as a potential time increment if there is.

Essentially a cost-benefit analysis lol.

GermaVinsmoke commented 5 years ago

But with Genius, we can scrap the cover image which can be used in --tab. It will make the localhost look good, UI wise.

aadibajpai commented 5 years ago

That also adds more data to fetch hence reduces speed and the whole minimal thing we've got going on.

GermaVinsmoke commented 5 years ago

Fetching lyrics like this is taking a maximum of 10 seconds. I made a new function to fetch the cover image so that's also taking more time, can be done within get_lyrics function but then the return statement has to be changed, it should return a tuple which will contain lyrics and cover image.

I think showing the cover picture will make the UI look a little bit better.

image

And check this part in this I selected Don't be a fool and it started fetching the lyrics, as soon as it started fetching I changed the song but it was fetching the previous song only (Completing that request) after it was completed, the second request was processed to fetch the new song.

image

aadibajpai commented 5 years ago

@GermaVinsmoke I honestly don't see adding the album art to SwagLyrics as an advantage.

Yes, I guess a request will be completed once it starts.

I'm more interested in seeing how Google compares to Genius in terms of speed and lyrics support.

TarunKurella commented 5 years ago

You might not want to scrap Google. They have quite tight security standards. It too easy to fend of by anit-bot mechanisms.

TarunKurella commented 5 years ago

Maybe we can look upon further improving the stripper, by utilizing various other providers, and pythons multiprocessing modules to handle requests asynchronously.

aadibajpai commented 5 years ago

@TarunKurella that sounds pretty vague. Do you want to elaborate why we shouldn't use Google? The only thing I'm not sure of is speed but apart from it I think google can support more songs as it itself scrapes from multiple lyrics providers so we could have support for popular songs which are not english as well.

anishk74 commented 5 years ago

@aadibajpai We can use selenium to get lyrics from google using headless browser. A demo implementation to get lyrics from google is here https://github.com/anishk74/scrap-lyrics.git

aadibajpai commented 5 years ago

@aadibajpai We can use selenium to get lyrics from google using headless browser. A demo implementation to get lyrics from google is here https://github.com/anishk74/scrap-lyrics.git

That's quite slow. Your implementation will take > 20s

anishk74 commented 5 years ago

@aadibajpai Sir I have removed the test commands. It now takes around 5 second to perform the same job. You can have a look at it https://github.com/anishk74/scrap-lyrics.git

anishk74 commented 5 years ago

@aadibajpai Sir here's a comparison between current implementation of SwagLyrics which finds Lyrics from Genius and the other which finding lyrics from Google. https://github.com/anishk74/lyrics-from.git By my observation, it's beneficial to use Google for finding lyrics in most of the cases .

aadibajpai commented 5 years ago

What's the result? You could do all sorts of statistical analysis 😉

anishk74 commented 5 years ago

@aadibajpai Sir, in most of the cases, scraping lyrics from Google takes almost half of the time as compared to the current implementation and I can safely say its lesser for all cases. Also Google supports lyrics for more songs over Genius. Moreover it is really not necessary to find the artist name in case of Google lyrics because it can work either way. So it will also increase the time efficiency. You can have a look at the comparison here: https://github.com/anishk74/scrap-lyrics.git I would like to listen from your side about this and would like to ask for permission to work on introducing Google lyrics to SwagLyrics.

aadibajpai commented 5 years ago

@anishk74 that's all qualitative and I don't want to take your word for it 🙂

It would be better to have a quantitative analysis is what I'm saying.

anishk74 commented 5 years ago

@aadibajpai Sir, I couldn't get it. Can you please explain what factors I should include in quantitative analysis?

aadibajpai commented 5 years ago

@anishk74 time taken as well as song support.

anishk74 commented 5 years ago

The programs here https://github.com/anishk74/scrap-lyrics.git also calculate the time taken to get lyrics from both providers. Time taken in case of Google lyrics is lesser than the time taken by Genius. Also till now, I have never encountered a song for which genius provides lyrics and google doesn't. but there are many songs(for ex. Let U Know by Bugy (The most recent issue of SwagLyrics-For-Spotify)) for which Google provides lyrics but Genius couldn't.

GermaVinsmoke commented 5 years ago

Aimer after rain Google can't Genius can Provide its lyrics :)

anishk74 commented 5 years ago

@GermaVinsmoke oh yes! Genius provides its lyrics. But it is also true that hit-rate is higher for Google in case of most of the songs. So we might go for searching lyrics from both providers. Since time taken by Google to find lyrics is lesser and also hit-rate is high for Google so for better efficiency SwagLyrics must ask Google first for lyrics and if unavailable then go for Genius to find .

anishk74 commented 5 years ago

@aadibajpai sir, I would like to listen from you what technology you want to see seaching for lyrics. Should we continue with the current implementation of Genius or should we introduce google to search for lyrics?

aadibajpai commented 5 years ago

@aadibajpai sir, I would like to listen from you what technology you want to see seaching for lyrics. Should we continue with the current implementation of Genius or should we introduce google to search for lyrics?

I haven't seen evidence to convince me to change anything yet so I don't think I'm going to change anything as of now.