jacquesh / foo_openlyrics

An open-source lyric display panel for foobar2000
MIT License
431 stars 24 forks source link

LyricFind.com returning errors. #391

Open Foobartius opened 2 weeks ago

Foobartius commented 2 weeks ago

Hello, there seems to be a problem with LyricFind. I tried with different radio stations and songs, and I always get an error. Even though those lyrics can be found on their website.

Steps to reproduce

  1. Open Panel
  2. Start radio stream
  3. LyricFind.com source returns errors, in the console
  4. Next source is able to retrieve the lyrics without issues

Expected behavior

Display lyrics that are available on that source's website (song from the included log).

Versions

Debug logs

INFO-OpenLyrics: No search avoidance info available for track
INFO-OpenLyrics: Searching for lyrics for artist='Phil Collins', album='', title='I Missed Again'...
INFO-OpenLyrics: Page https://lyrics.lyricfind.com/lyrics/phil-collins-i-missed-again retrieved
WARN-OpenLyrics: Received lyricfind result but the root json object was malformed: 
ERROR-OpenLyrics: Error of unrecognised type while searching LyricFind.com
INFO-OpenLyrics: Failed to retrieve lyrics from source: LyricFind.com
INFO-OpenLyrics: Querying for lyrics from https://www.azlyrics.com/lyrics/philcollins/imissedagain.html...
INFO-OpenLyrics: Successfully retrieved lyrics from https://www.azlyrics.com/lyrics/philcollins/imissedagain.html
INFO-OpenLyrics: Successfully retrieved lyrics from source: AZLyrics.com
INFO-OpenLyrics: Parsing lyrics text...
INFO-OpenLyrics: Loaded lyrics already form a valid UTF-8 sequence
INFO-OpenLyrics: Parsing LRC lyric text...
INFO-OpenLyrics: Lyric loading complete
INFO-OpenLyrics: Skipping lyric save. Type: 1, Local: no, Timestamped: no, Autosave: 0
INFO-OpenLyrics: No lyric metadata available for track

Additional information

I haven't encountered this with any of the other sources.

Thank you for your time.

jacquesh commented 2 weeks ago

I don't think its possible to completely avoid this. LyricFind sits behind Cloudflare and so in some cases will present a captcha (I've run into this myself when loading the site in a browser). OpenLyrics obviously can't (or isn't going to try to) solve the captcha. The best I could do is detect it and log accordingly. I expect LyricFind will work sporadically and will probably work better the less you query it.

My guess is that this is what's happening when it complains that the resulting page was "malformed". At the very least I can confirm that it works for me some of the time, and in fact I just tried myself locally and I got a result for "Phil Collins/I Missed Again" so it's not a case of "some pages have a different format".

Rexadev commented 2 weeks ago

LyricFInd is GeoBlocked and blocks VPN. Try open https://lyrics.lyricfind.com/lyrics/phil-collins-i-missed-again in browser. Sometimes website allows me sometimes it doesn't.

Foobartius commented 2 weeks ago

I don't think its possible to completely avoid this. LyricFind sits behind Cloudflare and so in some cases will present a captcha (I've run into this myself when loading the site in a browser).

Oh, I see. I didn't consider that. Because, in the days that I've been troubleshooting this problem, I haven't gotten a captcha request and it always shows me the lyrics correctly (using Firefox),

At the very least I can confirm that it works for me some of the time, and in fact I just tried myself locally and I got a result for "Phil Collins/I Missed Again" so it's not a case of "some pages have a different format".

I haven't been able to get lyrics from them, it always shows that error. (Very sporadically it adds some more info, including country codes, maybe that's a result of the captcha. I couldn't reproduce it after several tries, even restarting Foobar, so I didn't include that log) But, I search for it immediately after, using my web browser, and everything works fine there.

Well, thank you for having looked into this and for your reply. I thought it might have been a problem with using OL with an internet radio stream.

Foobartius commented 2 weeks ago

LyricFInd is GeoBlocked and blocks VPN. Try open https://lyrics.lyricfind.com/lyrics/phil-collins-i-missed-again in browser. Sometimes website allows me sometimes it doesn't.

Yeah, I just tried that again and it takes me directly to the lyrics. It's never shown me any captcha or block page (during the last several days when I started noticing this and double checking with FIrefox).

Maybe it's because I'm outside of the U.S. For some reason, perhaps something works OK when Firefox shows it some of my info (or the user agent or something...). But, when using Foobar their service may think I'm a bot or something... I don't know. :(

jacquesh commented 2 weeks ago

It's entirely possible that there are some headers we could add to the request to reduce the error rate. If you could reproduce the error using curl or something like that so we could see the exact output you're getting then that might help.

I'm not in the US either so if it's a location issue then it is at least not US-specific.

Foobartius commented 2 weeks ago

It's entirely possible that there are some headers we could add to the request to reduce the error rate. If you could reproduce the error using curl or something like that so we could see the exact output you're getting then that might help.

I'm sorry, could you please be so kind as to point me to a source that shows how to do that? I would appreciate it.

I'm now using OL 1.11 and, for the first time, I got a VERY long log from this source. Usually, I would only see either nothing super specific or, every now and then, some few lines detailing the error. But, maybe this can help, so here's the new (very long) log:

INFO-OpenLyrics: No search avoidance info available for track
INFO-OpenLyrics: Searching for lyrics for artist='Phil Collins', album='', title='Dance Into The Light'...
INFO-OpenLyrics: Page https://lyrics.lyricfind.com/lyrics/phil-collins-dance-into-the-light retrieved
WARN-OpenLyrics: Received lyricfind result but duration was malformed: {"props":{"pageProps":{"songData":{"response":{"code":101,"description":"SUCCESS: LICENSE, LYRICS"},"track":{"lfid":"002-595895","language":"en","rovi":"MT0002602887","gracenote":"GN9R4CAHQ8KMBHJ","apple":1081948508,"isrcs":["USRH11509276"],"instrumental":false,"viewable":true,"has_lrc":false,"has_elrc":false,"has_contentfilter":false,"has_emotion":true,"has_sentiment":true,"title":"Dance into the Light [DVD]","titleSimple":"Dance into the Light","version":"DVD","artists":[{"name":"Phil Collins","lfid":"lf:21775","slug":"phil-collins","is_primary":true}],"artist":{"name":"Phil Collins"},"last_update":"2023-01-01 00:00:00","lyrics":"It's there in the eyes of the children \nIn the faces smiling in the windows \nYou can come on out, come on open the doors \nBrush away the tears of freedom \n\nNow we're here, there's no turning back \nWe have each other \nWe have one voice \n\nHand in hand we will lay the tracks \nBecause the train is coming to carry you home \n\nCome dance with me \nCome on and dance into the light \nEverybody dance into the light \n\nThere'll be no more hiding in shadows of fear \nThere'll be no more chains to hold you \nThe future is yours - you hold the key \nAnd there are no walls with freedom \n\nNow we're here, we won't go back \nWe are one world \nWe have one voice \n\nSide by side we are not afraid \nBecause the train is coming to carry you home \n\nCome dance with me \nCome on and dance into the light \nEverybody dance into the light \n\nDo you see the sun, it's a brand new day \nOh, the world's in your hands, now use it \nWhat's past is past, don't turn around \nBrush away the cobwebs of freedom \n\nNow we're here, there's no turning back \nYou have each other \nYou have one voice \n\nHand in hand you can lay the tracks \nBecause the train is coming to carry you home \n\nCome dance with me \nCome on and dance into the light \nEverybody dance into the light","copyright":"Lyrics © CONCORD MUSIC PUBLISHING LLC, Peermusic Publishing","writer":"Phillip David Charles Collins","slug":"phil-collins-dance-into-the-light"}},"artistSlug":"phil-collins","albumSlug":"","countryCode":"MX","songIsInstrumental":false,"languageToLyricTranslationMap":{},"dehydratedState":{"mutations":[],"queries":[{"state":{"data":{"response":{"code":100,"description":"SUCCESS"},"totalresults":14,"totalpages":1,"albums":[{"id":"ms:1613716","title":"Other Sides","releaseYear":2019,"coverArt":"cover_art/curation/original/3/9/2/6/86a3-6d61-4e54-a305-d5aacda27424.jpg","artists":[{"name":"Phil Collins","lfid":"lf:21775","slug":"phil-collins"}],"artist":{"name":"Phil Collins"},"mainArtist":{"name":"Phil Collins"},"genres":[{"name":"Rock","slug":"rock"}],"slug":"phil-collins-other-sides"},{"id":"ms:1613720","title":"Remixed Sides","releaseYear":2019,"coverArt":"cover_art/curation/original/5/3/d/a/17ef-392c-466e-95f1-476acd73b32d.jpg","artists":[{"name":"Phil Collins","lfid":"lf:21775","slug":"phil-collins"}],"artist":{"name":"Phil Collins"},"mainArtist":{"name":"Phil Collins"},"genres":[{"name":"Rock","slug":"rock"}],"slug":"phil-collins-remixed-sides"},{"id":"ms:128735","title":"Going Back","releaseYear":2010,"coverArt":"cover_art/curation/original/e/7/d/b/8cc9-5025-464e-a76e-57670cd6a12b.jpeg","artists":[{"name":"Phil Collins","lfid":"lf:21775","slug":"phil-collins"}],"artist":{"name":"Phil Collins"},"mainArtist":{"name":"Phil Collins"},"genres":[{"name":"Blue-eyed soul","slug":"blue_eyed_soul"}],"slug":"phil-collins-going-back"},{"id":"ms:2357331","title":"No Way Out (Theme From Brother Bear)","releaseYear":2004,"artists":[{"name":"Phil Collins","lfid":"lf:21775","slug":"phil-collins"}],"artist":{"name":"Phil Collins"},"mainArtist":{"name":"Phil Collins"},"genres":[{"name":"Rock","slug":"rock"}],"slug":"phil-collins-no-way-out-theme-from-brother-bear"},{"id":"ms:2357332","title":"No Way Out (Theme From Brother Bear)","releaseYear":2003,"artists":[{"name":"Phil Collins","lfid":"lf:21775","slug":"phil-collins"}],"artist":{"name":"Phil Collins"},"mainArtist":{"name":"Phil Collins"},"genres":[{"name":"Rock","slug":"rock"}],"slug":"phil-collins-no-way-out-theme-from-brother-bear-1"},{"id":"ms:40289","title":"Testify","releaseYear":2002,"coverArt":"cover_art/curation/original/7/f/2/4/47bd-98ca-464f-8504-7cb3ac5db916.jpeg","artists":[{"name":"Phil Collins","lfid":"lf:21775","slug":"phil-collins"}],"artist":{"name":"Phil Collins"},"mainArtist":{"name":"Phil Collins"},"genres":[{"name":"Soft rock","slug":"soft_rock"}],"slug":"phil-collins-testify"},{"id":"ms:151109","title":"Tarzan [B.O.F.]","releaseYear":1999,"coverArt":"cover_art/curation/original/6/0/e/b/6c60-e0e5-4272-a02c-1ecac3d24142.jpeg","artists":[{"name":"Mark Mancina","lfid":"lf:38360","slug":"mark-mancina"},{"name":"Phil Collins","lfid":"lf:21775","slug":"phil-collins"}],"artist":{"name":"Mark Mancina and Phil Collins"},"mainArtist":{"name":"Mark Mancina"},"genres":[{"name":"Film score","slug":"film_score"}],"slug":"phil-collins-tarzan-b-o-f"},{"id":"ms:27464","title":"Dance into the Light","releaseYear":1996,"coverArt":"cover_art/curation/original/e/6/0/a/925a-98da-4e07-94ff-b93209cf93e7.jpeg","artists":[{"name":"Phil Collins","lfid":"lf:21775","slug":"phil-collins"}],"artist":{"name":"Phil Collins"},"mainArtist":{"name":"Phil Collins"},"genres":[{"name":"AM pop rock","slug":"am_pop_rock"}],"slug":"phil-collins-dance-into-the-light"},{"id":"ms:27468","title":"Both Sides","releaseYear":1993,"coverArt":"cover_art/curation/original/b/1/7/e/2404-90ad-4256-90a6-87e2a6d6f22c.jpeg","artists":[{"name":"Phil Collins","lfid":"lf:21775","slug":"phil-collins"}],"artist":{"name":"Phil Collins"},"mainArtist":{"name":"Phil Collins"},"genres":[{"name":"Soft rock","slug":"soft_rock"}],"slug":"phil-collins-both-sides"},{"id":"ms:27466","title":"...But Seriously","releaseYear":1989,"coverArt":"cover_art/curation/original/1/c/2/c/b88f-0dda-4113-8fc1-efaf7c34cc0b.jpeg","artists":[{"name":"Phil Collins","lfid":"lf:21775","slug":"phil-collins"}],"artist":{"name":"Phil Collins"},"mainArtist":{"name":"Phil Collins"},"genres":[{"name":"Soft rock","slug":"soft_rock"}],"slug":"phil-collins-but-seriously"},{"id":"ms:2601249","title":"A Groovy Kind Of Love","releaseYear":1988,"artists":[{"name":"Phil Collins","lfid":"lf:21775","slug":"phil-collins"}],"artist":{"name":"Phil Collins"},"mainArtist":{"name":"Phil Collins"},"genres":[{"name":"Rock","slug":"rock"}],"slug":"phil-collins-a-groovy-kind-of-love"},{"id":"ms:27467","title":"No Jacket Required","releaseYear":1985,"coverArt":"cover_art/curation/original/3/9/d/1/6bf9-1229-4ccd-bf1c-f2d1cdaa4b6b.jpeg","artists":[{"name":"Phil Collins","lfid":"lf:21775","slug":"phil-collins"}],"artist":{"name":"Phil Collins"},"mainArtist":{"name":"Phil Collins"},"genres":[{"name":"AM pop rock","slug":"am_pop_rock"}],"slug":"phil-collins-no-jacket-required"},{"id":"ms:27471","title":"Hello, I Must Be Going!","releaseYear":1982,"coverArt":"cover_art/curation/original/b/7/7/3/cef9-9f07-478e-9f0f-9bfcb2399726.jpeg","artists":[{"name":"Phil Collins","lfid":"lf:21775","slug":"phil-collins"}],"artist":{"name":"Phil Collins"},"mainArtist":{"name":"Phil Collins"},"genres":[{"name":"AM pop rock","slug":"am_pop_rock"}],"slug":"phil-collins-hello-i-must-be-going"},{"id":"ms:27465","title":"Face Value","releaseYear":1981,"coverArt":"cover_art/curation/original/9/c/6/1/f76c-274d-4c98-87e0-5886f22fa3d3.jpeg","artists":[{"name":"Phil Collins","lfid":"lf:21775","slug":"phil-collins"}],"artist":{"name":"Phil Collins"},"mainArtist":{"name":"Phil Collins"},"genres":[{"name":"Pop rock","slug":"pop_rock"}],"slug":"phil-collins-face-value"}]},"dataUpdateCount":1,"dataUpdatedAt":1725776589780,"error":null,"errorUpdateCount":0,"errorUpdatedAt":0,"fetchFailureCount":0,"fetchFailureReason":null,"fetchMeta":null,"isInvalidated":false,"status":"success","fetchStatus":"idle"},"queryKey":["artistAlbums","phil-collins","MX",30,0],"queryHash":"[\"artistAlbums\",\"phil-collins\",\"MX\",30,0]"},{"state":{"data":{"response":{"code":201,"description":"INVALID REQUEST","message":"The value of the 'slug' albumid field is not provided."}},"dataUpdateCount":1,"dataUpdatedAt":1725776589784,"error":null,"errorUpdateCount":0,"errorUpdatedAt":0,"fetchFailureCount":0,"fetchFailureReason":null,"fetchMeta":null,"isInvalidated":false,"status":"success","fetchStatus":"idle"},"queryKey":["albumTrack","","MX"],"queryHash":"[\"albumTrack\",\"\",\"MX\"]"}]},"lyricIqData":{"response":{"code":100,"description":"SUCCESS"},"track":{"lfid":"002-595895","language":"en","rovi":"MT0002602887","gracenote":"GN9R4CAHQ8KMBHJ","apple":1081948508,"isrcs":["USRH11509276"],"instrumental":false,"viewable":true,"has_lrc":false,"has_elrc":false,"has_contentfilter":false,"has_emotion":true,"has_sentiment":true,"title":"Dance into the Light [DVD]","titleSimple":"Dance into the Light","version":"DVD","artists":[{"name":"Phil Collins","lfid":"lf:21775","slug":"phil-collins","is_primary":true}],"artist":{"name":"Phil Collins"},"last_update":"2023-01-01 00:00:00","snippet":"It's there in the eyes of the children \r\nIn the faces smiling in the windows \r\nYou can come on out, come on open the doors \r\nBrush away the tears of f...","lyriciq":{"emotions":{"anger":0.04933,"disgust":0.00864,"fear":0.206843,"joy":0.746397,"sadness":0.035133},"sentiment":0.375526},"slug":"phil-collins-dance-into-the-light"}}},"__N_SSP":true},"page":"/lyrics/[songSlug]","query":{"songSlug":"phil-collins-dance-into-the-light"},"buildId":"vi-1owvNMLZ3EHpBM0kyp","isFallback":false,"dynamicIds":[32302],"gssp":true,"locale":"en-US","locales":["en-US","en-GB"],"defaultLocale":"en-US","scriptLoader":[]}
ERROR-OpenLyrics: Error of unrecognised type while searching LyricFind.com
INFO-OpenLyrics: Failed to retrieve lyrics from source: LyricFind.com
INFO-OpenLyrics: Querying for track ID from https://apic-desktop.musixmatch.com/ws/1.1/track.search?user_language=en&app_id=web-desktop-app-v1.0&subtitle_format=lrc&q_artist=Phil%20Collins&q_album=&q_track=Dance%20Into%20The%20Light&usertoken=
INFO-OpenLyrics: Get Musixmatch lyrics from https://apic-desktop.musixmatch.com/ws/1.1/track.subtitle.get?user_language=en&app_id=web-desktop-app-v1.0&commontrack_id=41139&usertoken=
INFO-OpenLyrics: Successfully looked-up lyrics from source: Musixmatch
INFO-OpenLyrics: Parsing lyrics text...
INFO-OpenLyrics: Loaded lyrics already form a valid UTF-8 sequence
INFO-OpenLyrics: Parsing LRC lyric text...
INFO-OpenLyrics: Lyric loading complete
INFO-OpenLyrics: Skipping lyric save. Type: 1, Local: no, Timestamped: yes, Autosave: 0
INFO-OpenLyrics: No lyric metadata available for track
Foobartius commented 2 weeks ago

Update: The crazy thing is that, after getting the error reported above, I now finally got lyrics from this source, for the very first time:

INFO-OpenLyrics: No search avoidance info available for track
INFO-OpenLyrics: Searching for lyrics for artist='Phil Collins', album='', title='Thru These Walls'...
INFO-OpenLyrics: Page https://lyrics.lyricfind.com/lyrics/phil-collins-thru-these-walls retrieved
INFO-OpenLyrics: Successfully retrieved lyrics from https://lyrics.lyricfind.com/lyrics/phil-collins-thru-these-walls
INFO-OpenLyrics: Successfully retrieved lyrics from source: LyricFind.com
INFO-OpenLyrics: Parsing lyrics text...
INFO-OpenLyrics: Loaded lyrics already form a valid UTF-8 sequence
INFO-OpenLyrics: Parsing LRC lyric text...
INFO-OpenLyrics: Lyric loading complete
INFO-OpenLyrics: Skipping lyric save. Type: 1, Local: no, Timestamped: no, Autosave: 0

Same radio station, since I opened this issue.

I wonder if I changed something with my Foobar or OL settings. But, I can't think of what that might be... Did something change with OL 1.11 that could affect this in any way?

jacquesh commented 2 weeks ago

Did something change with OL 1.11 that could affect this in any way?

No