search

kmille / freetar

freetar - an alternative frontend to ultimate-guitar.com
GNU General Public License v3.0
634 stars 28 forks source link

Client error 403 #43

Open theilgaard opened 3 weeks ago

theilgaard commented 3 weeks ago

Getting this while searching for a song.

Oops: Could not search for chords: 403 Client Error: Forbidden for url: https://www.ultimate-guitar.com/search.php?search_type=title&value=mr%20jones

kmille commented 3 weeks ago

Which instance do you use? UG started to block IPs. I tried to fix it by using Tor (don't know if/how often external ip s are changed). Ah, I see that now even with Tor it gets blocked.

Commit: https://github.com/kmille/freetar/commit/f0ab6a38e4b1141b30c0843b338d4973b0e2cdb0

4oo4 commented 2 days ago

Still getting this error on the main instance (freetar.de), but other instances seem to be OK. One solution might be to use AWS API Gateway to proxy requests to UG from a very large pool of IP addresses. API Gateway isn't free but it's extremely cheap.

https://github.com/Ge0rg3/requests-ip-rotator

kmille commented 1 day ago

Still getting this error on the main instance (freetar.de), but other instances seem to be OK. One solution might be to use AWS API Gateway to proxy requests to UG from a very large pool of IP addresses. API Gateway isn't free but it's extremely cheap.

https://github.com/Ge0rg3/requests-ip-rotator

Hey, that sound's interesting. Also thought about adding a simple caching. The thing with requrests-ip-rotator is:

Please note that these requests can be easily identified and blocked, since they are sent with unique AWS headers (i.e. "X-Amzn-Trace-Id").

Cloudflare will detect and block this (probably?). Also see this issue: https://github.com/Ge0rg3/requests-ip-rotator/issues/51. Maybe some of you has a Cloudflare account and can check that?

Hmm, but this seems to work:

curl 'https://www.ultimate-guitar.com/search.php?search_type=title&value=adele' -H "X-Amzn-Trace-Id: 123" > ~/Downloads/index.html