Invalid URL

[Giloh7]

Hello,

Recently I got many issues with track recognition.

Here is the exception I get (not on every song but very frequently) Error occurred during recognition: Check args, URL is invalid URL- https://amp.shazam.com/discovery/v5/en-US/GB/iphone/-/tag/69C006D3-5A23-4799-8C9A-5A21C5505B91/47C35018-6964-44CC-A7F4-18791B76949A?sync=true&webv3=true&sampling=true&connected=&shazamapiversion=v3&sharehub=true&hubv5minorversion=v5.1&hidelb=true&video=v3

Maybe something changed Shazam side ?

Thank you :)

[AMAMazing]

Yea i've gotten same issues just today aswell

[AMAMazing]

My temporary fix has been adding time.sleep inbetween use. More specifically this:

retry_count = 0 max_retries = 15 while retry_count < max_retries: sleep(retry_count) temp_file = f"song.mp3" result = await recognize_song(song)

[Giloh7]

Thank you for the temporary fix AMAM :)

[ryan-dockery-harkerbos]

I have also noticed this issue for the first time today.

[dotX12]

A couple of people wrote to me on telegram with this problem... I do not actively use song recognition, I only support the library for u, so I could not reproduce this error. But I can assume that nginx or cdn returns text/plain due to the rate limit. I'll test it today and see what I can do. But the alternative is to use a proxy or do the retry recognize.

https://github.com/dotX12/ShazamIO/blob/17627a805c5ac228b9dab42521591ac494001d6a/shazamio/utils.py#L21-L28

[AMAMazing]

It seems to be about a 30 second cooldown between uses

[mikemutex]

I sent https://github.com/dotX12/ShazamIO/pull/82 to fix this with an exponential backoff whenever an HTTP 429 is returned.

[DiyorbekAhmadjonov]

I have the same problem

[AMAMazing]

I could very well be wrong but I think its just gone back to normal

[AMAMazing]

Wait nvm only lasted an hour

[DiyorbekAhmadjonov]

Is it possible that Shazam has set a limit?

[eddieespinal]

I was facing the same issue and I added a delay with a retry logic to ensure that the last chunk gets processed properly. This is working fine for my implementation. I hope this helps others with this issue for now 🤷🏻‍♂️.

async def identify_song(filepath, retries=3, delay=15):
    shazam = Shazam()
    for attempt in range(retries):
        try:
            return await shazam.recognize_song(filepath)
        except Exception as e:
            print(f"Attempt {attempt + 1} failed with error: {e}")
            if attempt < retries - 1:
                await asyncio.sleep(delay * (attempt + 1))  # Exponential backoff
            else:
                print("Max retries reached. Moving to next chunk.")
    return None

  result = await identify_song(temp_file.name)

[kimherman]

@eddieespinal Thank you for the temporary workaround - works for me but any run just takes so much longer because of the delays and retries of course.

@dotX12 any view on when this will be merged into the library because there is definitely a change on the API side.

[dotX12]

I did research and see that shazam began to tighten the screws, after a certain number of requests (it is different each time, on average after 15-20 quick requests) error 429 starts, which lasts, according to my calculations, about 30 seconds.

[dotX12]

[dotX12]

I will add an exponential delay of up to 60 seconds, but you should now understand that waiting for a response from Shazam can be a long time (up to 30-40 seconds if you are processing a large number of songs). It is quite possible that several proxies will allow you to process more songs and not end up in 429.

[dotX12]

The fix will include:

• Ability to pass your own HTTPClient if you want to override pre/post request logic or replace aiohttp with something else.
• aiohttp_retry for retries for 500, 502, 503, 504, 429
• Logging base.
• Ability to use proxies.

[Giloh7]

Thank you :)

[dotX12]

Patch note. https://github.com/shazamio/ShazamIO/pull/91#issuecomment-1960536271 https://github.com/shazamio/ShazamIO/pull/91#issuecomment-1960547524

I plan to release a new version in a couple of hours.

[dotX12]

Better late than never, but it happened thanks to pyo3 and RUST. https://github.com/dotX12/shazamio-core

Fixed in 0.5.0. See comment: https://github.com/shazamio/ShazamIO/issues/76#issuecomment-1961610344 And see full release note: https://github.com/shazamio/ShazamIO/releases/tag/0.5.0

[dotX12]

Give feedback, is the problem fixed in the new version? @DiyorbekAhmadjonov @AMAMazing @eddieespinal @mikemutex @kimherman @Giloh7

[DiyorbekAhmadjonov]

the problem is solved, the speed has also increased, it is working stably with the help of proxy

[DiyorbekAhmadjonov]

I use proxies randomly, there is no limit

[mikemutex]

Thanks so much @dotX12 for the update! It seems to work much better now, and I can send more requests before getting throttled.

I think there are some rare cases where the 429 response ends up raising an exception in validate_json though. Here's a traceback:

<ClientResponse(https://amp.shazam.com/discovery/v5/en-US/GB/android/-/tag/766CA341-CD42-4B14-93EA-EB6F2A2558D3/49AC552E-E8A0-41FC-BA43-14B948352F3B?sync=true&webv3=true&sampling=true&connected=&shazamapiversion=v3&sharehub=true&hubv5minorversion=v5.1&hidelb=true&video=v3) [429 Too Many Requests]>
<CIMultiDictProxy('Content-Length': '142', 'Content-Type': 'text/html; charset=UTF-8', 'Date': 'Sat, 24 Feb 2024 19:21:50 GMT', 'Alt-Svc': 'h3=":443"; ma=2592000,h3-29=":443"; ma=2592000')>

Traceback (most recent call last):
  File "/opt/homebrew/lib/python3.11/site-packages/shazamio/utils.py", line 23, in validate_json
    return await resp.json(content_type=content_type)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/aiohttp/client_reqrep.py", line 1105, in json
    raise ContentTypeError(
aiohttp.client_exceptions.ContentTypeError: 0, message='Attempt to decode JSON with unexpected mimetype: text/html; charset=utf-8', url=URL('https://amp.shazam.com/discovery/v5/en-US/GB/android/-/tag/766CA341-CD42-4B14-93EA-EB6F2A2558D3/49AC552E-E8A0-41FC-BA43-14B948352F3B?sync=true&webv3=true&sampling=true&connected=&shazamapiversion=v3&sharehub=true&hubv5minorversion=v5.1&hidelb=true&video=v3')

The above exception was the direct cause of the following exception:

...
  File "/opt/homebrew/lib/python3.11/site-packages/shazamio/api.py", line 487, in recognize_song
    return await self.send_recognize_request(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/shazamio/api.py", line 503, in send_recognize_request
    return await self.http_client.request(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/shazamio/client.py", line 63, in request
    raise e
  File "/opt/homebrew/lib/python3.11/site-packages/shazamio/client.py", line 60, in request
    return await validate_json(resp, *args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/shazamio/utils.py", line 25, in validate_json
    raise FailedDecodeJson("Failed to decode json") from e
shazamio.exceptions.FailedDecodeJson: Failed to decode json

It's strange because I see that in api.py you specify 429 as one of the retried statuses, so I'm not sure why validate_json still gets called on the 429 response.

[dotX12]

@mikemutex, hello!

Fix: https://pypi.org/project/shazamio/0.5.1/ https://github.com/shazamio/ShazamIO/releases/tag/0.5.1

It looks like the current timeout time is not enough, you end up in max attempts + max timeout 30 seconds and after 429 error. A fix has been added to increase this max_timeout to 60 seconds.

And also if 60 seconds is not enough for you, you can override the wait timeout.

shazam = Shazam(
    http_client=HTTPClient(
        retry_options=ExponentialRetry(attempts=12, max_timeout=204.8, statuses={500, 502, 503, 504, 429}),
    ),
)

Default retry table (attempt/seconds)

[
0.1,
 0.2,
 0.4,
 0.8,
 1.6,
 3.2,
 6.4,
 12.8,
 25.6,
 51.2,
 102.4,
 204.8,
 409.6,
 819.2,
 1638.4,
 3276.8,
 6553.6,
 13107.2,
 26214.4,
 52428.8
 ]

---

But I'm afraid this won't be enough, it seems that using a proxy is the only option for those who send too many requests.

[dotX12]

Thanks so much @dotX12 for the update! It seems to work much better now, and I can send more requests before getting throttled.

I think there are some rare cases where the 429 response ends up raising an exception in validate_json though. Here's a traceback:

<ClientResponse(https://amp.shazam.com/discovery/v5/en-US/GB/android/-/tag/766CA341-CD42-4B14-93EA-EB6F2A2558D3/49AC552E-E8A0-41FC-BA43-14B948352F3B?sync=true&webv3=true&sampling=true&connected=&shazamapiversion=v3&sharehub=true&hubv5minorversion=v5.1&hidelb=true&video=v3) [429 Too Many Requests]>
<CIMultiDictProxy('Content-Length': '142', 'Content-Type': 'text/html; charset=UTF-8', 'Date': 'Sat, 24 Feb 2024 19:21:50 GMT', 'Alt-Svc': 'h3=":443"; ma=2592000,h3-29=":443"; ma=2592000')>

Traceback (most recent call last):
  File "/opt/homebrew/lib/python3.11/site-packages/shazamio/utils.py", line 23, in validate_json
    return await resp.json(content_type=content_type)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/aiohttp/client_reqrep.py", line 1105, in json
    raise ContentTypeError(
aiohttp.client_exceptions.ContentTypeError: 0, message='Attempt to decode JSON with unexpected mimetype: text/html; charset=utf-8', url=URL('https://amp.shazam.com/discovery/v5/en-US/GB/android/-/tag/766CA341-CD42-4B14-93EA-EB6F2A2558D3/49AC552E-E8A0-41FC-BA43-14B948352F3B?sync=true&webv3=true&sampling=true&connected=&shazamapiversion=v3&sharehub=true&hubv5minorversion=v5.1&hidelb=true&video=v3')

The above exception was the direct cause of the following exception:

...
  File "/opt/homebrew/lib/python3.11/site-packages/shazamio/api.py", line 487, in recognize_song
    return await self.send_recognize_request(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/shazamio/api.py", line 503, in send_recognize_request
    return await self.http_client.request(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/shazamio/client.py", line 63, in request
    raise e
  File "/opt/homebrew/lib/python3.11/site-packages/shazamio/client.py", line 60, in request
    return await validate_json(resp, *args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/shazamio/utils.py", line 25, in validate_json
    raise FailedDecodeJson("Failed to decode json") from e
shazamio.exceptions.FailedDecodeJson: Failed to decode json

It's strange because I see that in api.py you specify 429 as one of the retried statuses, so I'm not sure why validate_json still gets called on the 429 response.

There is no infinite retry in the code; as soon as the number of attempts is spent, we will receive an error. I don’t see the point in making an unlimited number of attempts, otherwise the task may never be completed if we get a complete IP block.

[dotX12]

Thanks so much @dotX12 for the update! It seems to work much better now, and I can send more requests before getting throttled.

I think there are some rare cases where the 429 response ends up raising an exception in validate_json though. Here's a traceback:

<ClientResponse(https://amp.shazam.com/discovery/v5/en-US/GB/android/-/tag/766CA341-CD42-4B14-93EA-EB6F2A2558D3/49AC552E-E8A0-41FC-BA43-14B948352F3B?sync=true&webv3=true&sampling=true&connected=&shazamapiversion=v3&sharehub=true&hubv5minorversion=v5.1&hidelb=true&video=v3) [429 Too Many Requests]>
<CIMultiDictProxy('Content-Length': '142', 'Content-Type': 'text/html; charset=UTF-8', 'Date': 'Sat, 24 Feb 2024 19:21:50 GMT', 'Alt-Svc': 'h3=":443"; ma=2592000,h3-29=":443"; ma=2592000')>

Traceback (most recent call last):
  File "/opt/homebrew/lib/python3.11/site-packages/shazamio/utils.py", line 23, in validate_json
    return await resp.json(content_type=content_type)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/aiohttp/client_reqrep.py", line 1105, in json
    raise ContentTypeError(
aiohttp.client_exceptions.ContentTypeError: 0, message='Attempt to decode JSON with unexpected mimetype: text/html; charset=utf-8', url=URL('https://amp.shazam.com/discovery/v5/en-US/GB/android/-/tag/766CA341-CD42-4B14-93EA-EB6F2A2558D3/49AC552E-E8A0-41FC-BA43-14B948352F3B?sync=true&webv3=true&sampling=true&connected=&shazamapiversion=v3&sharehub=true&hubv5minorversion=v5.1&hidelb=true&video=v3')

The above exception was the direct cause of the following exception:

...
  File "/opt/homebrew/lib/python3.11/site-packages/shazamio/api.py", line 487, in recognize_song
    return await self.send_recognize_request(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/shazamio/api.py", line 503, in send_recognize_request
    return await self.http_client.request(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/shazamio/client.py", line 63, in request
    raise e
  File "/opt/homebrew/lib/python3.11/site-packages/shazamio/client.py", line 60, in request
    return await validate_json(resp, *args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/shazamio/utils.py", line 25, in validate_json
    raise FailedDecodeJson("Failed to decode json") from e
shazamio.exceptions.FailedDecodeJson: Failed to decode json

It's strange because I see that in api.py you specify 429 as one of the retried statuses, so I'm not sure why validate_json still gets called on the 429 response.

If my math is good and I made the calculations correctly, then you did not receive a successful answer within 381.1 seconds for 20 attempts you received only a 429 error. Now you will have 20 attempts and 702.3 seconds to complete the request.