json parse error when calling `artist` method

[aome510]

Describe the bug Client now returns

Bad request: json parse error: invalid type: floating point `49022656`, expected u32 at line 1 column 130: invalid type: floating point `49022656`, expected u32 at line 1 column 130

for every artist request. The reason seems to be because Spotify changes the type of followers.total and other fields to float.

To Reproduce Steps to reproduce the behavior:

1. Run

   curl --request GET \
       --url https://api.spotify.com/v1/artists/1dfeR4HaWDbWqFHLkxsg1d \
       --header 'Authorization: Bearer $TOKEN'

2. Observe that follower.total is not an integer. Example result:

   {"external_urls":{"spotify":"https://open.spotify.com/artist/1dfeR4HaWDbWqFHLkxsg1d"},"followers":{"href":null,"total":4.9022656E7},"genres":["classic rock","glam rock","rock"],"href":"https://api.spotify.com/v1/artists/1dfeR4HaWDbWqFHLkxsg1d","id":"1dfeR4HaWDbWqFHLkxsg1d","images":[{"url":"https://i.scdn.co/image/b040846ceba13c3e9c125d68389491094e7f2982","height":806.0,"width":999.0},{"url":"https://i.scdn.co/image/af2b8e57f6d7b5d43a616bd1e27ba552cd8bfd42","height":516.0,"width":640.0},{"url":"https://i.scdn.co/image/c06971e9ff81696699b829484e3be165f4e64368","height":161.0,"width":200.0},{"url":"https://i.scdn.co/image/6dd0ffd270903d1884edf9058c49f58b03db893d","height":52.0,"width":64.0}],"name":"Queen","popularity":84.0,"type":"artist","uri":"spotify:artist:1dfeR4HaWDbWqFHLkxsg1d"}

3. Notice that some fields' type are changed to float such as followers.total, popularity

[ramsayleung]

It's weird that the total and popularity are marked as integer in their official document, but actually return float data.

I tried to reproduce this bug by following your instruction in the document console, but the response is different from yours.

curl --request GET \
  --url https://api.spotify.com/v1/artists/1dfeR4HaWDbWqFHLkxsg1d \
  --header 'Authorization: Bearer $TOKEN'

{
  "external_urls": {
    "spotify": "https://open.spotify.com/artist/1dfeR4HaWDbWqFHLkxsg1d"
  },
  "followers": {
    "href": null,
    "total": 49022656
  },
  "genres": ["classic rock", "glam rock", "rock"],
  "href": "https://api.spotify.com/v1/artists/1dfeR4HaWDbWqFHLkxsg1d?locale=en-US%2Cen%3Bq%3D0.9%2Czh-CN%3Bq%3D0.8%2Czh%3Bq%3D0.7",
  "id": "1dfeR4HaWDbWqFHLkxsg1d",
  "images": [
    {
      "url": "https://i.scdn.co/image/b040846ceba13c3e9c125d68389491094e7f2982",
      "height": 806,
      "width": 999
    },
    {
      "url": "https://i.scdn.co/image/af2b8e57f6d7b5d43a616bd1e27ba552cd8bfd42",
      "height": 516,
      "width": 640
    },
    {
      "url": "https://i.scdn.co/image/c06971e9ff81696699b829484e3be165f4e64368",
      "height": 161,
      "width": 200
    },
    {
      "url": "https://i.scdn.co/image/6dd0ffd270903d1884edf9058c49f58b03db893d",
      "height": 52,
      "width": 64
    }
  ],
  "name": "Queen",
  "popularity": 84,
  "type": "artist",
  "uri": "spotify:artist:1dfeR4HaWDbWqFHLkxsg1d"
}

followers.total, popularity are both integer rather than float.

[KaydaFox]

im getting this same error too, think spotify may have changed something because this was all fine yesterday response from curl:

{
    "external_urls": {
        "spotify": "https://open.spotify.com/artist/4bwIf0yXJf0F9AmOl2J78M"
    },
    "followers": {
        "href": null,
        "total": 194172.0
    },
    "genres": [
        "denpa-kei",
        "rhythm game"
    ],
    "href": "https://api.spotify.com/v1/artists/4bwIf0yXJf0F9AmOl2J78M",
    "id": "4bwIf0yXJf0F9AmOl2J78M",
    "images": [
        {
            "url": "https://i.scdn.co/image/ab6761610000e5ebc1f9dd30f82038e91e865d82",
            "height": 640.0,
            "width": 640.0
        },
        {
            "url": "https://i.scdn.co/image/ab67616100005174c1f9dd30f82038e91e865d82",
            "height": 320.0,
            "width": 320.0
        },
        {
            "url": "https://i.scdn.co/image/ab6761610000f178c1f9dd30f82038e91e865d82",
            "height": 160.0,
            "width": 160.0
        }
    ],
    "name": "Camellia",
    "popularity": 51.0,
    "type": "artist",
    "uri": "spotify:artist:4bwIf0yXJf0F9AmOl2J78M"
}```

[ramsayleung]

Yes, I can reproduce your problem now, I've submitted a ticket to Spotify to ask for clarity: https://community.spotify.com/t5/Spotify-for-Developers/The-response-type-of-artist-endpoint-is-marked-as-Integer-in/m-p/5800044

[gansheer]

Hi @ramsayleung, just so you know there is an error in the issue link https://github.com/ramsayleung/rspotify/issues/452m (trailing m) in your Spotify ticket.

[ramsayleung]

Thanks for your heads-up, I've revised the link to the correct one. But it's odd that a developer replied in the ticket saying what he got with curl was integer data instead of float data.

Unfortunately, this ticket is marked as spam...

I've submitted an abuse report.

[PMARZV]

Hey here! Developing an app for android and i have the same problem with the GETARTISTS endpoint

[aome510]

I don't think Spotify will give their response anytime soon. I wonder if we have any plans to mitigate this issue @ramsayleung ?

[ramsayleung]

One of the workarounds comes to my mind is converting the Integer type to float type to bypass the issue. But in my perspective, I don't think it's a great idea because

1. The SDK shouldn't silently swallow the bug caused by the Spotify server side, a better solution will be prompting the issue to Spotify and push them to fix the problem
2. It will introduce a breaking change in this library, which's definitely unnecessary and avoidable.

I personally prefer to keep escalating and making "noise" to the Spotify developers, the louder noise we could make, the higher probability the Spotify team could be aware of this problem and fix it.

[ramsayleung]

I've sent a message to escalate this issue to the Spotify team via this link: https://support.spotify.com/ca-en/contact-spotify-support/, here is the response from their associate:

I already forward this issue to the team backstage. Also, the tech folks backstage confirm that they're already ahead of this and currently working for a fix. No need to worry now :)

Also, our Tech folks are known for their hard work so this will be fixed in no time! Consider this issue resolved.

[aome510]

I've sent a message to escalate this issue to the Spotify team via this link: https://support.spotify.com/ca-en/contact-spotify-support/, here is the response from their associate:

I already forward this issue to the team backstage. Also, the tech folks backstage confirm that they're already ahead of this and currently working for a fix. No need to worry now :)

Also, our Tech folks are known for their hard work so this will be fixed in no time! Consider this issue resolved.

Great! Thank you @ramsayleung

[grantshandy]

This issue is also holding up the development of a project of mine. If it isn't fixed soon by the spotify developers would rspotify support deserializing all u32 fields with a custom Deserializer a la #[serde(deserialize_with = "as_u32")]? as_u32 would automatically convert float fields into int ones and keep the same default behavior as before. It's pretty hacky, but it would fix the issue for now/if it arises again (im feeling pretty pessimistic lol). It'd be a less permissive version of serde-this-or-that's as_u64.

This solution wouldn't introduce breaking changes in the API as @ramsayleung pointed out, and the fix would continue to work if spotify does get around to fixing the issue. Plus, it would only have the overhead of converting any given f64 to a u32. If there's any interest in this sort of fix, let me know and I can open up a PR pretty quick.

[KaydaFox]

This issue is also holding up the development of a project of mine. If it isn't fixed soon by the spotify developers would rspotify support deserializing all u32 fields with a custom Deserializer a la #[serde(deserialize_with = "as_u32")]? as_u32 would automatically convert float fields into int ones and keep the same default behavior as before. It's pretty hacky, but it would fix the issue for now/if it arises again (im feeling pretty pessimistic lol). It'd be a less permissive version of serde-this-or-that's as_u64.

This solution wouldn't introduce breaking changes in the API as @ramsayleung pointed out, and the fix would continue to work if spotify does get around to fixing the issue. Plus, it would only have the overhead of converting any given f64 to a u32. If there's any interest in this sort of fix, let me know and I can open up a PR pretty quick.

One thing you could do as a temporary workaround is this

        match rspotify
            .api_get(&format!("artists/{}", artist_id), &Query::new())
            .await
        {
            Ok(artist) => {
                // Manually parse the json for now
                let artist_json: Result<Value, _> = serde_json::from_str(&artist);
                match artist_json {
                }
        }

    im not sure if its the bestt way but its what im using for now and it seems to work alright, still learning stuff lol
    but it stopped me from needing to halt development on my stuff temporarily

[ramsayleung]

One thing you could do as a temporary workaround is this

But this temporary workaround only for the you, the sole developer who decides to engage this approach.

I would try to implement the as_u32 approach without imposing breaking change as a short term solution, as we actually don't know when will Spotify fix this issue.

[grantshandy]

Should I contribute the fix? I can do it in the next two days unless you'd like to implement it yourself.

[ramsayleung]

I could provide the code snippet for your reference, feel free to contribute and issue a PR :)

pub mod data_type_patcher {
    use serde::{Deserialize, Deserializer};
    pub fn as_u32<'de, D>(deserializer: D) -> Result<u32, D::Error>
where
    D: Deserializer<'de>,
{
    let float_data: f64 = Deserialize::deserialize(deserializer)?;

    let u32_data = float_data as u32;

    Ok(u32_data)
}
}

[m-torhan]

As I didn't see any PR related to this I let myself to implement it https://github.com/ramsayleung/rspotify/pull/457.

I tested it with https://github.com/aome510/spotify-player and it worked well.

[aome510]

Look like a workaround patch landed to master. @ramsayleung do you have any plans to release a new version?

[ramsayleung]

I am on vacation right now, I will release a new version when I am back :)

On Sun, Feb 4, 2024 at 1:34 AM Thang Pham @.***> wrote:

Look like a workaround patch landed to master. @ramsayleung https://github.com/ramsayleung do you have any plans to release a new version?

— Reply to this email directly, view it on GitHub https://github.com/ramsayleung/rspotify/issues/452#issuecomment-1925408250, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADMGCSMC32AKD5FDO26MFZ3YRZYLXAVCNFSM6AAAAABBPP63O2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMRVGQYDQMRVGA . You are receiving this because you were mentioned.Message ID: @.***>

[aome510]

I'm now getting the same error but with get-several-tracks API call. To reproduce, run

curl --request GET \
         --url "https://api.spotify.com/v1/tracks?ids=5O2P9iiztwhomNh8xkR9lJ" \
         --header 'Authorization: Bearer $TOKEN'

Observe that many fields expecting integer return float. The response that I got

{"tracks":[{"album":{"album_type":"album","artists":[{"external_urls":{"spotify":"https://open.spotify.com/artist/4AK6F7OLvEQ5QYCBNiQWHq"},"href":"https://api.spotify.com/v1/artists/4AK6F7OLvEQ5QYCBNiQWHq","id":"4AK6F7OLvEQ5QYCBNiQWHq","name":"One Direction","type":"artist","uri":"spotify:artist:4AK6F7OLvEQ5QYCBNiQWHq"}],"available_markets":["AR","AU","AT","BE","BO","BR","BG","CA","CL","CO","CR","CY","CZ","DK","DO","DE","EC","EE","SV","FI","FR","GR","GT","HN","HK","HU","IS","IE","IT","LV","LT","LU","MY","MT","MX","NL","NZ","NI","NO","PA","PY","PE","PH","PL","PT","SG","SK","ES","SE","CH","TW","TR","UY","US","GB","AD","LI","MC","ID","JP","TH","VN","RO","IL","ZA","SA","AE","BH","QA","OM","KW","EG","MA","DZ","TN","LB","JO","PS","IN","BY","KZ","MD","UA","AL","BA","HR","ME","MK","RS","SI","KR","BD","PK","LK","GH","KE","NG","TZ","UG","AG","AM","BS","BB","BZ","BT","BW","BF","CV","CW","DM","FJ","GM","GE","GD","GW","GY","HT","JM","KI","LS","LR","MW","MV","ML","MH","FM","NA","NR","NE","PW","PG","WS","SM","ST","SN","SC","SL","SB","KN","LC","VC","SR","TL","TO","TT","TV","VU","AZ","BN","BI","KH","CM","TD","KM","GQ","SZ","GA","GN","KG","LA","MO","MR","MN","NP","RW","TG","UZ","ZW","BJ","MG","MU","MZ","AO","CI","DJ","ZM","CD","CG","IQ","LY","TJ","VE","ET","XK"],"external_urls":{"spotify":"https://open.spotify.com/album/4gCNyS7pidfK3rKWhB3JOY"},"href":"https://api.spotify.com/v1/albums/4gCNyS7pidfK3rKWhB3JOY","id":"4gCNyS7pidfK3rKWhB3JOY","images":[{"url":"https://i.scdn.co/image/ab67616d0000b273d304ba2d71de306812eebaf4","width":640.0,"height":640.0},{"url":"https://i.scdn.co/image/ab67616d00001e02d304ba2d71de306812eebaf4","width":300.0,"height":300.0},{"url":"https://i.scdn.co/image/ab67616d00004851d304ba2d71de306812eebaf4","width":64.0,"height":64.0}],"is_playable":true,"name":"FOUR (Deluxe)","release_date":"2014-11-17","release_date_precision":"day","total_tracks":16.0,"type":"album","uri":"spotify:album:4gCNyS7pidfK3rKWhB3JOY"},"artists":[{"external_urls":{"spotify":"https://open.spotify.com/artist/4AK6F7OLvEQ5QYCBNiQWHq"},"href":"https://api.spotify.com/v1/artists/4AK6F7OLvEQ5QYCBNiQWHq","id":"4AK6F7OLvEQ5QYCBNiQWHq","name":"One Direction","type":"artist","uri":"spotify:artist:4AK6F7OLvEQ5QYCBNiQWHq"}],"available_markets":["AR","AU","AT","BE","BO","BR","BG","CA","CL","CO","CR","CY","CZ","DK","DO","DE","EC","EE","SV","FI","FR","GR","GT","HN","HK","HU","IS","IE","IT","LV","LT","LU","MY","MT","MX","NL","NZ","NI","NO","PA","PY","PE","PH","PL","PT","SG","SK","ES","SE","CH","TW","TR","UY","US","GB","AD","LI","MC","ID","JP","TH","VN","RO","IL","ZA","SA","AE","BH","QA","OM","KW","EG","MA","DZ","TN","LB","JO","PS","IN","BY","KZ","MD","UA","AL","BA","HR","ME","MK","RS","SI","KR","BD","PK","LK","GH","KE","NG","TZ","UG","AG","AM","BS","BB","BZ","BT","BW","BF","CV","CW","DM","FJ","GM","GE","GD","GW","GY","HT","JM","KI","LS","LR","MW","MV","ML","MH","FM","NA","NR","NE","PW","PG","WS","SM","ST","SN","SC","SL","SB","KN","LC","VC","SR","TL","TO","TT","TV","VU","AZ","BN","BI","KH","CM","TD","KM","GQ","SZ","GA","GN","KG","LA","MO","MR","MN","NP","RW","TG","UZ","ZW","BJ","MG","MU","MZ","AO","CI","DJ","ZM","CD","CG","IQ","LY","TJ","VE","ET","XK"],"disc_number":1.0,"duration_ms":226600.0,"explicit":false,"external_ids":{"isrc":"GBHMU1400165"},"external_urls":{"spotify":"https://open.spotify.com/track/5O2P9iiztwhomNh8xkR9lJ"},"href":"https://api.spotify.com/v1/tracks/5O2P9iiztwhomNh8xkR9lJ","id":"5O2P9iiztwhomNh8xkR9lJ","is_local":false,"is_playable":true,"name":"Night Changes","popularity":89.0,"preview_url":"https://p.scdn.co/mp3-preview/359be833b46b250c696bbb64caa5dc91f2a38c6a?cid=b28a574d8ad44fc7b5a89994431a8f88","track_number":7.0,"type":"track","uri":"spotify:track:5O2P9iiztwhomNh8xkR9lJ"}]}

Not sure if the above issue is only reproducible on my end because the same request on https://developer.spotify.com/documentation/web-api/reference/get-several-tracks still returns the correct type.

Found this issue when trying to investigate https://github.com/aome510/spotify-player/issues/383

[ramsayleung]

I have to say, it's "scope creep" of Spotify's bug, other developers have reported the same observation in other endpoints, it becomes less possible to be patched and bypassed individually by the community-driven library, this problem has to be fixed by the Spotify itself.

[aome510]

I have to say, it's "scope creep" of Spotify's bug, other developers have reported the same observation in other endpoints, it becomes less possible to be patched and bypassed individually by the community-driven library, this problem has to be fixed by the Spotify itself.

Yeah, I agree. At the same time, it's kinda frustrating to see that Spotify doesn't seem to take any actions.

[ramsayleung]

The new version of RSpotify would like to pend for a while for the response from Spotify, since it's meanless to release a new version while Spotify keeps breaking their endpoints, the community is unable to catch up to release new patch.

[ed9w2in6]

@aome510 @ramsayleung Seems to be fixed ^_^

As expected this is caused by some library that they are using.

I am using spotify_player today and found that at the the artist endpoint works now.

[ramsayleung]

Closing since this issue has been resolved by Spotify.