better free album check

yoshiyoshyosh commented 7 months ago

I believe that free download albums where the fallback price is > 0 causes bcdl-free to think that the album isn't free, when it really is

case study: https://dj-bamberino.bandcamp.com/album/bittersweet-saffron relevant data-tralbum:

  "current": {
    "audit": 0,
    "title": "Bittersweet Saffron",
    "new_date": "05 Mar 2021 21:44:16 GMT",
    "mod_date": "06 Mar 2021 00:11:13 GMT",
    "publish_date": "05 Mar 2021 22:03:53 GMT",
    "private": null,
    "killed": null,
    "download_pref": 1,
    "require_email": null,
    "is_set_price": null,
    "set_price": 7.0,
    "minimum_price": 7.0,
    "minimum_price_nonzero": 7.0,
    "require_email_0": null,
    "artist": null,
    "about": null,
    "credits": null,
    "auto_repriced": null,
    "new_desc_format": 1,
    "band_id": 71063555,
    "selling_band_id": 71063555,
    "art_id": 3468198509,
    "download_desc_id": null,
    "release_date": "05 Mar 2021 00:00:00 GMT",
    "upc": null,
    "purchase_url": null,
    "purchase_title": null,
    "featured_track_id": 711289386,
    "id": 278671220,
    "type": "album"
  },
  "preorder_count": null,
  "hasAudio": true,
  "art_id": 3468198509,
  "packages": null,
  "defaultPrice": 7.0,
  "freeDownloadPage": "https://bandcamp.com/download?id=278671220&ts=1712882855.1590790080&tsig=0ce97483ff467d90ffc9acdd70bd5141&type=album",
  "FREE": 1,
  "PAID": 2,
  "artist": "Dj Bamberino",
  "item_type": "album",
  "id": 278671220,
  "last_subscription_item": null,
  "has_discounts": false,
  "is_bonus": null,
  "play_cap_data": null,
  "client_id_sig": null,
  "is_purchased": null,
  "items_purchased": null,
  "is_private_stream": null,
  "is_band_member": null,
  "licensed_version_ids": null,
  "package_associated_license_id": null,
  "has_video": null,
  "tralbum_subscriber_only": false,
  "featured_track_id": 711289386,
  "initial_track_num": null,
  "is_preorder": false,
  "album_is_preorder": false,
  "album_release_date": "05 Mar 2021 00:00:00 GMT",

proposed fix: check if freeDownloadPage is not null in the same check as tralbum_data["current"]["minimum_price"] == 0 tested on the above album & some others, everything seems good

yoshiyoshyosh commented 6 months ago

that's annoying. I thought I covered everything, but check out this album: https://pekano.bandcamp.com/album/jungle-stab "freeDownloadPage": "https://bandcamp.com/download?id=1608026956&ts=1712937230.237207704&tsig=9478d89ad962cc6db5ab43059ddfeab8&type=album", without any tracks, which raises a KeyError when it goes to download a format that doesn't exist

alternative idea: there's a second JSON thingy that contains album metadata in the <head>, which can be isolated by searching only for head > script[type="application/ld+json"]. here's an example of me getting album metadata from it: https://git.unix.dog/yosh/misc-scripts/src/commit/4e839b2406b8397b53aa3f6c79651f1ff70c35bd/discogarchive#L184

or I guess we can move a strict track-check within the if statement

thoughts?

7x11x13 commented 6 months ago

that's annoying. I thought I covered everything, but check out this album: https://pekano.bandcamp.com/album/jungle-stab "freeDownloadPage": "https://bandcamp.com/download?id=1608026956&ts=1712937230.237207704&tsig=9478d89ad962cc6db5ab43059ddfeab8&type=album", without any tracks, which raises a KeyError when it goes to download a format that doesn't exist

I think that is a separate issue which also exists in the current version

7x11x13 commented 6 months ago

alternative idea: there's a second JSON thingy that contains album metadata in the <head>, which can be isolated by searching only for head > script[type="application/ld+json"]. here's an example of me getting album metadata from it: https://git.unix.dog/yosh/misc-scripts/src/commit/4e839b2406b8397b53aa3f6c79651f1ff70c35bd/discogarchive#L184

or I guess we can move a strict track-check within the if statement

thoughts?

Anything involving counting the number of tracks I would worry about how it handles albums with "hidden tracks" (i.e. would it download an album which has no publicly visible tracks?)

7x11x13 commented 6 months ago

I think the "hasAudio" key in tralbum_data might work, can you test that out?

yoshiyoshyosh commented 6 months ago

Anything involving counting the number of tracks I would worry about how it handles albums with "hidden tracks" (i.e. would it download an album which has no publicly visible tracks?)

do you have an example of this on hand?

7x11x13 commented 6 months ago

https://7x11x13.bandcamp.com/album/test-album-for-free-bandcamp-downloader

yoshiyoshyosh commented 6 months ago

I thiiiink that should cover it--downloads that test album, the album I raised this for, and doesn't raise an error on jungle stab.

yoshiyoshyosh commented 6 months ago

Actually, I think it'd be better if we put the audio check outside of the main check. Makes more sense for it to be there.

yoshiyoshyosh commented 6 months ago

I'll be damned. It's possible to have a free download album that requires email, which evades these checks: https://treecritters.bandcamp.com/album/real-trap-shit

I believe the only way to get around this is using the metadata from head > script[type="application/ld+json": https://x0.at/DPaz.txt

yoshiyoshyosh commented 6 months ago

alright, here's what I've got so far:

first, check if the album hasAudio. if not, then don't download, of course
then we get the data from the head > script[type="application/ld+json"]
then we find the item in the albumRelease list where it has an additionalProperty with an item_type of a or t. these correspond to digital album or digital track releases, respectively. physical releases use p and discography offerings use b
then we check if the album even has the offers available for this digital release, because there do exist albums on bandcamp that don't offer a digital format
if all that is good, then we know that the album has audio and it offers a digital download, so we can check the price
- given my last comment here, I take the price from the head_data rather than the data-tralbum. from my testing over months with my discography archive script, this has always been a correct reflection of the current album price for me

albums I tested this on that work as expected:

https://treecritters.bandcamp.com/album/real-trap-shit (free)
https://7x11x13.bandcamp.com/album/test-album-for-free-bandcamp-downloader (free, hidden tracks)
https://pekano.bandcamp.com/track/eeyup (track, free)
https://nanode.bandcamp.com/track/paralysis (track, not free)
https://nanode.bandcamp.com/album/journey (album, not free)
https://murderchannel.bandcamp.com/album/10-years-of-breaking-things (album, contains releases but no digital offering)

this should cover everything now... the only weird edge case I think might exist is offers containing multiple objects, but I have never seen that ever, nor would I think bandcamp would have an inconsistent list-if-multiple object-if-single object in their data... but you never know

7x11x13 / free-bandcamp-downloader

better free album check #7