magefree / mage

Magic Another Game Engine
http://xmage.today
MIT License
1.85k stars 762 forks source link

Current card image download failures #10833

Closed alexander-novo closed 1 month ago

alexander-novo commented 1 year ago

Some cards are failing to download:

WARN  2023-08-16 23:10:10,122 Image download failed for SLD - Darksteel Colossus, http code: 404, url: https://api.scryfall.com/cards/sld/1081b/en?format=image =>[pool-6-thread-3] DownloadPicturesService$DownloadTask.run 
WARN  2023-08-16 23:10:10,122 Image download failed for SLD - Darksteel Colossus, http code: 404, url: https://api.scryfall.com/cards/sld/1081b/?format=image =>[pool-6-thread-3] DownloadPicturesService$DownloadTask.run 
WARN  2023-08-16 23:10:10,125 Image download failed for SLD - Doubling Cube, http code: 404, url: https://api.scryfall.com/cards/sld/1080b/en?format=image =>[pool-6-thread-2] DownloadPicturesService$DownloadTask.run 
WARN  2023-08-16 23:10:10,125 Image download failed for SLD - Doubling Cube, http code: 404, url: https://api.scryfall.com/cards/sld/1080b/?format=image =>[pool-6-thread-2] DownloadPicturesService$DownloadTask.run 
WARN  2023-08-16 23:10:10,129 Image download failed for SLD - Blightsteel Colossus, http code: 404, url: https://api.scryfall.com/cards/sld/1079b/en?format=image =>[pool-6-thread-1] DownloadPicturesService$DownloadTask.run 
WARN  2023-08-16 23:10:10,129 Image download failed for SLD - Blightsteel Colossus, http code: 404, url: https://api.scryfall.com/cards/sld/1079b/?format=image =>[pool-6-thread-1] DownloadPicturesService$DownloadTask.run 
WARN  2023-08-16 23:10:10,318 Image download failed for SLD - Etali, Primal Storm, http code: 404, url: https://api.scryfall.com/cards/sld/1123b/en?format=image =>[pool-6-thread-2] DownloadPicturesService$DownloadTask.run 
WARN  2023-08-16 23:10:10,318 Image download failed for SLD - Etali, Primal Storm, http code: 404, url: https://api.scryfall.com/cards/sld/1123b/?format=image =>[pool-6-thread-2] DownloadPicturesService$DownloadTask.run 
WARN  2023-08-16 23:10:10,327 Image download failed for SLD - Ghalta, Primal Hunger, http code: 404, url: https://api.scryfall.com/cards/sld/1124b/en?format=image =>[pool-6-thread-1] DownloadPicturesService$DownloadTask.run 
WARN  2023-08-16 23:10:10,327 Image download failed for SLD - Ghalta, Primal Hunger, http code: 404, url: https://api.scryfall.com/cards/sld/1124b/?format=image =>[pool-6-thread-1] DownloadPicturesService$DownloadTask.run 
WARN  2023-08-16 23:10:10,337 Image download failed for SLD - Ulamog, the Ceaseless Hunger, http code: 404, url: https://api.scryfall.com/cards/sld/1122b/en?format=image =>[pool-6-thread-3] DownloadPicturesService$DownloadTask.run 
WARN  2023-08-16 23:10:10,337 Image download failed for SLD - Ulamog, the Ceaseless Hunger, http code: 404, url: https://api.scryfall.com/cards/sld/1122b/?format=image =>[pool-6-thread-3] DownloadPicturesService$DownloadTask.run 
WARN  2023-08-16 23:10:10,531 Image download failed for PL21 - Sethron, Hurloon General, http code: 404, url: https://api.scryfall.com/cards/pl21/1*/en?format=image =>[pool-6-thread-2] DownloadPicturesService$DownloadTask.run 
WARN  2023-08-16 23:10:10,531 Image download failed for PL21 - Sethron, Hurloon General, http code: 404, url: https://api.scryfall.com/cards/pl21/1*/?format=image =>[pool-6-thread-2] DownloadPicturesService$DownloadTask.run 
WARN  2023-08-16 23:10:10,550 Image download failed for CMM - Faithless Looting, http code: 404, url: https://api.scryfall.com/cards/cmm/220/en?format=image =>[pool-6-thread-1] DownloadPicturesService$DownloadTask.run 
WARN  2023-08-16 23:10:10,550 Image download failed for CMM - Faithless Looting, http code: 404, url: https://api.scryfall.com/cards/cmm/220/?format=image =>[pool-6-thread-1] DownloadPicturesService$DownloadTask.run 
WARN  2023-08-16 23:10:10,560 Image download failed for ONE - Lukka, Bound to Ruin, http code: 404, url: https://api.scryfall.com/cards/one/328/en?format=image =>[pool-6-thread-3] DownloadPicturesService$DownloadTask.run 
WARN  2023-08-16 23:10:10,560 Image download failed for ONE - Lukka, Bound to Ruin, http code: 404, url: https://api.scryfall.com/cards/one/328/?format=image =>[pool-6-thread-3] DownloadPicturesService$DownloadTask.run 
WARN  2023-08-16 23:10:10,763 Image download failed for SLD - Okaun, Eye of Chaos, http code: 404, url: https://api.scryfall.com/cards/sld/380b*/en?format=image =>[pool-6-thread-2] DownloadPicturesService$DownloadTask.run 
WARN  2023-08-16 23:10:10,763 Image download failed for SLD - Okaun, Eye of Chaos, http code: 404, url: https://api.scryfall.com/cards/sld/380b*/?format=image =>[pool-6-thread-2] DownloadPicturesService$DownloadTask.run 
WARN  2023-08-16 23:10:10,787 Image download failed for SLD - Zndrsplt, Eye of Wisdom, http code: 404, url: https://api.scryfall.com/cards/sld/379*/en?format=image =>[pool-6-thread-1] DownloadPicturesService$DownloadTask.run 
WARN  2023-08-16 23:10:10,788 Image download failed for SLD - Zndrsplt, Eye of Wisdom, http code: 404, url: https://api.scryfall.com/cards/sld/379*/?format=image =>[pool-6-thread-1] DownloadPicturesService$DownloadTask.run 
WARN  2023-08-16 23:10:10,806 Image download failed for SLD - Zndrsplt, Eye of Wisdom, http code: 404, url: https://api.scryfall.com/cards/sld/379b*/en?format=image =>[pool-6-thread-3] DownloadPicturesService$DownloadTask.run 
WARN  2023-08-16 23:10:10,806 Image download failed for SLD - Zndrsplt, Eye of Wisdom, http code: 404, url: https://api.scryfall.com/cards/sld/379b*/?format=image =>[pool-6-thread-3] DownloadPicturesService$DownloadTask.run 
WARN  2023-08-16 23:10:10,947 Image download failed for SLD - Okaun, Eye of Chaos, http code: 404, url: https://api.scryfall.com/cards/sld/380*/en?format=image =>[pool-6-thread-2] DownloadPicturesService$DownloadTask.run 
WARN  2023-08-16 23:10:10,947 Image download failed for SLD - Okaun, Eye of Chaos, http code: 404, url: https://api.scryfall.com/cards/sld/380*/?format=image =>[pool-6-thread-2] DownloadPicturesService$DownloadTask.run 
  1. Some of these cards, like Zndrsplt and Okaun, are using the wrong character in their collector number - it should be ★ instead of *. See:

https://api.scryfall.com/cards/sld/379*/en?format=image

https://api.scryfall.com/cards/sld/379★/en?format=image

  1. Some of these cards, like Lukka, Bound to Ruin are Phyrexian and can only be found by querying the correct language (not English);

https://api.scryfall.com/cards/one/328/en?format=image

https://api.scryfall.com/cards/one/328/ph?format=image

  1. It's unclear why faithless looting doesn't work. Not requesting image format works fine, but if you ask for an image, it 404s. Maybe an issue on scryfall's side?

https://api.scryfall.com/cards/cmm/220/en

https://api.scryfall.com/cards/cmm/220/en?format=image

  1. Rest are failing because they're double sided cards and the backsides don't have a different collector number. Don't know how the scryfall API works for this.

The * vs ★ character has come up before, and continues to come up as a problem. This should be looked for in a verify test.

alexander-novo commented 1 year ago

Figured out how to access back images in the scryfall api. We need to set the face parameter to back:

https://api.scryfall.com/cards/sld/1081/?face=back&format=image

alexander-novo commented 1 year ago

The language issue looks like it's a more recent change to Scryfall's API. If an English version of a card doesn't exist, then scryfall won't return it even if we don't specify a language. For instance:

https://api.scryfall.com/cards/neo/292/?format=image

works because it's English, but:

https://api.scryfall.com/cards/neo/293/?format=image

doesn't because the card is in Japanese, even though the language isn't specified. This needs to be:

https://api.scryfall.com/cards/neo/293/ja?format=image

JayDi85 commented 7 months ago

Scryfall fixed api and non-english images works again (except phyrexian from #11537).

Example: https://api.scryfall.com/cards/neo/293/?format=image

tiera3 commented 7 months ago

As mentioned in the other thread, if you remove the trailing / the phyrexian cards work Example: https://api.scryfall.com/cards/one/414/?format=image fails https://api.scryfall.com/cards/one/414?format=image succeeds

Baseball SLD have an erroneous b: 1453 Ajani Goldmane , 1454 Jace Beleren , 1455 Liliana Vess , 1456 Chandra Nalaar , Example: client tried to download https://api.scryfall.com/cards/sld/1457b/en?format=image but the b doesn't belong. Needs to access https://api.scryfall.com/cards/sld/1457/en?format=image instead

A bunch of calls for MB1 are failing. The set doesn't exist on scryfall, but it all the cards searched are on The List: Stave Off not https://api.scryfall.com/cards/mb1/246/en?format=image but https://api.scryfall.com/cards/plst/DDN-61/en?format=image Steadfast Sentinel not https://api.scryfall.com/cards/mb1/247/en?format=image but https://api.scryfall.com/cards/plst/HOU-24/en?format=image Steady Progress not https://api.scryfall.com/cards/mb1/504/en?format=image but https://api.scryfall.com/cards/plst/MM2-58/en?format=image Steamflogger Boss not https://api.scryfall.com/cards/mb1/1069/en?format=image but https://api.scryfall.com/cards/plst/UST-93/en?format=image Stinkweed Imp not https://api.scryfall.com/cards/mb1/782/en?format=image but https://api.scryfall.com/cards/plst/GK1-53/en?format=image Stitched Drake not https://api.scryfall.com/cards/mb1/505/en?format=image but https://api.scryfall.com/cards/plst/DDQ-49/en?format=image Stoic Builder not https://api.scryfall.com/cards/mb1/1342/en?format=image but https://api.scryfall.com/cards/plst/SOI-231/en?format=image Stone Haven Medic not https://api.scryfall.com/cards/mb1/248/en?format=image but https://api.scryfall.com/cards/plst/BFZ-51/en?format=image Storm Sculptor not https://api.scryfall.com/cards/mb1/506/en?format=image but https://api.scryfall.com/cards/plst/XLN-85/en?format=image Stormblood Berserker not https://api.scryfall.com/cards/mb1/1070/en?format=image but https://api.scryfall.com/cards/plst/E01-58/en?format=image Stormchaser Chimera not https://api.scryfall.com/cards/mb1/1490/en?format=image but https://api.scryfall.com/cards/plst/CN2-207/en?format=image Strategic Planning not https://api.scryfall.com/cards/mb1/507/en?format=image but https://api.scryfall.com/cards/plst/HOU-47/en?format=image Street Wraith not https://api.scryfall.com/cards/mb1/783/en?format=image but https://api.scryfall.com/cards/plst/A25-108/en?format=image Stream of Thought not https://api.scryfall.com/cards/mb1/508/en?format=image but https://api.scryfall.com/cards/plst/MH1-71/en?format=image Strength in Numbers not https://api.scryfall.com/cards/mb1/1343/en?format=image but https://api.scryfall.com/cards/plst/MM3-138/en?format=image Stunt Double not https://api.scryfall.com/cards/mb1/509/en?format=image but https://api.scryfall.com/cards/plst/CN2-38/en?format=image Stromkirk Patrol not https://api.scryfall.com/cards/mb1/784/en?format=image but https://api.scryfall.com/cards/plst/CN2-149/en?format=image and more

tiera3 commented 7 months ago

more list cards searching for FMB1 (I am guessing that means foil) FMB1 - Zur's Weirding, http code: 404, url: https://api.scryfall.com/cards/fmb1/32/en?format=image needs to be https://api.scryfall.com/cards/plst/8ED-116/en?format=image FMB1 - Yore-Tiller Nephilim, http code: 404, url: https://api.scryfall.com/cards/fmb1/92/?format=image needs to be https://api.scryfall.com/cards/plst/GPT-140/en?format=image

stars need to be encoded PROE - Pestilence Demon, http code: 404, url: https://api.scryfall.com/cards/proe/124*/en?format=image needs to be https://api.scryfall.com/cards/proe/124%E2%98%85?format=image PDKA - Gravecrawler, http code: 404, url: https://api.scryfall.com/cards/pdka/64*/en?format=image needs to be https://api.scryfall.com/cards/pdka/64%E2%98%85/en?format=image PL21 - Sethron, Hurloon General, http code: 404, url: https://api.scryfall.com/cards/pl21/1*/en?format=image needs to be https://api.scryfall.com/cards/pl21/1%E2%98%85/en?format=image PBNG - Fated Conflagration, http code: 404, url: https://api.scryfall.com/cards/pbng/94*/en?format=image needs to be https://api.scryfall.com/cards/pbng/94%E2%98%85/en?format=image SLD - Shattergang Brothers, http code: 404, url: https://api.scryfall.com/cards/sld/1315*/en?format=image needs to be https://api.scryfall.com/cards/sld/1315%E2%98%85/en?format=image PRTR - Supreme Verdict, http code: 404, url: https://api.scryfall.com/cards/prtr/201*/en?format=image needs to be https://api.scryfall.com/cards/prtr/201%E2%98%85/en?format=image

phi needs to be encoded (several more than the following examples) SLD - Virulent Sliver, http code: 404, url: https://api.scryfall.com/cards/sld/659Ph/en?format=image needs to be https://api.scryfall.com/cards/sld/659%CE%A6/en?format=image SLD - Toxin Sliver, http code: 404, url: https://api.scryfall.com/cards/sld/635Ph/en?format=image needs to be https://api.scryfall.com/cards/sld/635%CE%A6/en?format=image

SLD card number is character only SLD - Viscera Seer, http code: 404, url: https://api.scryfall.com/cards/sld/99999VS/en?format=image needs to be https://api.scryfall.com/cards/sld/VS/en?format=image

Token Failures DDD - Beast, http code: 404, url: https://api.scryfall.com/cards/tddd/1/en?format=image needs to be https://api.scryfall.com/cards/tddd/T1/en?format=image and https://api.scryfall.com/cards/tddd/T2/en?format=image DDD - Elephant, http code: 404, url: https://api.scryfall.com/cards/tddd/3/en?format=image needs to be https://api.scryfall.com/cards/tddd/T3/en?format=image

An example of a trailing slash causing a language fail POR - Horned Turtle, http code: 404, url: https://api.scryfall.com/cards/por/57s/?format=image needs to be https://api.scryfall.com/cards/por/57s?format=image

Something messed up with encoding these stars WAR - Karn, the Great Creator, http code: 404, url: https://api.scryfall.com/cards/war/1★/?format=image sorry, can't find a fix for this one WAR - Jiang Yanggu, Wildcrafter, http code: 404, url: https://api.scryfall.com/cards/war/164★/en?format=image same wierdness WAR - Gideon Blackblade, http code: 404, url: https://api.scryfall.com/cards/war/13★?format=image same wierdness

Incorrect set XANA - Shorecomber Crab, http code: 404, url: https://api.scryfall.com/cards/xana/20/en?format=image only found as https://api.scryfall.com/cards/anb/32a/en?format=image

=================================

So other than the star cards from WAR, hopefully the above changes will fix download issues.

tiera3 commented 7 months ago

Current log of download fails. Over the last few days, Ive been downloading all images (from scryfall) to a new directory (for test purposes). And this is the log generated now that it no longer finds any images to download.

imageDLfails.log

tiera3 commented 7 months ago

Checked through Cardviewer.

PLG21 (valid scryfall set) shows no card images, but it didn't come up in the fail warnings. Is it possible that these cards were somehow left off the download list?

JayDi85 commented 7 months ago

ScryfallImageSource contains all supported sets. So PLG21 must be added to it to enable download.

alexander-novo commented 7 months ago

Baseball SLD have an erroneous b: 1453 Ajani Goldmane , 1454 Jace Beleren , 1455 Liliana Vess , 1456 Chandra Nalaar , Example: client tried to download https://api.scryfall.com/cards/sld/1457b/en?format=image but the b doesn't belong. Needs to access https://api.scryfall.com/cards/sld/1457/en?format=image instead

It's trying to download the back side. This is the same issue as I described above.

JayDi85 commented 1 month ago

Outdated