search

mikage-emu / saveShop

Let's save the 3DS and Wii U eShop!
90 stars 2 forks source link

Retrofitting With Archive.org Fetching #18

Closed Lanklow closed 1 year ago

Lanklow commented 1 year ago

Since after the eShop closed, there has been intermittent access to metadata and media (understandably). Though, I noticed that restricted or unavailable URLs have often been archived in the Wayback Machine.

In the main.rs, I tried adding "web.archive.org/web/" to the beginning of each base URL, e.g: "web.archive.org/web/kanzashi-wup.cdn.nintendo.net/"

However, it just spits out "thread 'main' panicked at 'Unrecognized resource URL "kanzashi-wup.cdn.nintendo.net/i/e376763b4374a1239d5769bbaa89e7f1e843faa7b396e58d291ce05509c21811.jpg"', src/main.rs:896:14" -- ignoring pointing towards archive.org.

I have a few partially complete archives which I would like to finish, so I'd appreciate some help solving my "issue".

KLanausse commented 1 year ago

That's because you're missing the timestamp in the URL. instead of web.archive.org/web/ change it to web.archive.org/web/*/ and then it should work. Just note that archive.org wont have anything from https://ninja.ctr.shop.nintendo.net unless someone uploaded their warc since it requires you connect with your 3DS client certificate to access

Lanklow commented 1 year ago

Just tried running with your suggested changes, and it spits out the same error

KLanausse commented 1 year ago

Try this URL https://web.archive.org/web/*if_/https://samurai.ctr.shop.nintendo.net/samurai/ws/

I realize I forgot to include the *if_ so it fetches the page without the extra parts like the navigator that archive.org adds to the top of the page.

Lanklow commented 1 year ago

That worked for me. Thanks!

Edit: Though, to be clear (for anyone referring to this in the future), it's only working with that single field change. Changing any other URL results in an error.