search

xwingtmg / xwing-data2

An easy-to-use collection of data and images from X-Wing: The Miniatures Game (Second Edition) by Fantasy Flight Games.
MIT License
98 stars 67 forks source link

Convert FFG squadbuilder image links to InfiniteArenas #521

Open danrs opened 2 years ago

danrs commented 2 years ago

FFG squad builder is defunct and cout go offline at any time. Switch all FFG squad builder image links to InfiniteArenas (hosted by @J1mBob).

Before this can be done, we need a more accessible way to view image links on InfiniteArenas. The current design is great for ordinary users, but clicking through images and dismissing the "link copied to clipboard" popup is very repetitive when doing data entry. @J1mBob could we have a page on InfiniteArenas that lists all the links, or a page with the links for each expansion?

If all the links were on a single page (or even available as json via an API query) that would make it a lot easier to scrape. In fact, if all the infinite arenas images used the xws id as the image name, that would be a very simple task to scrape them all into XWS (but I realise that would force you to update image links on old images, which could break something)

J1mBob commented 2 years ago

I would prefer to make the image names fully compliant with xws naming convention. To me that makes the most sense. I can clean up broken links on my site.

Here are the locations for all the different images currently:

Pilot card images: https://infinitearenas.com/xw2/images/pilots/{xws}.png

Pilot card art: https://infinitearenas.com/xw2/images/artwork/pilots/{xws}.png

Upgrade card images: https://infinitearenas.com/xw2/images/upgrades/{xws}.png

Upgrade card art: https://infinitearenas.com/xw2/images/artwork/upgrades/{xws}.png

Condition card images: https://infinitearenas.com/xw2/images/conditions/{xws}.png

danrs commented 2 years ago

I won't argue with that, it makes life a lot easier for me! Then I can just write a scraper script to update all the cards in one go.

I'll keep this issue open until you update your image links. No rush on that from my end.

If I can make one additional request: it would be great if each of the folders showed a list of all the file names in that folder (eg https://infinitearenas.com/xw2/images/upgrades/ shows all the upgrade file names) instead of 403 error. That would make it a lot faster to figure out bad link names and see what links are present/missing.

J1mBob commented 2 years ago

Sounds good. I’ll work on creating that file.

On Mar 2, 2022, at 7:27 PM, Dan Smith @.***> wrote:

 I won't argue with that, it makes life a lot easier for me! Then I can just write a scraper script to update all the cards in one go.

I'll keep this issue open until you update your image links. No rush on that from my end.

If I can make one additional request: it would be great if each of the folders showed a list of all the file names in that folder (eg https://infinitearenas.com/xw2/images/upgrades/ shows all the upgrade file names) instead of 403 error. That would make it a lot faster to figure out bad link names and see what links are present/missing.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.

danrs commented 2 years ago

It might not need to be a file: if there's no security risk you might be able to just open the directory to read access from the web

J1mBob commented 2 years ago

Good point, I’ll make that switch.

On Mar 2, 2022, at 7:34 PM, Dan Smith @.***> wrote:

 It might not need to be a file: if there's no security risk you might be able to just open the directory to read access from the web

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.

J1mBob commented 2 years ago

I've opened the above directories to read access from the web.

J1mBob commented 2 years ago

And for good measure I also opened this one: https://infinitearenas.com/xw2/images/remotes/ Future request would be to add remotes to the data structure.

danrs commented 2 years ago

Lovely, that works great

On Fri, 4 Mar 2022 at 9:58 pm, J1mBob @.***> wrote:

And for good measure I also opened this one: https://infinitearenas.com/xw2/images/remotes/ Future request would be to add remotes to the data structure.

— Reply to this email directly, view it on GitHub https://github.com/guidokessels/xwing-data2/issues/521#issuecomment-1059185023, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABLOVLOSEXBLCG4HHU5EZCTU6IJITANCNFSM5PY7PAEA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you authored the thread.Message ID: @.***>

J1mBob commented 2 years ago

I had another thought. I really don't mind hosting the images but I'm wondering if it's a little more "open source" friendly if we pointed to CrazyVulcan's GitHub repo of the images: https://raw.githubusercontent.com/CrazyVulcan/x-wing2.0-project-goldenrod/main/src/images/En/pilots/066.png

danrs commented 2 years ago

Off the top of my head, yes that probably is more friendly. Does he host all the ffg images on there as well?

On Thu, 10 Mar 2022 at 11:14 pm, J1mBob @.***> wrote:

I had another thought. I really don't mind hosting the images but I'm wondering if it's a little more "open source" friendly if we pointed to CrazyVulcan's GitHub repo of the images:

https://raw.githubusercontent.com/CrazyVulcan/x-wing2.0-project-goldenrod/main/src/images/En/pilots/066.png

— Reply to this email directly, view it on GitHub https://github.com/guidokessels/xwing-data2/issues/521#issuecomment-1064169054, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABLOVLNLYFQD25HNEB4IAJDU7IGVFANCNFSM5PY7PAEA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you authored the thread.Message ID: @.***>

J1mBob commented 2 years ago

He scraped them before the squad builder went down. I could contribute any images I have that are missing to his repo.

Just in case I get hit by a bus.

On Mar 10, 2022, at 3:58 PM, Dan Smith @.***> wrote:

 Off the top of my head, yes that probably is more friendly. Does he host all the ffg images on there as well?

On Thu, 10 Mar 2022 at 11:14 pm, J1mBob @.***> wrote:

I had another thought. I really don't mind hosting the images but I'm wondering if it's a little more "open source" friendly if we pointed to CrazyVulcan's GitHub repo of the images:

https://raw.githubusercontent.com/CrazyVulcan/x-wing2.0-project-goldenrod/main/src/images/En/pilots/066.png

— Reply to this email directly, view it on GitHub https://github.com/guidokessels/xwing-data2/issues/521#issuecomment-1064169054, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABLOVLNLYFQD25HNEB4IAJDU7IGVFANCNFSM5PY7PAEA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you authored the thread.Message ID: @.***>

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.

danrs commented 2 years ago

Only thing is it would still be good if his images are referenced by xws, and I suspect they are not

On Fri, 11 Mar 2022 at 7:14 am, J1mBob @.***> wrote:

He scraped them before the squad builder went down. I could contribute any images I have that are missing to his repo.

Just in case I get hit by a bus.

On Mar 10, 2022, at 3:58 PM, Dan Smith @.***> wrote:

 Off the top of my head, yes that probably is more friendly. Does he host all the ffg images on there as well?

On Thu, 10 Mar 2022 at 11:14 pm, J1mBob @.***> wrote:

I had another thought. I really don't mind hosting the images but I'm wondering if it's a little more "open source" friendly if we pointed to CrazyVulcan's GitHub repo of the images:

https://raw.githubusercontent.com/CrazyVulcan/x-wing2.0-project-goldenrod/main/src/images/En/pilots/066.png

— Reply to this email directly, view it on GitHub < https://github.com/guidokessels/xwing-data2/issues/521#issuecomment-1064169054 , or unsubscribe < https://github.com/notifications/unsubscribe-auth/ABLOVLNLYFQD25HNEB4IAJDU7IGVFANCNFSM5PY7PAEA

. Triage notifications on the go with GitHub Mobile for iOS < https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675

or Android < https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub .

You are receiving this because you authored the thread.Message ID: @.***>

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.

— Reply to this email directly, view it on GitHub https://github.com/guidokessels/xwing-data2/issues/521#issuecomment-1064604841, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABLOVLNMSH7ZTRQH3K3BUTDU7J64DANCNFSM5PY7PAEA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you authored the thread.Message ID: @.***>

J1mBob commented 2 years ago

I think they mostly are, but likely with some errors. And omissions on recent releases.

So I don’t think we change yet, but state it as a goal.

On Mar 10, 2022, at 4:33 PM, Dan Smith @.***> wrote:

 Only thing is it would still be good if his images are referenced by xws, and I suspect they are not

On Fri, 11 Mar 2022 at 7:14 am, J1mBob @.***> wrote:

He scraped them before the squad builder went down. I could contribute any images I have that are missing to his repo.

Just in case I get hit by a bus.

On Mar 10, 2022, at 3:58 PM, Dan Smith @.***> wrote:

 Off the top of my head, yes that probably is more friendly. Does he host all the ffg images on there as well?

On Thu, 10 Mar 2022 at 11:14 pm, J1mBob @.***> wrote:

I had another thought. I really don't mind hosting the images but I'm wondering if it's a little more "open source" friendly if we pointed to CrazyVulcan's GitHub repo of the images:

https://raw.githubusercontent.com/CrazyVulcan/x-wing2.0-project-goldenrod/main/src/images/En/pilots/066.png

— Reply to this email directly, view it on GitHub < https://github.com/guidokessels/xwing-data2/issues/521#issuecomment-1064169054 , or unsubscribe < https://github.com/notifications/unsubscribe-auth/ABLOVLNLYFQD25HNEB4IAJDU7IGVFANCNFSM5PY7PAEA

. Triage notifications on the go with GitHub Mobile for iOS < https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675

or Android < https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub .

You are receiving this because you authored the thread.Message ID: @.***>

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.

— Reply to this email directly, view it on GitHub https://github.com/guidokessels/xwing-data2/issues/521#issuecomment-1064604841, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABLOVLNMSH7ZTRQH3K3BUTDU7J64DANCNFSM5PY7PAEA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you authored the thread.Message ID: @.***>

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.

chrisalleng commented 2 years ago

I think they mostly are, but likely with some errors. And omissions on recent releases. So I don’t think we change yet, but state it as a goal.

Slight followup on the above, many/most of the new release images are missing from/are not referenced by xws. Would compiling a list of the missing images be useful/helpful? I don't want to create/add references to the images hosting in infinitearenas into an xws pull request or anything without permission, but if we have permission/that's the desired end state, I could save a step and just do that as well!

EDIT: Just realized who I was responding to, I am an idiot 😉. Looking through current images in the xws spec, it's mostly the new stuff that's still pointing at infinitearenas. I'm in the process of using XWS as a library, part of which uses grabbing/embedding the images referred to in the XWS spec. What's the "correct" solution to the missing or out of date references? For these missing images for pilot art and card art, would it be better to open a pull request adding them to the above linked project from CrazyVulcan, or to update references in XWS to point towards infinite arenas?

danrs commented 2 years ago

I don’t think there is a “correct” image location for xws to point at, but I feel like this thread was vibing towards CrazyVulcan’s repo as the “nicest” solution, so let’s go with that unless anyone objects.

Any help on updating xws is good help so you can go right ahead with PRs in my opinion

AlexRaubach commented 1 year ago

FFG recently turned off the servers hosting the images from their squadbuilder. I went ahead and changed all the pilot image and artwork links to InfiniteArenas in #607

Still todo: all upgrade image links (I believe Chris Allen was looking into doing this) and the icon images for each ship and upgrade type. There also a handful of alt art image links that are still have ffg urls, example.

chrisalleng commented 1 year ago

Yeah, apologies, life™ came up the past few days, but tackling upgrades is on my TODO list for tomorrow. I'm surprised to see alt art cards exist in XWS. I know there must be a huge set of them missing. Is there any reason/motivation to maintain those?

chrisalleng commented 1 year ago

Missing upgrades and other misc issues from infinite arenas in case it is useful for @J1mBob:

Artwork

Images

Other Issues

danrs commented 1 year ago

610 fixes a good number of images, but I believe some remain.

chrisalleng commented 1 year ago

@danrs all references in xws have been moved over to infinite arenas, but some of those don't exist in infinite arenas and are currently dead links. It should all be correct as they are added, but probably best to keep open to track known issues.

chrisalleng commented 1 year ago

@J1mBob is keeping this open of any use to you? I was planning on doing a big audit to see what was missing artwork wise, if anything.

661 still open and resolves some of this as well

J1mBob commented 1 year ago

I don’t need it open. Let me know what images are missing and I’ll chase them down.

https://infinitearenas.com/

From: chrisalleng @.> Sent: Friday, September 22, 2023 2:56 PM To: guidokessels/xwing-data2 @.> Cc: J1mBob @.>; Mention @.> Subject: Re: [guidokessels/xwing-data2] Convert FFG squadbuilder image links to InfiniteArenas (Issue #521)

@J1mBob https://github.com/J1mBob is keeping this open of any use to you? I was planning on doing a big audit to see what was missing artwork wise, if anything.

661 https://github.com/guidokessels/xwing-data2/pull/661 still open and resolves some of this as well

— Reply to this email directly, view it on GitHub https://github.com/guidokessels/xwing-data2/issues/521#issuecomment-1732023384 , or unsubscribe https://github.com/notifications/unsubscribe-auth/AHVNJDEDZN3HA6VBWI4AQ43X3X3M5ANCNFSM5PY7PAEA . You are receiving this because you were mentioned. https://github.com/notifications/beacon/AHVNJDDYBMEWG4GXHO6KZQDX3X3M5A5CNFSM5PY7PAEKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOM46JIWA.gif Message ID: @. @.> >

danrs commented 1 year ago

@chrisalleng I'd like to close this issue then unless you still want it open?

danrs commented 11 months ago

I will close this if no further activity in ~7 days

chrisalleng commented 11 months ago

Sorry, missed the other note. Yeah, I don't think there's any reason to keep this open. Any further missing images that come up we can just put up PRs for rather than track via this