Some images missing

[Darxide23]

Sometimes Datacore has issues with fetching images from the game files. A current and prominent example is Kivas Fajo who is missing his small portrait image as well as images for his equipment. He has been in the game for weeks now with his images have been available in the game data since September 2nd while he has himself been available in game since at least September 16th. Emory Erickson also has his small portrait missing as another example. All crew who have been added after these crew such as Ambassador V'Lar, Alton Soong and Roger Korby.

This is an issue that has happened multiple times in the past, so I have to assume there is a bug in the code that fetches these images.

[AlexCPU]

All the small portraits appear correctly for me, you can see them all in one page here: https://datacore.app/bigbook2/. Let us know where you're seeing an issue and we can investigate further.

Kivas Fajo's equipment is more interesting however, taking his outfit as an example, we're expecting to find it downloaded here: https://assets.datacore.app/items_equipment_fajo_kivas_outfit_equipment.png but it appears to have been mis-named, and has been successfully scrapped and is available here https://assets.datacore.app/items_equipment_fajo_kivas_outfit.png

This looks like we've made an assumption about the naming format being "itemsequipment[itemname]_equipment.png", which it always has been, however it looks like TP have messed up on Kivas Fajo and missed the "_equipment" from the end. @TemporalAgent7 can you point us in the direction of where the mapping of equipment to assets happens?

[Darxide23]

The small portrait: As stated previously, Emory Erickson is also missing his small portrait.

Also it isn't just Kivas's outfit. His other card-specific equipment is also missing the icon. The Kivas’s Proximity-Activated Field. I was not able to find it on Datacore by removing "_equipment" from the end, so it doesn't seem to be the same naming issue as the outfit.

Currently, these are the only issues that I've found, but as I mentioned, this has happened semi-regularly where images are just missing from Datacore for extended periods of time after the crew has been released in-game. I had always just assumed it was a bug in the asset fetching code.

Edit: To clarify, when I say "small portrait" what I mean is that each crew has two portraits. The 256x256px one would be named "crew_portraits_cm_kivas_fajo_sm" while the 128x128px one would be named "crew_icons_cm_kivas_fajo_icon" and it's this 128x128px one that is missing for Kivas Fajo and Emory Erickson. The Big Book link uses the 256x256px image which is why they are visible there.

[AlexCPU]

There's something else wrong with the activated field, as we don't seem to have the asset for that at all, so we'll look into this.

However the crew icons seem to work for me, and we've got them all in the URLs that you've mentioned, so I'm wondering if we've got a caching issue somewhere between us and you.

https://assets.datacore.app/crew_portraits_cm_fajo_kivas_sm.png

https://assets.datacore.app/crew_icons_cm_fajo_kivas_icon.png

If you're not seeing both of these images above please can you:

1. Open a new tab in Chrome and bring up the developer tools (F12).
2. Go to the Network tab, tick disable cache.
3. Enter https://assets.datacore.app/crew_icons_cm_fajo_kivas_icon.png in the new tab you opened.
4. Click the image name, and take a screenshot of the headers and post it back here

[Darxide23]

I see both images here in your post, but still not on datacore.app. I've attached screenshots of the header from both below. The "_icon" image is showing fine while the "_sm" image is giving 404. I mixed up the two files in my previous post. The smaller of the two (the icon file) is coming up alright for me. It's the larger (the sm file) that is not.

[AlexCPU]

That's great, thanks! Sorry, I didn't realise you would see both of the images as Github has downloaded them from us, and is hosting a copy of them for this issue! I was expecting them to pull directly from the source I linked for everyone.

Anyway, it looks like a caching issue on our Cloudflare configuration. We had caching rules setup as below, and it appears that the edge cache TTL set to 1 month (seen as "cache-control: max-age=2678400" in the screenshot) also causes Cloudflare to cache 404 errors, rather than use it's standard rule of 4 hours for error responses.

I have removed the two highlighted instructions, which should cause Cloudflare to cache successes as defined in our nginx.conf (which is now set to 1 month), and failures for 4 hours.

I've purged the cache of: https://assets.datacore.app/crew_icons_cm_fajo_kivas_icon.png https://assets.datacore.app/crew_portraits_cm_fajo_kivas_sm.png So they should both start working for you immediately. I think the change to the page rules should cause Emory Erickson's images to show correctly straight away as well, but if it doesn't, let me know and I'll purge the whole cache.

[Darxide23]

It worked for Kivas, but Emory still shows a broken image. Kivas's Outfit and Activation Field are also still no shows, but I know they had nothing to do with this particular problem.

This explains why in the past some images seemed broken for weeks on end before resolving themselves. I guess I should have spoken up earlier. Seemed like an easy fix.

[Darxide23]

Emory is still broken and a new one popped up. Sarek's Benjisidrine. Datacore is pointing to https://assets.datacore.app/items_equipment_asha_benjisidrine_equipment.png

But the actual image can be found at https://assets.datacore.app/items_equipment_benjisidrine_equipment.png

I can see why. In items.json it does say _items_equipment_asha_benjisidrineequipment.png but the actual image is named without "asha" which can be seen if you go to http://stt.disruptorbeam.com/player once you've got the item in your inventory or equipped to Sarek.

[TemporalAgent7]

I think this is related to how we load item information. I guess WRG is pushing "placeholder" items in the item list before the crew is finalized.

Our script tries to only load items once and cache forever (to avoid pinging their servers too much). WRG later updates the items in the list with proper image, stats, name, etc. but we never refresh our cache. Not sure of a good long term fix (maybe refresh the last X items every time we reload crew in case they changed), but I'll try to run a manual import across all items when I get some time.

[Darxide23]

One good thing that can now be said about DB is that this didn't happen until WRG took over. I won't report anymore of these since it's all out there now.

Thanks!

[ineffyble]

We refresh the last 100 items every time we reload crew now, which has hopefully been addressing this. @Darxide23 have you noticed any issues recently?

[Darxide23]

Yes, there are some really old ones (relatively speaking) that are still missing.

items_equipment_robinson_ag_outfit_equipment.png items_equipment_robinson_ag_unusal_telemetry_equipment.png items_equipment_robinson_ag_escapepod_equipment.png items_equipment_omicron_ceti_flora.png items_equipment_janeway_polaric_equipment.png items_equipment_seska_maquis_outfit_equip.png items_equipment_paris_insurrection_ending_equipment.png items_equipment_pistol_7_equipment.png items_equipment_ddsc_32c_personal_transporter_equipment.png

But nothing newer (except a few items for unreleased crew), so I guess it's working for the newly released stuff.

[TemporalAgent7]

I see items_equipment_janeway_polaric_outfit_equipment.png but not items_equipment_janeway_polaric_equipment.png; similarly, I see items_equipment_omicron_ceti_flora_equipment.png but not items_equipment_omicron_ceti_flora.png.

Wondering if we need to refresh our items cache from scratch. Alternatively, maybe the image names you mention belong to dead equipment (that was created and later abandoned and not really used anywhere).

[Darxide23]

If you look at Polaric Janeway you'll see all of her outfits show up except for her 1* outfit at level 10. It points to the non-existant icon that I listed.

Something similar happens with Laneth where all of his pistol items display fine except for the 5* which points to items_equipment_pistol_7_equipment.png while the others all point to a preexisting item items_equipment_klingon_disruptor_pistol_22c.png.

There are several crew who do have items that change their icon as they level up (Tribble Herder Scotty's Tribbles is one), but these don't seem to fit quite the same way.

I don't know if the others are "dead" equipment, but they are what the pages of their respective crew point to with the exception of items_equipment_ddsc_32c_personal_transporter_equipment.png which I cannot find any crew reference this. Of course, I didn't do any kind of exhaustive search so there may be someone whose equipment points to that.

[Darxide23]

Midwife Worf's Uniform is also broken. I'm assuming this is because they changed the icon after his release. If the last 100 items are supposed to be reloaded when new crew are added then this should have reloaded, but it has not.