Make site data driven with JSON data source

[snollygolly]

This is my first attempt at making this tools page more data driven. I've done a good amount of reverse engineering and digging to extract proper item data and item IDs. I've taken this data and started to organize it in a master items.json object. Key highlights regarding this data file:

• It's a work in progress, I will be continuing to refine and enrich it
• There are several spelling differences between my items.json and the data on your existing page. The game's spelling was given preference (SCIENTIST FOOK-0 is actually SCIENTIST F00K-0 in the game, zeros instead of o's).
• There are a large number of items represented in the game not displayed on your original page. I am hiding those and only showing items with art at the moment.

Additionally, the page is now fetching the JSON and rendering the table directly from it. This should allow the JSON to continue to be updated and the table should reflect these changes. This should also allow you to add more features and show more advanced stats as you mentioned in #5

Let me know if you have questions.

[igonro]

Thank you so much for this PR! I will release the JSON file with all the data I gathered, because there is a little bit more data in the JSON than in the static HTML.

If it is possible, I may try to upload the "full project" to the main branch, and then create a Github Action to automatically render the static website to another gh-pages branch. Not sure if it's possible, because this is my first project using GH Pages, but I will try. This way we can make changes to the original project.

Also I'm not sure if it is better to have all the data rendered in the HTML or load the data using JS in terms of performance. Right now it is in the static HTML because I used Flask.

[igonro]

Hi @snollygolly, I have uploaded the full code and the items.json file that I had. Now the static website is built and deployed using Github Actions.

I'm not sure how I should proceed with this PR because there have been a lot of changes.

• Regarding the index.html and script.js. Has there been any significant change? Or have you just modified the code to load the data using javascript? If there aren't significant changes, I think the main branch is okay.
• Regarding the items.json. We should try to compare your items.json and the items.json that I uploaded, and join them in order to have a more complete items.json.

Sorry for replying so late, but in these dates I was a little disconnected.

By the way, Happy New Year!

[igonro]

I'm analyzing your items.json and I found something weird: some of the items have different weight values.

For example, 9x39mm BP ammo:

• In the ["weight"] key, it has the value "0.01 kg"
• In the ["ext"]["weight"] key, it has the value "0.013"

I found other similar examples for the category ammo (which is the one I started analyzing). In these cases I will drop the ["ext"]["weight"] value, since it doesn't match with the in-game values, it looks like the right one is the ["weight"].

[snollygolly]

Thanks for uploading your items.json! Regarding how to proceed with the PR, I'll leave it up to you. It would be my preference that data on that page is loaded from a JSON, but if that doesn't work for you, that's alright. The JS isn't materially different other than loading JSON from an AJAX call, and then building the table with template strings. It will be less performant, but not enough so to matter IMO.

Regarding item discrepancies in my items.json file, I see that you have the weight listed at 0.01kg and ext.weight is 0.013kg. I would actually defer to the ext version of that data. I think the difference is that your scraped data isn't quite as precise. I scraped the ext data directly from the decompiled GameMaker Studio files, so there really isn't any way for them to be wrong. I suspect the game does some rounding and that's why the values aren't an exact match.

Lastly, I would rethink the format/information in your items.json that you uploaded to the repo, it's not really structured like a JavaScript object, more like a CSV that has been jammed into a JSON format. I'd argue that it's really only useful for this specific project in its current format. Additionally, the IDs you are using in that file are wrong. Correct IDs are listed in the items.json in my PR (again, taken directly from decompiling GameMaker Studio files).

[igonro]

I've just checked in the game and you're right. The correct weight is 0.013 kg. As you say the game shows 0.01 kg, but looking at a stack of 150 bullets, it shows 1.95 kg (which is 150 * 0.013), so there is some rounding.

I would like to just use your data (after cleaning it a little bit) and drop mine since it's much more accurate, but I have some problems:

1. I cannot link the images that I have with your data. Do you know if it is possible to scrap also the inventory images? I would try it myself, but I don't know how to decompile a GameMaker game... There is a parameter called "sprite_inv" and I think it refers to the visual model of the item in the inventory.
2. In your data there are some weird items (I guess the dev used them for testing), like "NO ITEM", "ITEM DESCRIPTION"... Also other items that are not available in the game (not sure if they were available in the past or will be in the future, or just were discarded...) like "FORT 12", "REVOLVER GOLD"... Some of these items would be easy to discard based on the data, but not sure if I would be able to discard them all.

If it's possible to get the images from the decompiled files, then I would prefer that option, because it would be easier to update the data in the future (right now the way I scraped the data was depending on the "debug menu" and I had to do some manual work...)

I will try to learn how to decompile a GameMaker game.

[snollygolly]

1. When I initially scraped your table to build my first version of this data, I saved the position in the table so I could reference the images later. I messed up a bit and all of them are off by 1, but you should be able to refer to the legacy property to get the number for the image. You can see me doing this in my PR to get the images for the table: https://github.com/Zakov-Tools/zakov-tools.github.io/blob/98c3498daaae95eb600cf3d58a54be8cbfa1a2af/static/js/script.js#L41

2. In that same vein, I suppressed showing these "bad" items in my PR version of the site by checking to see if the legacy property existed and only drawing the image if it did. https://github.com/Zakov-Tools/zakov-tools.github.io/blob/98c3498daaae95eb600cf3d58a54be8cbfa1a2af/static/js/script.js#L35

If you're wanting to get into decompiling and seeing more data, https://github.com/krzys-h/UndertaleModTool is a great place to start.