goatfungus / NMSSaveEditor

No Man's Sky - Save Editor
1.63k stars 228 forks source link

No longer copy/export JSON data #954

Open meachware opened 4 months ago

meachware commented 4 months ago

Just wanted to get this out here, I am still investigating and will update when/as I find out more.

I have three brand new saves and only one of them can export the full JSON data. The other two files won't even allow me to copy and paste (sub-sections). The copy is empty.

Even with the save file that exported JSON successfully cannot be read by JSON editors. There are items in my inventories (on ALL three saves) that are products that are described as follows (each entry is one product):

^\x80\x80\x062\xAA\x17#00000 ^\x80\x80\xD4\xD7\xEF\xF4#00000 ^\x80\x84\xFELT\xB8#00000 ^\x80\x80\x1C\x8F*\xB3#00000 ^\x80\x80\x17\xDC3\xCB#00000 ^\x80\x80\xFF\xEE\xC2\xC4#00000 ^\x80\x80\x14\x99\x9C\xE7#70920

They are enclosed in quotes (" ") and the carrot is used throughout NMS JSON data but isn't the backslash character used for something else?

Here's what a known product looks like: "^PRODFUEL2" or "^NAV_DATA" or "^BP_SALVAGE"

=============================== This all started because I wanted to know "how" to edit my save to allow me to run thru an expedition a second, third, etc time. Which I did figure out and there are some drawbacks but it all worked except I could not claim any items after the save reverted... thus why I'm now digging into the exporting data.

Oh and good lord! The file that DID export is over 500,00 lines (almost 10 million characters) if Notepad & Notepad++ are correct.

meachware commented 4 months ago

For reference: a brand new Survival save (w/skip tutorial) exports JSON and is 325,000 lines and 5 million characters. I can also copy just the BaseContext key for example.

For my saves that have run the Expedition it seems like the editor is hitting some sort of buffer (for the copy) limit.

And for those sharp-eyed folks, yes basically all HG did was to separate the previous data into chunks (CommonStateData / BaseContext / ExpeditionContext) where the 2nd and 3rd chunks, while completely different data are a huge duplication in the save file structure.

goatfungus commented 3 months ago

This is not a bug in the editor, but rather a complexity in the save file format that I had to correct for.

The save files were never designed to be edited as a whole block of text, but rather each value would be read into memory using data structures that the game has predefined. Some of these values appear to be pointers (memory addresses) to various preloaded items rather than the item name or id value, so they appear in the file as raw address values and are not in the same character encoding as the rest of the file.

It's kinda like having a book written mostly in English and replacing a bunch of individual words randomly with Chinese. They still mean the same thing, but if you only know how to read English then you're not going to understand it.

To cater for this issue in the save format itself, I have had to build a special way of representing these id values using \x escapes. So for example: \x80 literally means the number 128, where if it were treated as a text value would come back as invalid JSON.

meachware commented 3 months ago

Thank you so much for the very detailed reply and your continued support for this project