Goz3rr / SatisfactorySaveEditor

297 stars 78 forks source link

Can't open Update 6/7/8/1.0 Release Saves #267

Open budak7273 opened 1 year ago

budak7273 commented 1 year ago

Error SatisfactorySaveParser.Exceptions.UnknownSaveVersionException is thrown.

This is because the editor has not yet been updated to handle saves made in these versions of the game.

Instead, use another tool like SCIM - https://satisfactory-calculator.com/en/interactive-map

An issue popup like the one below will be displayed: image

budak7273 commented 1 year ago

There is no known ETA for @Goz3rr fixing this.

R3dByt3 commented 1 year ago

@budak7273 @Goz3rr I'm currently working on a fix, but in a new repo / solution, since I honestly don't like some of the parser code. But basically you should be able to transfer my changes when I'm done.

Since I learned from this repo how this wild file format works, please feel free to take something back.

I will write here again when I'm done, hopefully around this weekend.

budak7273 commented 1 year ago

@R3dByt3 Hey, great to hear the codebase has helped you, and thanks for keeping us in the loop. Have you seen the new parser branch? You might find the code there more agreeable.

It's not clear to me what Goz3rr's plans for the repo are, but I lack the save structure knowledge (and time) to continue this project.

MGerdel commented 1 year ago

@budak7273 Oh thanks for the info with the branch, I will take a look at this one (Silly me, didn't check that). No problem, you'r welcome, that's what open source was meant to be.

R3dByt3 commented 1 year ago

Ah yes don't mind me forgetting that I was still on my company github account.. sorry ^^

budak7273 commented 1 year ago

The secret is out!!!! They're looking at SF stuff during work!!! Haha

budak7273 commented 1 year ago

Any updates @R3dByt3 ?

R3dByt3 commented 1 year ago

@budak7273 Hi, sorry for the late response my initial success got a little bit delay so here is the current status:

I just stumbled on 2 little issues.

Currently I would shift the ETA by 2 weeks, so I can finish my bachelor first. But I expect to be able to finish it.

budak7273 commented 1 year ago

@R3dByt3 Thanks for the update. Good luck with your degree, and no rush on this side. I'm sure some other changes will be needed for U8 as well, and there isn't a release date for that yet.

You might have already seen this, but some other people have also modified the codebase in PRs and forks. Their work could help with setting up your version. Also, there are a few other save editor devs on the modding discord if you want to bounce ideas off of others. A join link can be found at ficsit.app

R3dByt3 commented 1 year ago

Yea thanks for this info as well, but I think I'm fine using the satisfactory calculator project. The discord like is anyway rly nice, thanks for this too.

I also hope that U8 does not break everything directly after again ^^'.

Since you seem well informed, do you know a proper way to get in touch with the SatisFactory devs? Because I'm sure, that I can implement a save game data structure, that will have a comparable file size without the use of compression. And by that this would cause the saves to load and save multiple times faster.

AnthorNet commented 1 year ago

A new save data game structure won't be compatible with the unreal engine ;)

R3dByt3 commented 1 year ago

They could implement it very cheap on their own ^^' Instead of writing so many duplicated data that a compression from 180 mb to 6 mb becomes possible

Goz3rr commented 1 year ago

I think the save format was a conscious choice for development ease. If they had picked something for speed or size instead it wouldn't have been close to what UE serializes

R3dByt3 commented 1 year ago

I know what you mean, but u can still have both at once, thats what I would like to demonstrate. My version supports deprecation, extension, modification and even reading data without having to serialize all data ^^.

UE simply serializes all data present, nothing wrong about it. But if a few MB already require compression, wouldn't it be nice to remove duplicated data instead of writing duplications to compress these afterwards?

After I have finished reading and writing the current save files I will create my format; Feel free to take a look when I'm done.

R3dByt3 commented 1 year ago

@budak7273 Yet another update:

Work is still in progress, while im figuring out the bad documentated parts of the file format, also I won't be able to spend much time on this project in the current week. Maybe I will be finishing after U8 anyway, since new Items will be added, like you said..

Anyway I will finish this v1 after best guess regarding some parts of the datastructure; Maybe @AnthorNet has some more knowledge about the format, since he wrote the InteractiveMap. But sometimes I have the feeling, that we all have to guess at some point.

What do you think about the following proposal?

After finishing the v1 Reader / Writer lib, I would like to introduce an API to collect anonymised save files in order to create statistics for analysis. Would you be willing to add an option later on, that your UI would send, if the user agreed to, opened savefiles to this API?

From this knowledge we could propably improve the editors for savefiles or have the satisfactory devs published anything regarding the format themselfes?

budak7273 commented 1 year ago

@R3dByt3 Update 8 is delayed on Coffee Stain's side as well, so it's still feasible you could finish before it comes out. Some of the SF devs frequent the modding discord, you can find a join link on ficsit.app.

I think the save uploading service is a notable departure from this tool's original purpose; let's focus on getting it working again before adding new capabilities. If you're looking for saves to test with, people frequently post them on the reddit.

budak7273 commented 1 year ago

@R3dByt3 Checking in again. I have a bit more time available now.

R3dByt3 commented 1 year ago

@budak7273 Great to hear, I will be continuing work the upcoming week too. Last week was very unpleasant for me so I made no progress. If you should have any questions feel free to ask.

budak7273 commented 1 year ago

@R3dByt3 any updates?

R3dByt3 commented 1 year ago

@budak7273 Sadly not, like I said some time ago that I'm currently very busy. All I can tell is, that update 8 save files contain breaking changes. I was able to reverse engineer a few of these already. Hopefully I can continue my work on this stuff in ~1 month.

simplexidev commented 1 year ago

Are there any resources on reverse engineering the save files, and/or is it a structure listed in their community header files (or SML headers)? I'd be more than happy to help with this.

R3dByt3 commented 1 year ago

@simplexidev The last time I took a look (2-3 months ago) the community resources were incomplete for up to SF7 and not existant for SF8. Your best options are:

  1. Translating SCIM
  2. Reverse engineering / Try by error
  3. Trying to get some contacts or help from other ppl.

I think these are all the information I can share with you.

Additional helpfull would be aquiring knowledge about the UE save file format.

After you got to read the header correctly you have to weirdly decompress multiple body chunks and recombine the bytes again. At this point lots of repeating objects wait to be read from you and when you think you'r just about to be done the "Extra-Bytes" are lurking around the corner. (This is the incomplete part of the docs).

Now you can translate a few hundred lines from SCIM.

Also check repeatedly if the read content seems correct to you. If not you may be reading without an error but in the end the content is simply corrupted.

simplexidev commented 1 year ago

Well, that will be something to look into while I'm free for the next week or so.

kfpopeye commented 10 months ago

@simplexidev @budak7273 @R3dByt3 Are you guys still working on this? I have some C#, .net and WPF experience and would like to help update this to SF8.

R3dByt3 commented 10 months ago

@kfpopeye hi I'm about to pick up my work on this project again, but I don't have much spare time at the moment. The actual state is the following: Reading up to SF7 looks good but is not done since all the extra bytes properties are very time consuming. Also I maybe would like to update my architecture after understanding the full format. SF8 saves contain several heavy breaking changes and I guess it was also not wrong to wait for the release or / and at least some documentation.

Currently I mostly would like to have a collection of save files for each version of the game. So I can run more tests and analysis.

As soon as my version looks presentable for the internet, I will publish it on GitHub.

budak7273 commented 10 months ago

@kfpopeye I'm not actively working on this project any more, as I have zero save structure knowledge and I'm working on the modding toolkit instead, but (as far as I know) I can still approve PRs here and make releases and such.

R3dByt3 commented 6 months ago

@kfpopeye @budak7273 @simplexidev @Goz3rr

Its finally done... I've got a working reader for the current version, but please notice: This is as experimental as the experimental branch of the game ^^'

Feel free to check out my reader and test it with your files; If you run into any issues create me an issue. As soon as I can consider the reader stable I will create a prerelease NuGet package and start with the writer.

The repo may contain bugs and code that has to be removed: https://github.com/R3dByt3/SatisfactorySaveNet

R3dByt3 commented 5 months ago

@kfpopeye @budak7273 @simplexidev @Goz3rr

FYI:

I tested my component with a bunch of save files and after some minor fixes I would consider the component as reliable and stable with support for all values available in SCIM for example.

If you should encounter any issues or need of help, feel free to contact me.

Pysis868 commented 1 week ago

Update 8, not 1.0 now? Last month's release of yours (0.1.5) might just be a small fix?

R3dByt3 commented 1 week ago

@Pysis868 The reader currently supports all save file versions but 1.0 that is correct. As "stated" here: https://github.com/R3dByt3/SatisfactorySaveNet/issues/6 support for 1.0 is intended to be added "soon tm".

But im honest about a few things:

Overall I'm sorry about the delay, havn't found time to look into it so far; Its on my soon personal roadmap. But I guess I have to start with getting an overview of whats changed, bcs this update and some other information, give me the feeling that it will be comparable to the 8.0 update about the amount of required work.

Stay tuned - The repository will not be forgotten!

Pysis868 commented 1 week ago

It's good to have another large project. Found this one because someone linked it. Just didn't see version support explicitly stated and wanted that clarified.