[Feature proposal] Save game screenshot

[DeviateFish]

Let me preface this by saying that I really enjoy the work you've done here, and think you've built a wonderful tool for the Satisfactory community. I use the interactive map almost every time I play, to figure out where to build, to sanity check the inputs/outputs of factories. I haven't yet dabbled in any of the more advanced features, like editing save files, but it looks like quite the suite of features.

Using this tool has given me an idea. I have a proposal for a new feature, one that I'd like to implement if I get your approval.

I would like to be able to provide the tool (or a new tool) with a whole list of save games, and have it generate a timelapse of the whole map across all those saves. I think this would be a really cool way to look back on the history of a particular Satisfactory playthrough, to see how factories and setups evolve over time. The quality of the timelapse would obvious depend heavily on the saves themselves, but I think it could produce some really cool results, especially for people who regularly produce Satisfactory content.

This is a huge feature. So far, from browsing through the codebase, looking at the save file parsing, and looking at the supporting libraries, some big concerns arise:

• Is this possible to even do in the browser?
  • My gut says "no", but this might be wrong. The size of the save file, as well as the performance of loading the saves and rendering the layers seems like it would lead to a pretty poor experience. Perhaps that's acceptable to produce what would be largely static content? I'd be pretty concerned about the memory footprint of trying to load more than a few save files into memory, as well. I have one playthrough which so far has many tens (perhaps more than a hundred) saves. I'm not sure the individual size of each (I'm currently away from that computer for a couple days, so I can update this later), but I suspect they only get larger over time.
  • Even if this is feasible in the browser, I don't know that doing the UI components for it would be the sort of thing I would do particularly well at. I can certainly do the technical bits from "list of saves" to "list of frames for a timelapse" (or video/gif), but figuring out how to incorporate that into the existing UI would be something I wouldn't feel confident doing.
• Is the codebase in a place where this could even be done?
  • The answer here is pretty clearly "no". There would need to be a decent amount of refactoring done to separate the various concerns involved here. A timelapse would probably only be interested in the output of the Leaflet canvas multiple times over, meaning all the code that changes the UI would probably need to be decoupled from the map generating code. This doesn't seem particularly hard, but it would require a very large number of small changes.
• Is this something that can be done on the server side?
  • Currently, the answer to this seems to also be "no". Leaflet makes a lot of assumptions about being run in the browser environment, and there doesn't seem to be any current plan to change that. This isn't an insurmountable problem, but would involve reaching into the internals of Leaflet and shimming out quite a number of things. This can probably be done with some combination of that and something like node-canvas, so this seems like a surmountable challenge. I've definitely pulled off these sorts of shimming things in the past, so it's definitely possible.
  • I think I'd prefer to do this as a server-side or command-line tool, since I think it would be capable of far better performance. Beyond the initial load of the save files to determine ordering, this is definitely a highly parallelizable problem (probably one thread/process per save file/frame). It's possible this would be better suited as a separate tool entirely, that uses this library as a dependency, but this would still require the above refactors.

Given all the above, this seems like a pretty big project, but I do think it can be broken down into a set of smaller projects that each are more tractable:

• Refactor mapping code to better separate map- and UI-rendering concerns. I think this would look like breaking all of the map-related (i.e. Leaflet) code out of GameMap and into a separate class that is controlled by GameMap. This would let GameMap still render everything it needs, while also manipulating the UI as it currently does, while also letting the new component class be used independently.
• A similar refactor for BaseLayout which also directly manipulates a number of UI elements.
• Refactor some of the mapping code to take a Leaflet reference, instead of depending on the L global
• Refactor a bunch of the mapping code/systems to be a little more modular. Many of the functions that render significant parts of the map are large monoliths that could be broken into smaller functions, allowing certain functionality to be skipped when rendering for the timelapse mode. My rationale here is that when rendering a timelapse, we probably wouldn't ever care about showing markers for things like resource nodes, loot crates, etc, so making them easy to skip would be beneficial. This might essentially boil down to having multiple "renderers", one which is much more configurable and wired to UI elements, and a second which is much more static and can be used to produce image outputs (i.e. frames in the timelapse).

Some other questions/concerns:

• If I were to take this on, what kind of testing would be required to make sure these refactors aren't breaking existing code? Is there a current test script or something you run through to validate your own changes?
• Would you be interested in building out (or letting me build out) some automated testing in the form of unit tests and the like? Refactoring the various rendering components to take a configurable Leaflet reference would actually make this pretty straightforward in many cases, since assertions can be made that certain markers are created/destroyed/modified, etc
• Is this even a project you would be willing to let someone else take on? This set of tools is obviously you're proud of, and I wouldn't want to step in and make large sweeping changes that would leave you feeling less ownership over the project.
• There's some inconsistencies across the codebase that I'd be very tempted to standardize along the way. This might be overreach, as far as you're concerned--or it might be welcome cleanup. The biggest example I've seen so far is the inconsistent usage of () => {}... and function() {}.bind(this), which could be standardized to the former in (almost?) all cases. If this is a welcome cleanup, it could function as a good decent "first issue"; if it's not a welcome change, I can obviously leave it alone entirely.

Thoughts? Questions? I'd really like to make this a reality. I also know that this is your project, and I don't want to intrude or impose in any way. I think what you've built is pretty incredible, which is why I would love to use it as a starting point for this particular project. I could in theory build all the requisite pieces from scratch, drawing on your work for inspiration, but I think in many ways that would be a disservice to the work you've done here. You've also made it pretty clear that this is your code, and that it's not to be used for any external purposes. I would like to respect that.

[AnthorNet]

Lot to think of :D If you have Discord it would be easier to discuss all that. Not against PR, but some times they take more time for me to handle than keeping things in order.

[DeviateFish]

Yeah, I can definitely pop into Discord in the near future. It'll probably be in a couple days or so, since I'm currently away from home. What's your timezone, and when do you think you'd be most available to chat?

I did a little more digging and it doesn't seem like there's really a good pure-JS option for creating a timelapse from individual frames, especially if you want to do something fancy like have variable input framerates. I think variable framerates is probably a must, since I think it would make the most sense to have the playback be x seconds per hour of playtime or something like that. Since there's no good guarantee that each save in a set of files will be created at a regular interval (like every hour or something), it would probably be best to accommodate for uneven intervals from the start. Most examples of creating videos from frames in js seem to just delegate to ffmpeg to get the job done, which is... probably the right call, honestly. This does limit them to being server-side only, which introduces its own set of complications.

The "good news" is that this means any changes here could be limited to those needed to support the generation of a single static image per save game (and maybe some JSON metadata for timing purposes?), and whatever scaffolding is required to make that friendly to automation. I've done automated frame capture in the browser before (aside: I see we both like FileSaver.js 😄) with decent results (see below), but it's kind of janky, and would still require some changes to where the lines of abstraction currently lie.

Given a list of images generated from a list of save games and some information about how much playtime had elapsed for each snapshot, the process of creating the timelapse can be entirely independent and happen in any environment.

[DeviateFish]

So after sleeping on this, I realized that the most basic MVP would essentially just be the ability to export a screenshot of the map after loading a save file. Technically, everything else that needs to be done to create the timelapse can be done outside the tool, even if it requires a bunch of manual work. Some of it probably has to happen outside the tool, especially if things like ffmpeg are requried.

My preference would still be to automate it, or be able to use parts of your codebase as a library to do so, but from the perspective of introducing a minimal set of changes to the thing you've built, I think just exporting a screenshot would be sufficient.

It looks like several options exist as Leaflet plugins to do this, so it might be pretty straightforward. Some of them look like they have some caveats (some kinds of markers aren't supported since they're html and not drawn directly in the canvas element, etc), but for at least my purposes, that's an acceptable limitation.

If none of those plugins work, I think it would be possible to do manually by:

1. Getting a reference the Leaflet's canvas
2. Using something like this code to export the canvas as a PNG and save it

Obviously being able to automate the above and manipulate the canvas itself (setting zoom and canvas size, for example) for multiple save games would be more ideal, but I'd definitely be able to accomplish what I want with just the above and some additional scripts (either Tampermonkey or cli) to extract the header information from each save to get the playtime information.

I'll edit the title of the issue to better reflect this, and we can still chat on Discord about this tomorrow, but I wanted to get this down in writing so I don't forget 😄

[SignpostMarv]

@DeviateFish

Is this possible to even do in the browser?

past experience says yes

"list of frames for a timelapse" (or video/gif)

see #35

Most examples of creating videos from frames in js seem to just delegate to ffmpeg to get the job done

see https://developer.mozilla.org/en-US/docs/Web/API/HTMLCanvasElement/captureStream and https://developer.mozilla.org/en-US/docs/Web/API/MediaRecorder

[DeviateFish]

see https://developer.mozilla.org/en-US/docs/Web/API/HTMLCanvasElement/captureStream and https://developer.mozilla.org/en-US/docs/Web/API/MediaRecorder

Oh that's pretty nifty. Also, thanks for linking the old issue, it seems I'm definitely not the only one who has had this idea.

I'm still not entirely sure the browser is indeed the right place to do this, though. My biggest concern is the memory limitations, since it looks like the save files can get pretty big, and that's in a compressed format. I also didn't really think of doing timelapses for specific areas of the map (as opposed to just the entire map over time), so I can see that being a slightly different use-case. One is just a superset of the other, so it definitely could be built to do so. Being able to easily control the viewpoint is certainly a point in favor of doing it natively in the browser, however.

I'll take a peek at the linked APIs and see if I can cobble something together.

Fair warning though: I have no intention of doing anything here that could compete with anything @AnthorNet has built, so whether or not any of the stuff I end up building makes it into main is entirely up to them. I'll certainly be sharing my results with them, but I won't be releasing anything publicly without express consent 😄

[SignpostMarv]

@DeviateFish it may be more practical to knock up a standalone CLI tool that uses https://github.com/puppeteer/puppeteer

[DeviateFish]

@DeviateFish it may be more practical to knock up a standalone CLI tool that uses https://github.com/puppeteer/puppeteer

I had considered that, but I really hate puppeteer. It might be the least worst option out of all the browser orchestration tools, but it's still bad. Using a headless browser is probably going to be my last resort, rather than first 😄

[AnthorNet]

https://webrtc.github.io/samples/src/content/capture/canvas-record/

[SignpostMarv]

@AnthorNet 🤔 I just had a terrible idea