search

Dadido3 / noita-mapcap

A tool to capture a Noita world as a huge image
MIT License
66 stars 8 forks source link

Not fully loaded area resulting in broken map #1

Closed noxifoxi closed 4 years ago

noxifoxi commented 5 years ago

While the mod overall worked fine, Noita was a bit slow loading the map fully before the screenshot was made. This results in some faulty chunks on the stitched image.

Is there a way around it (like waiting for the screen portion to fully load or speeding up the map generation/loading on Noitas side)? I was using the the settings on noita-dev.exe given in README.md. My image looks like shit in comparison to the example one.

The broken image.

My PC specs are pretty beefy (on the CPU side at least):

I don't think my specs are an issue.

Dadido3 commented 5 years ago

It's hard to tell why noita loads chunks slowly on your system. No idea if you are using an SSD or if that even matters when noita generates the world.

Unfortunately i haven't found a way to tell if the map is fully loaded, so i have to rely on timing (ingame frames). There are a few ways to to reduce missing chunks:

You have to test if one of these or a combination are helping to fix the problem. Once i find a solution that works better, or when API functions are introduced that help to fix those problems, i'll update the mod. For now everyone has to tweak the parameters a bit if there are artifacts.

noxifoxi commented 5 years ago

I'm using an Samsung 850 EVO, SATA-SSD, but I was sure this doesn't even matter. I will try different framerates and/or increasing CAPTURE_DELAY, because I was already using 840x840 ingame resolution according to README.md. I'm not sure what mTrailerMode does, but I had it disabled.

noxifoxi commented 4 years ago

I still don't know what mTrailerMode does, but keeping it enabled helped a lot having the entire game window filled, but made the whole process a bit slower. Also increasing STREAMING_CHUNK_TARGET to 32 helped a little bit. I lowered the framerate to 300, this might also helped. Having the framerate set to 60 wasn't helpful at all and just made the capturing process unnecessarily slower. CAPTURE_DELAY was set to 30, I still had a few screenshots with black rectangles, I deleted them all (I hope, 675), set the CAPTURE_DELAY to 150 and ran the capture process again. 150 frames delay was a bit overpowered (took longer than needed) but my CPU was able to load the required area before the capture happened. I guess a capture delay of 60-100 might be a good setting for me, but it would actually extend the capturing process a lot. I guess I have to try a bit more.

tl;dr, the results so far:

The resulting image

A little bit of troubleshooting experience:

I had a weird result when I tried 60 fps with a CAPTURE_DELAY of 60 and mTrailerMode disabled. The game window was fully filled and generated after moving the view to the desired coordinates. But just in the moment the capture started, some portion of the game window got blacked out, resulting in those weird rectangular holes on the full map. I'm not sure why but as the result of this I think it has something to do with the capturing process itself.

btw, my taskmanager while capturing a full map: tskmng it jumps between 50% and 80% Load (Its just 10% because I paused the game to take a screenshot)

Sometimes the game doesn't keep up generating anything when the CPU load is 100%, but the capture process still continues. Pausing the game to let it finish generating until the CPU is back to normal helps a small amount but requires continuous monitoring of the process and doesn't give you back the already lost frames.

Dadido3 commented 4 years ago

It's strange that Noita behaves so different on your side, your machine is definitely not too slow.

Normally 100% CPU load shouldn't be a problem, atleast here. In that case the game will run slower, but the world tiles get loaded after some ingame frames anyway. Perhaps Noita has some "problem" with that many cores, as in that it creates many worker threads, but the limiting factor is the IO or RAM throughput. In that case it could be that some of the worker threads don't get a tile ready in time, but i have no idea how it really works inside.

As i wrote in discord, you can try to disable some multithreading related settings, as in set single_threaded_loading="1" inside of your config.xml. There are some more in the in magic_numbers.xml:

If you create a map the next time, you can experiment with those.

Just as a sidenote, saving of the screenshots uses a hardcoded amount of 4 worker threads. That shouldn't be a problem on your machine either.

Regarding STREAMING_CHUNK_TARGET, i assume it has something to do with the amount of chunks/tiles that are generated/loaded simultaneously. And mTrailerMode seems to cause chunks to be recalculated/refreshed every (few) frame(s), but similar to STREAMING_CHUNK_TARGET it's pure speculation and not tested thoroughly. Only the Noita developers really know.

noxifoxi commented 4 years ago

I tried a few options in order to render the Nightmare mode, as you can see here. single_threaded_loading="1" in config.xml made the whole process slower, increased my CPU load on all cores about 20% and still produced faulty images. So I dropped that idea after a few tries. For the capture this time I switched back to 600 fps and set CAPTURE_DELAY = 40. It almost worked out but there are still a few spots wrong (besides the broken map generation of Noita).

Maybe I have to increase the capture delay further, but every frame more waiting will result in a much longer capture session.

The next time I will experiment with USE_CUSTOM_THREADPOOL and DEBUG_THREADED_WORLD_CREATION.

noxifoxi commented 4 years ago

Update on working settings

I captured another one of the updated Nightmare mode and got really good results but to the cost of a really long capture process:

I got about 5 faulty images, which where easy to delete and to recapture. The not so exiting part about this is the time it took. I spent 06:09 hours on the full map capturing process, in my opinion this is way to high.

I still didn't touched USE_CUSTOM_THREADPOOL and DEBUG_THREADED_WORLD_CREATION

tl;dr Very good results with these settings but it was very very slow.

Dadido3 commented 4 years ago

Thanks for sharing your settings, it will help people with similar problems.

Btw, you don't need to delete faulty images if the other images overlapping parts are captured correctly. The stitcher will filter those faulty images automatically.

noxifoxi commented 4 years ago

The stitcher will filter those faulty images automatically.

I know, but in my experience this these parts will be darker and I want a perfect image. OCD kicks in

I think this capturing time is unacceptable and I need to optimize something somewhere, if I'm able to...

Dadido3 commented 4 years ago

One workaround i could think of is to add a mode to the stitcher to detect and remove faulty images automatically. This way you could recapture the faulty images (But with a higher CAPTURE_DELAY setting)

Dadido3 commented 4 years ago

I added a cleanup mode to the stitcher, see its README.md

This should get rid of most missing chunks:

./stitch -cleanup 0.999
noxifoxi commented 4 years ago

The cleanup mode is actually a life saver, really good job. (Here is the current map capture using the cleanup mode)

I talked a bit with the developers, Petri fixed the assault errors and it doesn't seem to crash anymore. I also asked if its possible to implement an API-function to check if the screen area is done generating. Maybe they'll implement such thing, I'll let you know after they answered.

Dadido3 commented 4 years ago

Such an API function would be great, it would speed up capturing tremendously and fix all the problems. Thanks for asking the devs.

Dadido3 commented 4 years ago

All the problems should be gone now, as i use DoesWorldExistAt() now. Also capturing should be much faster than before, as there are no delays anymore.

noxifoxi commented 4 years ago

Good job @Dadido3, unluckily the Noita devs didn't reply to any of my messages anymore and I got a bit mad about that fact, that I stopped following the development of Noita and the mod API. So I didn't noticed they finally added DoesWorldExistAt(), otherwise I would've messaged you.