The orbit analysers in the flight plan have unreasonable memory usage for long flight plans

[uriyuzhui]

I have an old RP-0 save using Gallai and just switched to use the latest Goldbach version. Now once I get the save loaded and even if I sit there doing nothing, the game crashes within one minute. This is consistent and occured everytime. I pulled out resource manager and can visually confirm a steady growth of commit memory until it exceeds my RAM limit (32GB). The below screenshot was taken just after game crashed.

I read the version notes and seems that now it calculate the trajectory on scene change, so I wonder this might be the issue? I had about 12 spacecrafts in-flight, most of them being LEO satellites with no flight plan. But there were a couple deep space probes which had long flight plans going 2000+days ahead and utilizing multiple gravity assists. I guess they might be the root cause, but at least I was able to create these flight plans with the older Gallai version.

Logs (the journal alone is 1GB before compression) https://public-data-share-3258311398.s3.amazonaws.com/crash_logs_0314.zip

[uriyuzhui]

I reverted to 2020111505-Гельфанд-0-g55ba2866b81ac4e181c515180fbe5e963bbdddb2 and so far things are fine.

[al2me6]

I observed the exact symptoms while playing on a newly created sandbox game with only two vessels in LEO, one of which has an extremely long (~2-6k days) flight plan for a Jupiter-assisted trip to Neptune.

It would appear to me that the memory usage growth is triggered by extending the flight plan length, the number of steps for the flight plan, or the number of steps for normal trajectory prediction (I can't tell which).

Curiously, physical time warp seems to be correlated to the rate of increase (note the flatter parts near the beginning and the end; the center part was running at 4x physics warp):

I have also seen the memory growth stop before OOM is triggered, though I neglected to take a screenshot of Task Manager when that happened. However, I recall that the memory usage looked approximately like so (apologies for the rough sketch):

Logs (though they aren't particularly illuminating to me): https://gist.github.com/al2me6/3d4a001747210ae53988b799ece37fba

Journal (compressed): https://drive.google.com/file/d/1GtzvaboffHinh2K7n2pEhtIh485cyUZf/view?usp=sharing

Specs, for reference: Ryzen 7 2700X + 32GiB DDR4 @ 3466MHz.

This was observed with Principia Goldbach on both KSP 1.9.1. and 1.10.1.

(@hxl11211 IMO you should reopen this issue.)

[pleroy]

The original report was about "memory usage grew between Gallai and Goldbach", which is almost certainly related to orbit analysis in the flight plan (although I didn't feel like doing the dichotomy needed to ascertain this).

Your report is about "long flight plans require lots of memory". Other than the word "memory", there is not much in common between the two. There are always going to be complex situations (lots of vessels, long trajectories, etc.) which are going to require tons of memory, and surely the memory usage will grow as you ask for more data. That's life.

[al2me6]

I apologize if I was unclear, but I do believe that I am, in fact, experiencing the same problem.

If I may quote OP,

But there were a couple deep space probes which had long flight plans going 2000+days ahead and utilizing multiple gravity assists. I guess they might be the root cause, but at least I was able to create these flight plans with the older Gallai version.

An extremely long plan is exactly what I am experiencing problems with. Solely from a player's perspective, I believe that a Jupiter-Neptune trajectory is routine and not particularly complex (I realize that this may be technically complex due to its length -- I do not mean to discount that).

I should have clarified, but what I observed was not just "high memory usage," but rather, what appears to me to be unbound growth in memory usage, triggered by extending the flight plan. Note that the increase continued even after the flight plan appears to have been fully computed (as in, the plotted trajectory has visually reached the requested length).

I have just downgraded to Principia Гельфанд, as OP did, and can confirm that there are no issues with the exact same flight plan. In fact, I am able to extend it past 10,000 days without problem:

[eggrobin]

Oh, this is the analyser in the flight plan computing orbits at history parameters for the entire length of the flight plan (which is its extended_mission_duration) and keeping them as computed—as opposed to downsampling them like the actual histories. In contrast, the flight plan itself, being computed at a variable step, has a very low memory footprint and has no need for downsampling.

We should probably do downsampling for the trajectories that back the analysis.

In a sense, this is #228 again (WE_LOVE_228).