search

ferram4 / Ferram-Aerospace-Research

Aerodynamics model for Kerbal Space Program
Other
239 stars 131 forks source link

Intermittent Failure to Apply Vehicle Aero Properties #83

Closed ferram4 closed 9 years ago

ferram4 commented 9 years ago

As noted on the forums, as of v0.15.2 "Ferri" drag can still end up being 0 on valid vessels in odd circumstances. Logs so far have no indication of anything being wrong, nor has anyone been able to provide reproduction steps beyond "I launched a few rockets and it broke." Also, most of the installs this occurs on seem to have a large modlist, so mod interactions can't be ruled out. Finally, I've never been able to hit it myself, so I can't test this alone, unfortunately.

Indications are this is not related to #74, as that required a game restart to fix, while this only requires a scene reload. Further, none of the results seem to indicate a race / deadlock in the voxelization code, which would result in log spam (from the main thread repeatedly trying to queue a voxelization). There is no info on whether another vessel update (from decoupling or deploying gear or anything) fixes the issue.

This implies that there are three main possibilities for the error occurring:

  1. The part mesh geometry is not transformed into the proper basis for voxelization. This would lead to the voxel being empty, and then everything breaking. The error would then be somewhere in GeometryPartModule or MeshData.
  2. The cross-section data from the voxel is broken somehow. This would lead to the aero properties and section smoothing functions trying to work on something invalid. In that case, the bug would be in either VehicleVoxel.CrossSectionData() or in VehicleAerodynamics.GaussianSmoothCrossSections().
  3. The aerodynamic property determination is broken somehow. Not sure how this would happen on its own, but if that's the case, then it would be in VehicleAerodynamics.CalculateVesselAeroProperties().

So, with that in mind, here's what needs to be done:

Any help would be greatly appreciated, I'm somewhat at a loss here.

gayski commented 9 years ago

Hi Ferram, this seems to be happening to me on the odd occasion. What's the best way to go about helping you out, as at the moment there hasn't been any similarities that I've noticed when it's happened?

ferram4 commented 9 years ago

Basically, find what similarities there are. See if animating a part on the vehicle when it happens (or staging away parts) fixes the issue or not. The biggest issue has been that I can't reproduce the issue, so I'm stuck being unable to do anything.

mjn33 commented 9 years ago

I had experienced this issue once, but couldn't reproduce it. But after you mentioned staging, I realised I was staging quite often at the time it happened so I quickly threw together a test "vehicle".

FARtest.craft

Mods used:

So the test vehicle is basically one small probe core with 184 large stack separators all staging at once.

Reproduction steps:

  1. Using a sandbox game
  2. Go the SPH and load the FARtest.craft file
  3. Immediately go to launch
  4. Separate as soon as possible
  5. Wait for the probe core to land / explode
  6. Continuously switch between each stack separator until you find one that in the FAR Flight Data window has Cl / Cd / Cy of NaN.
  7. Revert to SPH
  8. Delete all the stack separators
  9. Try to go the the FAR Analysis window and Sweep AoA
  10. Result: Nothing happens
  11. Try loading some reasonable vehicle
  12. Try Sweep AoA ---> Nothing
  13. Launch vehicle
  14. Note the Cl / Cd / Cy being NaN, no lift or drag at all

I tried reloading an earlier save, but that did not fix it, neither did quitting to the main menu and coming back. Only solution was to restart the game, so my issue might be related to #74.

Player.log

Hope this helps.

ferram4 commented 9 years ago

Okay, so this particular instance looks like it's a re-occurrence of the previous voxelization issues rather than the one that people have been complaining about, simply because the one people have been complaining about can be fixed by re-loading the vehicle. So this points to another situation where the voxelization can lock up... somehow.

gayski commented 9 years ago

I've been messing around and it doesn't seem to be related to staging at all. Encountered this with a single stage plane and it's only fixed by restarting the game. The FAR GUI was showing an atmosphere - however vehicle speed wasn't translated into the Mach number and the flight data screen wasn't showing any values for lift, drag etc On 4 Jun 2015 17:36, "ferram4" notifications@github.com wrote:

Okay, so this particular instance looks like it's a re-occurrence of the previous voxelization issues rather than the one that people have been complaining about, simply because the one people have been complaining about can be fixed by re-loading the vehicle. So this points to another situation where the voxelization can lock up... somehow.

— Reply to this email directly or view it on GitHub https://github.com/ferram4/Ferram-Aerospace-Research/issues/83#issuecomment-108960362 .

ferram4 commented 9 years ago

@mjn33, I couldn't reproduce the issue with your test case. For the case you were looking at, there should be a constant logspam as it tries to revoxelize the debris (and fails), but I couldn't cause it.

mjn33 commented 9 years ago

Welp, from my testing it seems it might be a Mono-on-Linux issue (or just Mono, not 100% sure), I added a few extra Debug.Log statements to figure out what was happening exactly (https://gist.github.com/mjn33/3e0534062233e98e451c). It turns out after a while ThreadPool.QueueUserWorkItem just breaks and doesn't actually ever call CreateVoxel. I did some searching and found this http://stackoverflow.com/a/22793086, following the advice there using export MONO_THREADS_PER_CPU=300 seems to have helped.

Found this as well: http://www.mono-project.com/archived/articlethreadpool_deadlocks/

EDIT: It might not be a Mono issue, see https://msdn.microsoft.com/en-us/library/ms973903.aspx#threadpool_topic8.

ferram4 commented 9 years ago

Yeah, what's happening is that CreateVoxel is first queued up for the ThreadPool, and then that thread queues up various tasks to handle voxelization, like creating a voxel shell from a mesh or solidifying the voxel. The problem is, what happens when the only active ThreadPool threads are handling CreateVoxel? Since it can't actually run any of the mesh voxelization or solidification tasks, it hits a timeout.

So, https://github.com/ferram4/Ferram-Aerospace-Research/commit/bd6d2749e147437859b6616cffe20bc0fb05f379 includes its own threadpool specifically for CreateVoxel calls; still uses the base ThreadPool to handle the underlying voxelization tasks, though I may change that as well. This should fix the issue completely, I think.

mjn33 commented 9 years ago

Made sure I had unset MONO_THREADS_PER_CPU, tested again and it works perfectly! (touch wood) :smile:

mjn33 commented 9 years ago

Think I've found another deadlock (VehicleVoxel.cs#L179), this one caused by a memory leak it would seem.

Patch with extra debug messages Player.log (had KW Rocketry installed at the time, reproduced without) FARtest10.craft

Reproduction steps:

  1. Start new sandbox game
  2. Go to the SPH
  3. Load test craft
  4. Open "Transonic Design" tab (ignore the NaN wave drag-area, that seems related to the new subsonic drag changes)
  5. Launch
  6. Make sure the FAR Flight GUI is open
  7. Revert to the SPH immediately
  8. Repeat 5-7 until Reynolds number readout is 0 (i.e. aerodynamics breaks)

This requires a fair few launch-and-revert cycles. I added a few more debug messages in the patch linked above, from this a grep "chunks in use" Player.log produces the following:

[VehicleVoxel] 0 out of 12891 chunks in use
[VehicleVoxel] 800 out of 12891 chunks in use
[VehicleVoxel] 800 out of 12891 chunks in use
[VehicleVoxel] 800 out of 12891 chunks in use
[VehicleVoxel] 1600 out of 12891 chunks in use
[VehicleVoxel] 1600 out of 12891 chunks in use
[VehicleVoxel] 1600 out of 12891 chunks in use
[VehicleVoxel] 2400 out of 12891 chunks in use
[VehicleVoxel] 2400 out of 12891 chunks in use
[VehicleVoxel] 2400 out of 12891 chunks in use
[VehicleVoxel] 3200 out of 12891 chunks in use
[VehicleVoxel] 3200 out of 12891 chunks in use
[VehicleVoxel] 3200 out of 12891 chunks in use
[VehicleVoxel] 4000 out of 12891 chunks in use
[VehicleVoxel] 4000 out of 12891 chunks in use
[VehicleVoxel] 4000 out of 12891 chunks in use
[VehicleVoxel] 4800 out of 12891 chunks in use
[VehicleVoxel] 4800 out of 12891 chunks in use
[VehicleVoxel] 4800 out of 12891 chunks in use
[VehicleVoxel] 5600 out of 12891 chunks in use
[VehicleVoxel] 5600 out of 12891 chunks in use
[VehicleVoxel] 5600 out of 12891 chunks in use
[VehicleVoxel] 6400 out of 12891 chunks in use
[VehicleVoxel] 6400 out of 12891 chunks in use
[VehicleVoxel] 6400 out of 12891 chunks in use
[VehicleVoxel] 7200 out of 12891 chunks in use
[VehicleVoxel] 7200 out of 12891 chunks in use
[VehicleVoxel] 7200 out of 12891 chunks in use
[VehicleVoxel] 8000 out of 12891 chunks in use
[VehicleVoxel] 8000 out of 12891 chunks in use
[VehicleVoxel] 8000 out of 12891 chunks in use
[VehicleVoxel] 8800 out of 12891 chunks in use
[VehicleVoxel] 8800 out of 12891 chunks in use
[VehicleVoxel] 8800 out of 12891 chunks in use
[VehicleVoxel] 9600 out of 12891 chunks in use
[VehicleVoxel] 9600 out of 12891 chunks in use
[VehicleVoxel] 9600 out of 12891 chunks in use
[VehicleVoxel] 10400 out of 12891 chunks in use
[VehicleVoxel] 10400 out of 12891 chunks in use
[VehicleVoxel] 10400 out of 12891 chunks in use
[VehicleVoxel] 11200 out of 12891 chunks in use
[VehicleVoxel] 11200 out of 12891 chunks in use
[VehicleVoxel] 11200 out of 12891 chunks in use
[VehicleVoxel] 12000 out of 12891 chunks in use
[VehicleVoxel] 12000 out of 12891 chunks in use
[VehicleVoxel] 12000 out of 12891 chunks in use
[VehicleVoxel] 12800 out of 12891 chunks in use
[VehicleVoxel] 12800 out of 12891 chunks in use
[VehicleVoxel] 12800 out of 12891 chunks in use
[VehicleVoxel] 13600 out of 12891 chunks in use

When a vehicle is voxelized in the editor, the voxel doesn't ever get cleaned up.

Also if "Voxels Controllable Vessel" is changed to a different value, MAX_CHUNKS_ALLOWED doesn't seem to get updated which can also be used to cause a deadlock (e.g. change from 50000 to 5000000).

ferram4 commented 9 years ago

Alright, so the changes I just pushed should have fixed that issue. Adding a destructor for VehicleAerodynamics that ensures the voxel is cleaned up if it still exists should cover the main one, and recalcing the number of chunks needed each scene reload should fix the latter.

Hopefully this is the last bit of the issue. But if this is it, I have to wonder why no reproduction steps before included, "leave the editor scene. Like, a lot. Many times."

ferram4 commented 9 years ago

Since this seems to have been fixed in Froude and Garabedian, I'll be closing this issue.