Crash when staging/decoupling

[lpgagnon]

Crash to desktop when hitting spacebar for a staging even involving at least one decoupler. I think I've only seen it happen after having reached LEO.

Last seen in test-2626 build, but I've been getting this one every few days since Fuchs; earliest FATAL.log instance is 2020-06-05.

ERROR.log:

Log file created at: 2020/07/12 10:42:29
Running on machine: LPG-GTX
Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg
    @   00007FFA1ADD63EF    google::LogMessageFatal::~LogMessageFatal [0x00007FFA1ADD63EE+46]
    @   00007FF9E13170D8    (No symbol) [0x00007FF9E13170D7]
    @   00007FF9E14FF990    principia__VesselVelocity [0x00007FF9E14FF98F+1342959]
    @   00007FF9E14FEF9D    principia__VesselVelocity [0x00007FF9E14FEF9C+1340412]
    @   00007FF9E149E025    principia__VesselVelocity [0x00007FF9E149E024+943236]
    @   00007FF9E1385288    principia__CatchUpLaggingVessels [0x00007FF9E1385287+231]
    @   000001CEF5A158B7    (No symbol) [0x000001CEF5A158B6]
F0712 10:42:29.588973  8164 discrete_trajectory_body.hpp:196] Check failed: --timeline_.end() == it Append out of order at -1.28352951710541439e+09 s, last time is -1.28352951496634579e+09 s

INFO.log

[eggrobin]

Same remark as in #2640: We need the INFO log, not the ERROR log.

Also, « append out of order » issues can be very tricky to debug, so a journal would help if you can reproduce the issue.

[lpgagnon]

INFO.log added. journal's not likely; I haven't seen any pattern to the occurrences

[pleroy]

Decoded stack trace below. I have a vague recollection of seeing out-of-order appends in this code before... Ah, it was #2507.

https://github.com/mockingbirdnest/Principia/blob/6dc7d900c5eef378c1a28d8160db602e0d9925a4/ksp_plugin/interface.cpp#L348-L353 https://github.com/mockingbirdnest/Principia/blob/6dc7d900c5eef378c1a28d8160db602e0d9925a4/ksp_plugin/plugin.cpp#L772-L799 https://github.com/mockingbirdnest/Principia/blob/6dc7d900c5eef378c1a28d8160db602e0d9925a4/ksp_plugin/vessel.cpp#L234-L240 https://github.com/mockingbirdnest/Principia/blob/6dc7d900c5eef378c1a28d8160db602e0d9925a4/ksp_plugin/vessel.cpp#L624-L672 https://github.com/mockingbirdnest/Principia/blob/6dc7d900c5eef378c1a28d8160db602e0d9925a4/physics/discrete_trajectory_body.hpp#L181-L196

[pleroy]

Last bit of the INFO log:

I0712 10:42:29.467294  8164 vessel.cpp:126] Extracting part KzProcFairingSide2.TU (77FABE01) from vessel 0133 station crew (018369b3-02b6-4ab1-8955-bd2f6ea2f96f)
I0712 10:42:29.467294  8164 vessel.cpp:116] Adding part KzProcFairingSide2.TU (77FABE01) to vessel 0133 station crew Debris (adcaef30-19ba-4ccf-ab40-b2450aee3946)
I0712 10:42:29.468291  8164 plugin.cpp:565] Collision between HeatShield2 (97846DF6) and KzProcFairingSide2.TU (77FABE01)
I0712 10:42:29.468291  8164 plugin.cpp:565] Collision between KzProcFairingSide2.TU (77FABE01) and HeatShield2 (97846DF6)
I0712 10:42:29.468291  8164 vessel.cpp:168] Preparing history of vessel 0133 station crew Debris (adcaef30-19ba-4ccf-ab40-b2450aee3946) at -1.28352951496634579e+09 s
I0712 10:42:29.587976  8164 plugin.cpp:565] Collision between HeatShield2 (97846DF6) and KzProcFairingSide2.TU (77FABE01)
I0712 10:42:29.587976  8164 plugin.cpp:565] Collision between KzProcFairingSide2.TU (77FABE01) and HeatShield2 (97846DF6)

The "Preparing history" message seems to indicate that we have a newly created vessel with a history containing a single point (the barycentre of the parts at the time when the vessel was created), and that we later try to insert the barycentre of the parts' histories, which has a point before the start of the new history.

@lpgagnon: Did you have any engine/RCS burning at the time of the decoupling? (Grasping at straws...)

[lpgagnon]

In this case, rcs was probably active on the primary vessel ("133 station crew"). "133 station crew Debris" would be a single-part "vessel" (KzProcFairingSide2.TU) being jettisoned, so nothing firing there.

[lpgagnon]

Haven't seen this happen since upgrading from Galileo to Gallai now seen with Gallai as well

[Bellabong]

I get this from time to time as well and just as unpredictably/inconsistently, but when I turned on journalling for the two times I could replicate the crash from a save... it didn't crash.

[lpgagnon]

not seen since Gallois, I believe

[Growflavor]

Might the following fatal when decoupling while orbiting Trappist1e be related?

To make the Principia logs short here is a fatal at decoupling of fairings of a probe already orbiting 1e by loading a saved game:

https://drive.google.com/drive/folders/1rBWbw6P4icUrzB2m8YriKyuL_eRKSjjn?usp=sharing

[barcharcraz]

INFO.20210430-124413.18744.log

I've gotten this crash in Grassmann, info log attached

[pleroy]

To all the people affected by this bug: I don't think we have a hope of solving it without a journal. The logs don't tell us much, other that the curious "Preparing history" message milliseconds before the crash. And as for the stack: well, I have stared at this code for quite some time without finding anything suspicious.

Given that someone runs into the problem every few months I have to believe that this is a rare cornercase, and these typically require that I can reproduce them under a debugger.

[pleroy]

Note to self: might be the same crash as #2931, for which we have a journal.