Stream Simulation Resources

Tickets addressed: Closes #1096
Review: By commit
Merge strategy: Merge (no squash)

Description

Should not be breaking.

Extracts resource management out of the engine and into a new class called the SimulationResourceManager the InMemorySimulationResourceManager operates nearly the same way as on develop, except that the serialized form of the profile segment is kept around, rather than the pre-serialized form. The Simulate action uses the StreamingSimulationResourceManager, which streams profile segments to the DB in batches.

Additionally, the engine has been slightly refactored:

the step and init logic which was repeated verbatim in all simulate methods has been pulled into methods in the engine
the engine now tracks its own timeline, cells, and elapsedTime
- related, the logic of tracking mutliple timelines for checkpoint simulation has been simplified into the engine containing a referenceTimeline, which is defined as the referenceTimeline combined with the timeline of the engine it was copied from
computeResults and related methods have been made non-static, as they require an instance of a simulationEngine to execute and now rely on internal fields of the engine.

Additionally, since all but the SynchronousSimulationAgent are unused, the interface SimulationAgent has been flattened.

Verification

Manually verified that streaming is working.

Did some profiling on streaming vs develop. I used Clipper’s gnc-all-activities plan and ran profiling twice, once with high-overhead memory instrumentation and once with async sampling. Findings:

Streaming has a positive impact on overall Heap memory usage. From what I can tell, the heap usage is always <= develop, with the difference being ~0.1GB, and given that the max heap usage is ~1GB, that’s up to a 10% save (tho its closer to a 0% save while streaming). I’d definitely prefer to do this comparison again against a plan that consumes more memory, though.
- I took a look at the breakdowns and, from what I can tell, in Streaming most Heap memory was in the young “Eden” scape, while on develop most Heap memory was in the older “Old Gen/Survivor” scape (preference seemed to depend on whether I was profiling memory)
For non-heap memory, streaming appeared to have a neglible to slight negative impact--although we’re talking within a couple MBs. Of note is that Develop has an increase of non-heap memory at the end, while Streaming is consistent all the way through--at least when doing memory profiling. When not profiling, while Streaming starts with a higher non-heap memory, at about halfway through the usage is just under halved, while develop’s usage remains approximately constant. I don’t know what the subcategories for non-heap memory mean, so I’ll skip an analysis.
Streaming took slightly longer than develop - 27min vs 25min with profiling on, 2min v 1min with profiling off. This may be avoidable by having streaming work in its own thread.

I also ran a JDBC profiler alongside the memory profiler and of note:

the threshold for streaming may need to be tweaked, as Streaming spent 50 seconds total awaiting Network I/O calls when inserting profile segments (reminder that this simulation took 27 minutes total), although the average individual wait was 150 microseconds. Similarly, it spent 18 seconds total on the segments being actively written, with the average execution of 50 microseconds
- Taking a quick look at the code, I accidentally enabled the “returned generated columns” flag on the “insert segments” statement, which may be the reason the Network I/O was so high for that statement, since that flags asks the JDBC to request excess data back from the DB
We may want to take a quick peek at simulation_extent, as an abnormally long amount of time was spent updating this table--8 seconds overall on develop, 3 seconds on streaming, with the average execution time for the statement being 11ms on develop, 3ms on streaming.

Raw Snapshots for those curious: Streaming.zip

Documentation

No docs need to be updated.

Future work

Investigate simulation_extent

NASA-AMMOS / aerie