Closed ChristopherRabotin closed 3 months ago
With changes from https://github.com/nyx-space/anise/pull/252 :
The test_monte_carlo_epoch
now runs in 0.61 s.
Monte carlo multi-thread test (100 runs) was 5.73 seconds with Cosm
is now 9.41 seconds with anise v0.4 (but was 74 seconds with anise 0.3). The decrease in performance here is due to ANISE, not the cloning itself since removing that does not lead to any improvement increase.
High level description
The dataset structure is designed to be very generic. The current architecture enables this by storing all of the raw bytes of any dataset on the heap as
Bytes
, cf. https://github.com/nyx-space/anise/blob/master/anise/src/structure/dataset/mod.rs#L56, and decoding the data just in time.The issue is that this approach is extremely slow in the real world because the DER representation is not constant size, so it isn't as simple as casting the data from a raw bytes into another representation: the data needs to be decoded. Decoding data is slow.
As an example, in https://github.com/nyx-space/nyx/issues/86, where Nyx uses ANISE for all the computations, the
test_monte_carlo_epoch
only runs 10 runs of similar spacecraft dynamics: yet its performance decreased from 0.60 seconds for all ten runs to over 7 seconds for the same number of runs.With ANISE:![with-anise-no-meta-mc](https://github.com/nyx-space/anise/assets/4823784/699c3b92-b439-49f2-9c30-699ec86363d3)
Baseline with the lower fidelity![without-anise-no-meta-mc](https://github.com/nyx-space/anise/assets/4823784/2bf1b818-9969-4cb6-a982-c02a6b18c52b)
Cosm
in Nyx:Cosm
(%)Cosm
(s)Dataset are used to store planetary constants, so every frame query requires decoding. Similarly, every rotation to a low fidelity frame also requires decoding. With an RK89, that equates to 16 requests per time step.
Requirements
Test plans
No additional test plans, the current ones are good enough. It may be worth adding some flamegraphs for fetching by ID.
Design
The
DataSet
structure is already typed by the type of data that it stores, and it also includes a maximum number of entries. The original intent was to store everything on the stack. However,Bytes
, as per its documentation, keeps track of the data in anArc
, which itself lives on the heap.As such, I think that the DataSet should own a heap allocated pointed to the underlying data so that the decoding only happens once, probably using an
Arc
for easy cloning.