[enhancement] [good first issue] Store expectation values in Dynamics

[piperfw]

Currently expectation values are calculated using dynamics.expectations() after a call to tempo.compute() (or tempo.tempo_compute()). Whenever .compute() is called, dynamics.expectations() must be called again. While not exactly computationally expensive, this feels clunky. It is likely that anyone computing dynamics will want access to expectations, and those of the same operators.

To make expectation values more accessible, I would suggest similar function signatures to those of the solvers of the QuTip library, which take as an optional argument a list of operators for which expectation values should be computed.

Specifically, tempo.Tempo() (or .tempo_compute()) could accept a list of operators whose expectations would be stored in a corresponding list, expectations, of the returned Dynamics object. Whenever .compute() is called, new expectation values are appended (and no values recalculated). Expectations of the first operator would be referenced with dynamics.expectations[0], for example.

[gefux]

Thank you Piper!

First let me first try to clarify a few things (in case there exists confusion about it): Note that the tempo.tempo_compute() function is really nothing more than just a shortcut for:

1. creating a tempo.Tempo object
2. running its .compute() function and
3. returning a tempo.Dynamics object

and that one can get expectations from this tempo.Dynamics object for lots of different operators very cheaply without having to start another costly TEMPO computation. TEMPO returns the evolution of the density matrix which is stored in a tempo.Dynamics object.

I believe you implicitly also suggest that step 2 and step 3 will always occur together and hence they should be put together - I agree.

The reason for this separation of step 1 and 2 is that TEMPO allows to make a computation up to some point in time and then continue up to a later point in time (note that this is non trivial for non-Markovian, i.e. time non-local dynamics!). This is particularly useful if one has a computationally challenging problem where you might want to separate the computations into bigger chunks.

If I understand you correctly, you suggest that the tempo.Dynamics object should remember all the previously computed expectation values and store them in a list. I think we could do that. @peterkirton : do you have any opinion on that?

Of course one can also write a function that puts all steps together. However, I am a bit reluctant in doing so, because it might lead to encourage nonsensical usage (like unnecessarily re-computing the entire TEMPO simulation for different expectation operators).

What do you think?

[piperfw]

Thank you for the explanation (it is as I understood, which is a plus for the logic/clarity of the code).

If I understand you correctly, you suggest that the tempo.Dynamics object should remember all the previously computed expectation values and store them in a list.

Exactly. Perhaps an example would make the intentions clear:

my_tempo = tempo.Tempo(...)
my_tempo.compute(5)
my_dynamics = my_tempo.get_dynamics()
results = my_dynamics.expectations(tempo.operators.sigma("z"))
my_tempo.compute(10)

Currently, results only contains expectations up to time 5. I think it would be a good idea to be able to get a reference to a list (instead of the literal returned by .expectations()) in my_dynamics containing the expectations which were automatically updated by the second .compute() call, just how states are appended to a list in my_dynamics (the difference of course is that states are expensive to calculate whilst expectations are cheap, but I think the principle is the same: if I calculate expectations for the first 1000 timesteps then evolve the system for a further 100, I shouldn't have to calculate the first 1000 expectations again if I want the last 100).

I also think it is more intuitive from a user's point of view to have expectations update alongside states, but you may disagree.

[piperfw]

A simple implementation would be to have a dictionary with operators as keys, expectations as items. Then whenever .get_expectations(operator=op) is called it is first checked whether expectations for op have already been calculated. I'm happy to work on this if you agree it's sensible (it would be a good first test of getting an idea through review).

[gefux]

Ok, I understand. I agree with your suggestion on the changes on the tempo.Dynamics object. For your suggestion above to work, however, the tempo.Tempo object must posses a reference to the tempo.Dynamics object (otherwise continuing the computation can't have an effect on the tempo.Dynamics obeject). This makes sense. I'd suggest to have a member variable ._dynamics in the tempo.Tempo class.

This is probably not the easiest extension to the code, but unless @peterkirton has any objections, I think it would be great if you have a go at this.

[piperfw]

For now I have done the minimal change to dynamics.py only: a list _expectation_lists contains previously computed expectation values as lists (corresponding to operators in a second list, _expectation_operators) with a few lines in .expectations() such that expectations are not recalculated if they have been already. I'm not sure if this is worth the additional code complexity; let me know what you think (see the PR). Nothing changes with how the user retrieves expectation values.

I could also add a hook in Tempo.compute() so that the expectation values are updated whenever the states are, but realise this probably isn't necessary as someone using the package should always call .expectations() rather than directly accessing _expectation_lists.

A final question is whether we should export expectation values (alongside times, states). I would think no, since anyone loading previously exported dynamics has access to the package and so can just calculate all expectations with .expectations() (an edge case is when someone wants to use tempo data for plotting/analysis on a system where they don't have tempo installed).

[gefux]

Hi piper, sorry, I overlooked this post when commenting on your pull request. I would advice against a hook in the Tempo.compute() method (code should never do more than necessary). However, I think it would be alright if (as mentioned above) the Tempo object would return a reference to "its" dynamics object instead of returning a fresh copy every time Tempo.get_dynamics() is called. Does that make sense?

[piperfw]

Hi, sorry I was having some fun with python versions :).

Right, I think I understand the source of confusion: .get_dynamics() returned a shallow copy of the dynamics only, so its inner variables were in fact references to the originals and hence updated by Tempo.compute(). I've removed this copying and added a new test to distinguish a copy.

[gefux]

Yes, that's great. I just merged it. I will also make Tempo.compute() return a reference to the dynamics object (just to make the usage more convenient).