search

tudat-team / tudatpy-examples

Collection of examples showcasing Tudatpy functionalities. They can be run on mybinder.org as well.
https://tudat-space.readthedocs.io/en/latest/
4 stars 18 forks source link

Create Jupyter notebooks from existing examples #2

Closed FilippoOggionni closed 2 years ago

FilippoOggionni commented 3 years ago

This issue is low-priority. It is more important to come up with new examples (see #3) and make the existing ones more on point. In addition, the .py file is preferred for example code.

However, Jupyter notebooks are more practical to describe examples. Using mybinder.org is a possibility.

EDIT: the new strategy is to convert .py scripts to Jupyter notebooks and have notebooks as main scripts. .py will still be provided for users by converting Jupyter notebooks with pandoc.org.

Tasks

Notebook conversion

gaffarelj commented 2 years ago

The tools provided by pandoc.org can greatly help for the conversion.

Edit: it can help to later convert notebooks to rst or md files, but not to convert python files to notebooks (afaik).

FilippoOggionni commented 2 years ago

Mybinder repo

I have created a repo on mybinder.org starting from the master branch of this repository (link). Every time we push to master, the mybinder repo will be updated automatically. I have verified that it runs, so everything should be set up correctly. I think we don't need a separate repository for the mybinder version (as it was done by Geoffrey here), so to avoid code duplicates I'd delete such repository to avoid confusion. If nobody objects (@DominicDirkx @ggarrett13 @gaffarelj), I will go ahead and do it.

README

I have also renamed a few examples and wrote a README, where the link to mybinder can also be found. The README summarizes the content of this repo and lists the examples with a short description.

Conversion of existing examples to notebooks

@gaffarelj I will leave this to you, as agreed. If you need help because you don't have enough time, we can split the work! I think the only important point of this process is to ensure consistency in the examples (in terms of comments and code style). This was attempted by @jo11he in the .py by implementing PEP-derived guidelines (or using a code linter, like pylint for instance) and especially by equipping each example with a header and abstract. This will be easier to do in notebooks, I guess!

gaffarelj commented 2 years ago

I see that you created a notebook-addition branch. However, since mybinder uses master, should I just ignore notebook-addition and push to master?

Edit: I see that there is the option to switch between branches on mybinder. I will then use notebook-addition, and merge it in master when the conversion is done.

gaffarelj commented 2 years ago

I will also take care of adding in the README a few instructions on how to run the notebooks directly with jupyter notebook. Specifically, how to make the tudatpy-examples (or tudat-space) environment be used by the jupyter kernel.

FilippoOggionni commented 2 years ago

I see that you created a notebook-addition branch. However, since mybinder uses master, should I just ignore notebook-addition and push to master?

Sorry about this, the notebook-addition branch was just a test. I did everything on master and also mybinder should stay on master, so please leave it like it is. I have just deleted the notebook-addition branch (I had deleted it locally but forgot to do the same on Github).

I will also take care of adding in the README a few instructions on how to run the notebooks directly with jupyter notebook. Specifically, how to make the tudatpy-examples (or tudat-space) environment be used by the jupyter kernel.

This is a good idea! However, let's not make confusion between environments. I think there should be one and only one environment that users should deal with. Currently, this is referenced in the tudat-space. Let's make sure that the environment referenced in tudat-space and the one in the tudatpy-examples repo are exactly the same (by content and by environment name).

DominicDirkx commented 2 years ago

Hi guys, really cool to see this being realized. I agree that having a single repo for mybinder and the examples is best, and in fact one of the cool features of this new setup

gaffarelj commented 2 years ago

I'm adding a list of the notebooks to be created here to make things more visible:

gaffarelj commented 2 years ago

@FilippoOggionni @jo11he could one of you two take a look at the estimation notebook, linked here, and give me your comments?

I feel like some of the markdown could use a little bit more detail starting from the Simulate the observations title.

jo11he commented 2 years ago

I will have a look tomorrow and make some suggestions :)

FilippoOggionni commented 2 years ago

I think @jo11he is indeed the best fit for the estimation notebook. I can definitely review the other notebooks while they are converted!

jo11he commented 2 years ago

I modified the notebook locally (sorry) and will upload it here.

I modified quite a bit: more extensive comments, reordering of the steps and headlines. I also took out the apriori correction, since I think it is a bit confusing and not very well suited for a basic example. (@DominicDirkx maybe I don't really get the point of the apriori perturbation on the satellite state, is it required for the estimation that goes beyond covariance analysis?) (@gaffarelj I tried to match the existing notebook style as well as possible, but there may be a cosmetic issue here and there - it may be good to have a look at it from that perspective)

DominicDirkx commented 2 years ago

I also took out the apriori correction, since I think it is a bit confusing and not very well suited for a basic example.

Good choice :) But, then I think it's good to rename this to covariance analysis example

maybe I don't really get the point of the apriori perturbation on the satellite state, is it required for the estimation that goes beyond covariance analysis?)

Indeed, it perturbs the values of the parameters before the first estimation iteration

gaffarelj commented 2 years ago

Thank you. @jo11he I added your changes to the notebook with b2c1c4fcc2d541e2877050841b6123764fc042bc.

I also edited the static .py example to reflect these, with 992ac68a350abfab32a25d00093a791ec7280595.

Both of these have now also been renamed to covariance_analysis with dbab37a32b95e4f3a27524abf92c43b880180115.

jo11he commented 2 years ago

Hi Jeremie,

sorry I should have commented on this earlier. I believe now we have something in-between an estimation demo and a covariance analysis (of an estimation) demo...

In a covariance analysis, the residual history is irrelevant, so usually the inversion is stopped after a single iteration (can be set via the estimation_convergence_checker). Instead of plotting residuals, it'd make much more sense to visualise the propagated formal errors of the estimated s/c state. I suggest we implement those changes and make it a covariance analysis. As to the name of the file, is there a better alternative to covariance_analysis.py, that somehow indicates that it is about the covariance of estimated parameters? maybe @DominicDirkx has a suggestion?

For an estimation demo, where we are really interested in showing a good fit for certain parameters, I suggest wait a few weeks and we'll have a fantastic example

gaffarelj commented 2 years ago

Just a quick update on this issue: all of the estimation and propagation examples have now been converted to notebooks.

I will convert the pygmo examples a little later, because Pygmo becomes important only later on for the P&O course, and I first want to work on the new utilities and thrust guide.

FilippoOggionni commented 2 years ago

Great job! I will review the converted ones and embed them in tudat-space.

gaffarelj commented 2 years ago

I just pushed the last two notebooks, for the Pygmo examples: for the asteroid orbit optimisation example with 831c24d95ef3204f056e7bf65b8bdd2319209a04 and for the Himmelblau minimisation with f1b13708a6ac863d9b0add77b6bf06144fffe104.

Hopefully, this could be helpful for some P&O students before they start the group assignment.

As always, feedback would be appreciated 🙂

FilippoOggionni commented 2 years ago

@gaffarelj just checking, the two-stage rocket ascent example is not working, is it?

gaffarelj commented 2 years ago

It should be 🙃 I see from the changes you pushed (showing the state of the notebook) that you could not import time_conversion from tudatpy.kernel.astro. I added this module to tudatpy back in January.

Since I setup all the notebooks to use the tudat-space conda environment, I would assume that your version of tudatpy is not up-to-date in you tudat-space environment.

Running this notebook on mybinder also works, hinting that the issue is most likely on your side.

FilippoOggionni commented 2 years ago

True, my bad, I am having a hard time using an updated conda environment to launch Jupyter notebooks from PyCharm.

In the end, the only example that does not work is the parameter estimation. An error is thrown because the function estimation_setup.create_parameters_to_estimate() is not found, but I can't remember if that example is operative.

I was messing with the examples because I needed to add some tags to include the plots as thumbnails in the documentation... I will provide a more detailed explanation on Slack 😄

gaffarelj commented 2 years ago

All of the notebooks worked when they were pushed, but there were quite a lot of changes lately, so the estimation one may have been broken since. I start my notebooks directly from the terminal (running jupyter notebook). I tried running them from VS Code but that was more of a hassle.