Add jupytext

[tmieslinger]

Following @reredl's request for easier reviews, this PR adds jupytext to the repo which makes it easier to review changes in ipython notebooks. Jupytext will give you the plain text changes from notebook cells such that git diff will result in reasonable and readable info again :)

Biggest change is that we don't save .ipynb files, but markdown .md files. jupytercan still open them as notebook and you can work as usual. In detail, this PR includes the following changes:

• update the environment.yaml files to include jupytext
• clean up the environment specs by moving them to top level and deleting the outdated and duplicate requirements.txt
• convert previous segmentation_template.ipynb to segmentation_template.md
• adjustment the README.md to mirror the new workflow with jupytext

Note: if you have segmentation notebooks in an open (draft) PR, you can easily convert them to markdown by jupytext --to markdown your_notebook.ipynb in a new commit and before merging them into the main branch.

[tmieslinger]

While testing this new feature, I found out that problems arise when python version 3.13 is used. I would therefore suggest to change the python dependency to python==3.12 in which case everything ran smoothly.

What problem do you encounter? Could you be more specific?

[Lucalino]

Yes, the error read:

AttributeError: module 'numpy' has no attribute 'float'. np.float was a deprecated alias for the builtin float. To avoid this error in existing code, use float by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use np.float64 here. The aliases was originally deprecated in NumPy 1.20; for more details and guidance see the original release note at: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations

[d70-t]

Hmm, interesting... How did you install the environment? I'd like to reproduce this here.

I tried installing it freshly on a newly created ubuntu virtual machine using

micromamba env create -f environment.yaml

when I do this, micromamba resolves the dependencies such that Python version 3.12.7 is installed.

Note that my micromamba would indeed prefer Python 3.13.0, e.g. when I try to create an empty python environment using

micromamba env create -n foo python

I also tried to forceibly install a flight_segmentation environment with Python 3.13:

micromamba env create python=3.13.0 -f environment.yaml

... but that resulted in an error:

python 3.13.0 conflicts with any installable versions previously reported;

If I get the messages right, pandas (which is an indirect requirement) doesn't like Python 3.13 yet and thus won't let me install it.

---

Sidenode: My environment resolves to numpy 2.1.2, which as of now, is only the second-newest numpy release, but the newest one available on conda-forge.

[Lucalino]

I used mamba env create -f environment.yaml -y which installed python 3.13.0 and numpy 2.1.2 (and in case it matters, the installed pandas version is 2.2.3).

[tmieslinger]

I still cannot reproduce your error... when do you run into AttributeError: module 'numpy' has no attribute 'float'?

[Lucalino]

My workflow was as follows:

1. Create environment from scratch using mamba env -f environment.yaml -y
2. Activate the environment
3. Run Jupyter lab --> open segmentation_template.md as a notebook
4. Execute the first code cell which loads the packages: This is where the error occurs.

And if I repeat the exact same process with python>=3.12 set to python==3.12 in the environment.yaml file, then the error disappears and everything works well.

But regardless of the specific problem at hand: Is there any benefit to keeping the requirement for the python version ambiguous?

[tmieslinger]

I'd say it's ususally not a good idea to limit/fix any package versions if not for a good reason. conda/mamba solve for the current best dependency combination. If some packages improve such that they can work with a more recent python version in the future, we would also want to get this update.

[Lucalino]

Perhaps we can have a look at the problem together on Monday next week and figure out what is different between your and my local environment. Could the reason be that I use mamba and you micromamba?

[tmieslinger]

I'd suggest to fix teh python version to 3.12 as this is sufficient and the development of the current package is very limited in time. We don't aim to keep it up to date for several months/years, but rather create a version that we can use within the next 2-3 months to create segmentation files. If we redo something or add/process other (platform) segmentation files later on, we might consider solving the dependency issue at it's root