Python 3.10 support

[fzimmermann89]

Change our code to support python3.10 again --> colab!

Import all typing stuff from typing_extension which backports typing features and in current python versions, reexports from typing.

Replace tuple unpacking in typehints by Unpack. This look a bit less nice but might be an ok trade-off. We can revert this commit if we want to drop python3.10 support again

Add a docker container for python 3.10

Refactor docker files: installing from github via clone will created chicken and egg problem as main was not installable in python 3.10. docker containers are now split in steps that might be cached.

Biggest difference is, our dependencies in the docker container will only update if we touch the pyproject.toml. If we want to, we can also update them on updates to VERSION, for example.

Closes #456

[fzimmermann89]

Should be merged after #460

[github-actions[bot]]

Coverage Report

File	Stmts	Miss	Cover	Missing
src/mrpro/algorithms/csm
inati.py	24	1	96%	44
walsh.py	16	1	94%	34
src/mrpro/algorithms/dcf
dcf_voronoi.py	53	4	92%	15, 48–49, 76
src/mrpro/algorithms/optimizers
adam.py	20	1	95%	69
src/mrpro/algorithms/reconstruction
DirectReconstruction.py	28	16	43%	51–71, 85
IterativeSENSEReconstruction.py	42	23	45%	77–78, 88–98, 113–124, 138–149
Reconstruction.py	52	24	54%	42, 54–56, 80–87, 104–115
src/mrpro/data
AcqInfo.py	128	2	98%	174, 214
CsmData.py	29	3	90%	15, 82–84
DcfData.py	45	8	82%	18, 66, 78–83
IData.py	67	9	87%	119, 125, 129, 159–167
IHeader.py	75	7	91%	75, 109, 127–131
KHeader.py	165	17	90%	25, 127–131, 158, 208, 219, 226–227, 230, 237, 276–287
KNoise.py	31	15	52%	39–52, 56–61
KTrajectory.py	69	5	93%	178–182
MoveDataMixin.py	127	14	89%	15, 110, 126, 140–142, 203, 266, 280, 359, 379–380, 397–398
QData.py	39	7	82%	42, 65–73
Rotation.py	448	22	95%	97, 278, 363, 365, 392, 447, 452, 455, 470, 487, 492, 635, 640, 643, 659, 663, 737, 739, 747–748, 988, 1070
SpatialDimension.py	47	2	96%	65, 104
TrajectoryDescription.py	14	1	93%	23
acq_filters.py	10	1	90%	47
src/mrpro/data/_kdata
KData.py	105	16	85%	108–109, 118, 126, 180–181, 216, 221–222, 241–252
KDataRemoveOsMixin.py	29	2	93%	44, 46
KDataSelectMixin.py	21	2	90%	48, 64
KDataSplitMixin.py	49	3	94%	51, 81, 90
src/mrpro/data/traj_calculators
KTrajectoryCalculator.py	25	2	92%	23, 45
KTrajectoryIsmrmrd.py	13	2	85%	41, 50
KTrajectoryPulseq.py	29	1	97%	54
src/mrpro/operators
CartesianSamplingOp.py	50	9	82%	49–50, 55–56, 61–62, 88, 91, 114
ConstraintsOp.py	60	2	97%	46, 48
EndomorphOperator.py	65	2	97%	228, 234
FiniteDifferenceOp.py	27	2	93%	40, 105
FourierOp.py	77	1	99%	131
Functional.py	71	5	93%	20–22, 117, 119
GridSamplingOp.py	136	9	93%	72–73, 82–83, 90–91, 94, 96, 98
LinearOperator.py	161	12	93%	55, 91, 190, 220, 261, 270, 278, 287, 295, 306, 404, 409
MultiIdentityOp.py	13	2	85%	43, 48
Operator.py	78	2	97%	25, 74
ProximableFunctionalSeparableSum.py	39	3	92%	50, 103, 110
SliceProjectionOp.py	173	8	95%	44, 61, 63, 69, 206, 227, 260, 300
WaveletOp.py	120	5	96%	152, 170, 205, 210, 233
ZeroPadOp.py	16	1	94%	30
src/mrpro/utils
filters.py	62	2	97%	44, 49
modify_acq_info.py	17	1	94%	12
slice_profiles.py	46	6	87%	20, 36, 113–116, 149
sliding_window.py	34	1	97%	34
split_idx.py	10	2	80%	43, 47
summarize_tensorvalues.py	11	9	18%	20–29
typing.py	14	8	43%	8–18
zero_pad_or_crop.py	31	6	81%	26, 30, 54, 57, 60, 63
TOTAL	4073	309	92%

Tests	Skipped	Failures	Errors	Time
1872	0 :zzz:	0 :x:	0 :fire:	1m 41s :stopwatch:

[github-actions[bot]]

:books: Documentation

:file_folder: Download as zip :mag: View online

[fzimmermann89]

@ckolbPTB did you have a docker file for python3.10? I think we need stupid if case in the install script:

[fzimmermann89]

I am getting hit by a chicken and egg problem: During build of the docker image, mrpro cannot be installed with python 3.10 as we currently install from github main (and there the min version is python 3.11) This results in missing requirements later on This results in the tests not passing. So I have to split this into two PRs.

One PR will enable 3.10 compatibility Second PR will enable tests for 3.10

...or we would have to fix the docker logic, but this would also be a second PR as I want to keep this one only python3.10 related to be able to just revert it later on

Any other suggestions? @ckolbPTB @schuenke @JoHa0811 ?

[fzimmermann89]

This should be merged as two commits to make undo of the python3.10 changes easier in the future.

[schuenke]

A few questions out of curiosity:

1. Any idea why https://github.com/PTB-MR/mrpro/actions/runs/11451086815 is still running after ~6h ?
2. Any idea why for the build of py310 and py311 the "Cached" value is ~7%, but for py312 ~28%? see here
3. In general, splitting the docker build is a good idea imo. However, it seems like it's not resulting in a faster build yet, no? Would this change with the next run?

[ckolbPTB]

Seems to me a lot of work for something which is outdated and we will have to revert soon again. Would it not make more sense to try to overcome the problem of python 3.10 in colab? We are not the only ones complaining about this. Are there no options to upgrade python or run one of our containers in colab (maybe something like this: https://github.com/drengskapur/docker-in-colab)?

[fzimmermann89]

Seems to me a lot of work for something which is outdated and we will have to revert soon again. Would it not make more sense to try to overcome the problem of python 3.10 in colab? We are not the only ones complaining about this. Are there no options to upgrade python or run one of our containers in colab (maybe something like this: https://github.com/drengskapur/docker-in-colab)?

There is no good way to use notebooks in a docker at colab.

When we moved to python 3.11, we wanted to get "Self" to work and I did not know about typing_extensions...

[fzimmermann89]

A few questions out of curiosity:

1. Any idea why https://github.com/PTB-MR/mrpro/actions/runs/11451086815 is still running after ~6h ?

I have to manually kill them or wait till they time out. There is a bug in pip that installing packages from '/' does not work and freezes. In docklerfiles, it is common to COPY to '/', run and remove afterwards. Took me a few tries to realize this is the issue.

2. Any idea why for the build of py310 and py311 the "Cached" value is ~7%, but for py312 ~28%? see here

No idea. One possible difference is how far previous runs for this task got.

3. In general, splitting the docker build is a good idea imo. However, it seems like it's not resulting in a faster build yet, no? Would this change with the next run?

It should. Here the pyproject.toml changed, which it will not do most of the time..

[fzimmermann89]

[fzimmermann89]

2. Any idea why for the build of py310 and py311 the "Cached" value is ~7%, but for py312 ~28%? see here

No idea. One possible difference is how far previous runs for this task got.

3. In general, splitting the docker build is a good idea imo. However, it seems like it's not resulting in a faster build yet, no? Would this change with the next run?

It should. Here the pyproject.toml changed, which it will not do most of the time..

Lets try adding a scope for each docker image (py10,py11,py12): https://docs.docker.com/build/cache/backends/gha/#scope

Scope is a key used to identify the cache object. By default, it is set to buildkit. If you build multiple images, each build will >overwrite the cache of the previous, leaving only the final cache.

To preserve the cache for multiple builds, you can specify this scope attribute with a specific name. In the following example, the >cache is set to the image name, to ensure each image gets its own cache:

GitHub's cache access restrictions, still apply. Only the cache for the current branch, the base branch and the default branch is >accessible by a workflow.

[fzimmermann89]

This now also changed when dependencies are updated to newest versions. We do no longer update on each push to main all dependencies. Our normal PRs should not fail because a dependency is updated, this was a bit annoying in the past.

Instead, dependencies are updated if either the pyproject.toml is changed, or the VERSION is changed.

Long term, it would be great to get a dependabot-like action checking for requirements going.