OverflowError in transform.py `Randomizable` `set_random_state()` with seed

GenevieveBuckley commented 5 days ago

Fixes # https://github.com/teamtomo/membrain-seg/issues/70

Description

I'm using the membrain_seg package, which depends on MONAI, and have found that it consistently triggers an overflow error in MONAI

I've been able to fix the problem by ensuring both _seed and MAX_SEED are 64 bit integers, like this:

_seed = int(_seed) % MAX_SEED

and this has fixed the problem for me.

I notice that when the OverflowError occurs, _seed is a numpy uint32 bit value. I suspect this is the source of the problem, since MAX_SEED is by definition int64 https://github.com/Project-MONAI/MONAI/blob/08d572839ede5d087e49d4dfadba033e512528df/monai/utils/misc.py#L84-L85

For reference, here is a copy of the original bug report

membrain-seg version: current git main branch
Python version: tested with python 3.9 and python 3.11
Operating System: Zorin OS 16.3

Description

I followed the instructions in the installation guide, and all the packages appeared to install correctly with pip. But, when I tried to launch membrain from the command line, I got an error.

What I Did

git clone https://github.com/teamtomo/membrain-seg.git
cd membrain_seg
conda create -n membrainseg python=3.9 pip
conda activate membrainseg
pip install .
membrain

$ membrain

Traceback (most recent call last):
  File "/home/supervisor/miniforge3/envs/membrainseg/bin/membrain", line 5, in <module>
    from membrain_seg.segmentation.cli import cli
  File "/home/supervisor/miniforge3/envs/membrainseg/lib/python3.9/site-packages/membrain_seg/segmentation/cli/__init__.py", line 5, in <module>
    from .segment_cli import segment  # noqa: F401
  File "/home/supervisor/miniforge3/envs/membrainseg/lib/python3.9/site-packages/membrain_seg/segmentation/cli/segment_cli.py", line 12, in <module>
    from ..segment import segment as _segment
  File "/home/supervisor/miniforge3/envs/membrainseg/lib/python3.9/site-packages/membrain_seg/segmentation/segment.py", line 17, in <module>
    from .dataloading.memseg_augmentation import get_mirrored_img, get_prediction_transforms
  File "/home/supervisor/miniforge3/envs/membrainseg/lib/python3.9/site-packages/membrain_seg/segmentation/dataloading/memseg_augmentation.py", line 18, in <module>
    from membrain_seg.segmentation.dataloading.transforms import (
  File "/home/supervisor/miniforge3/envs/membrainseg/lib/python3.9/site-packages/membrain_seg/segmentation/dataloading/transforms.py", line 608, in <module>
    AxesShuffle = Compose(
  File "/home/supervisor/miniforge3/envs/membrainseg/lib/python3.9/site-packages/monai/transforms/compose.py", line 251, in __init__
    self.set_random_state(seed=get_seed())
  File "/home/supervisor/miniforge3/envs/membrainseg/lib/python3.9/site-packages/monai/transforms/compose.py", line 263, in set_random_state
    _transform.set_random_state(seed=self.R.randint(MAX_SEED, dtype="uint32"))
  File "/home/supervisor/miniforge3/envs/membrainseg/lib/python3.9/site-packages/monai/transforms/transform.py", line 207, in set_random_state
    _seed = _seed % MAX_SEED
OverflowError: Python integer 4294967296 out of bounds for uint32

Types of changes

[x] Non-breaking change (fix or new feature that would not break existing functionality).
[ ] New tests added to cover the changes.
[ ] Integration tests passed locally by running ./runtests.sh -f -u --net --coverage.
[ ] Quick tests passed locally by running ./runtests.sh --quick --unittests --disttests.

GenevieveBuckley commented 5 days ago

I'd like to add a test for this, however I'm having trouble reducing it to a minimal reproducible example, without involving the external membrain_seg package.

Here's where I think the extra test case should go: https://github.com/Project-MONAI/MONAI/blob/08d572839ede5d087e49d4dfadba033e512528df/tests/test_randomizable.py#L37-L43

What I've tried so far is hitting the error, entering the debugger, and then copy-pasting the exact value of _seed at that point into a new line in test_seed, eg: inst.set_random_state(seed=np.uint32(1126710810)) ... but running just this test case doesn't produce the same overflow error as before. I don't know if there are other types of hidden state complicating things.

Do you have any suggestions?

ericspod commented 5 days ago

Hi @GenevieveBuckley This is cause by our incompatibility with Numpy 2.0. If you install a 1.* version this will go away. We're working on a fix now as a stopgap. I also had a previous PR on the same subject but if you wanted to work on other fixes we'll need these eventually. Thanks!

Project-MONAI / MONAI