a problem of ERROR: RuntimeError: unexpected EOF, expected 39359 more bytes. The file might be corrupted

[puallee]

there was a problem when I tried to start processing of the Nii image or dicom image

Runlist: ['C:/Users/V/Desktop/imagesTr/004_0000.nii.gz']. 1 volumes detected.

Will save outputs in C:/Users/V/Desktop/pred.

Click start processing to start.

ERROR: RuntimeError: unexpected EOF, expected 39359 more bytes. The file might be corrupted.

Aborting processing! This was most likely an error regarding the data input or missing weights before doing pip install.

Feel free to create an issue in our GitHub.

Closing worker thread...

Done.

Runlist: ['C:/Users/V/Desktop/imagesTr/059_0000.nii.gz']. 1 volumes detected.

Will save outputs in C:/Users/V/Desktop/pred.

Click start processing to start.

ERROR: RuntimeError: unexpected EOF, expected 39359 more bytes. The file might be corrupted.

Aborting processing! This was most likely an error regarding the data input or missing weights before doing pip install.

Feel free to create an issue in our GitHub.

Closing worker thread...

Done.

Please load a folder or image before starting processing.

Please load a folder or image before starting processing.

Please load a folder or image before starting processing.

Runlist: ['G:/DICOM/dicom/dicom/1.2.276.0.7230010.3.1.4.3507096321.30414.1535700143.988584.dcm']. 1 volumes detected.

Will save outputs in C:/Users/V/Desktop/pred.

Click start processing to start.

ERROR: RuntimeError: unexpected EOF, expected 39359 more bytes. The file might be corrupted.

Aborting processing! This was most likely an error regarding the data input or missing weights before doing pip install.

Feel free to create an issue in our GitHub.

Closing worker thread...

Done.

[dscarmo]

Hi! Can you please answer these questions and I can try and help you.

Can you share what is the full output in the terminal where you ran the "medpseg" command?

Have you downloaded the network weights? They might be corrupted, I suggest downloading them again.

The DICOM/Nift files open in software such as ITK-Snap or 3D Slicer without errors?

[puallee]

Yes， the full output in the terminal is： Click start processing to start.

ERROR: RuntimeError: unexpected EOF, expected 39359 more bytes. The file might be corrupted.

Aborting processing! This was most likely an error regarding the data input or missing weights before doing pip install.

Feel free to create an issue in our GitHub.

ERROR: RuntimeError: unexpected EOF, expected 39359 more bytes. The file might be corrupted.

Aborting processing! This was most likely an error regarding the data input or missing weights before doing pip install.

Feel free to create an issue in our GitHub.

Closing worker thread...

Done.

Closing worker thread...

[puallee]

I downloaded the network weights again， but I still have this problem.

[puallee]

yes, all CTs can open ITK-Snap or 3D Slicer without errors

[puallee]

PS D:\medpseg> medpseg Could not find platform dependent libraries Running MEDPSeg with GPU support. CUDA available? False. C:\Users\V\AppData\Local\Programs\Python\Python311\Lib\site-packages\lightning_fabric\utilities\cloud_io.py:57: You are using torch.load with weights_only=False (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for weights_only will be flipped to True. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via torch.serialization.add_safe_globals. We recommend you start setting weights_only=True for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature. WARNING: default expand_bifpn changed to upsample_sum after many ablations and literature support. Using upsample for bifpn result expansion and upsample 2 expansion of convnext features Traceback (most recent call last): File "C:\Users\V\AppData\Local\Programs\Python\Python311\Lib\site-packages\medpseg\pipeline.py", line 118, in pipeline poly_model = PolySegmentationPipeline(weight=poly_weight, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\V\AppData\Local\Programs\Python\Python311\Lib\site-packages\medpseg\poly_pipeline.py", line 212, in init self.model = PolySeg2DModule.load_from_checkpoint(weight, map_location="cpu").eval() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\V\AppData\Local\Programs\Python\Python311\Lib\site-packages\pytorch_lightning\utilities\model_helpers.py", line 125, in wrapper return self.method(cls, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\V\AppData\Local\Programs\Python\Python311\Lib\site-packages\pytorch_lightning\core\module.py", line 1586, in load_from_checkpoint loaded = _load_from_checkpoint( ^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\V\AppData\Local\Programs\Python\Python311\Lib\site-packages\pytorch_lightning\core\saving.py", line 91, in _load_from_checkpoint model = _load_state(cls, checkpoint, strict=strict, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\V\AppData\Local\Programs\Python\Python311\Lib\site-packages\pytorch_lightning\core\saving.py", line 165, in _load_state obj = instantiator(cls, _cls_kwargs) if instantiator else cls(cls_kwargs) ^^^^^^^^^^^^^^^^^^ File "C:\Users\V\AppData\Local\Programs\Python\Python311\Lib\site-packages\medpseg\poly_seg_2d_module.py", line 126, in init self.model = MEDSeg(self.hparams.nin, self.hparams.seg_nout, apply_sigmoid=False, backbone=self.backbone, expand_bifpn=self.expand_bifpn, pretrained=not self.scratch, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\V\AppData\Local\Programs\Python\Python311\Lib\site-packages\medpseg\architecture.py", line 64, in init self.model = EfficientDetForSemanticSegmentation(num_classes=nout, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\V\AppData\Local\Programs\Python\Python311\Lib\site-packages\medpseg\edet\modeling_efficientdet.py", line 224, in init self.backbone_net = EfficientNet(self.backbone_compound_coef[self.compound_coef], load_weights) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\V\AppData\Local\Programs\Python\Python311\Lib\site-packages\medpseg\edet\efficientdet\model.py", line 563, in init model = EffNet.from_pretrained(f'efficientnet-b{compound_coef}') ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\V\AppData\Local\Programs\Python\Python311\Lib\site-packages\efficientnet_pytorch\model.py", line 378, in from_pretrained load_pretrained_weights(model, model_name, weights_path=weights_path, File "C:\Users\V\AppData\Local\Programs\Python\Python311\Lib\site-packages\efficientnet_pytorch\utils.py", line 602, in load_pretrained_weights state_dict = model_zoo.load_url(url_map[model_name]) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\V\AppData\Local\Programs\Python\Python311\Lib\site-packages\torch\hub.py", line 769, in load_state_dict_from_url return torch.load(cached_file, map_location=map_location, weights_only=weights_only) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\V\AppData\Local\Programs\Python\Python311\Lib\site-packages\torch\serialization.py", line 1114, in load return _legacy_load( ^^^^^^^^^^^^^ File "C:\Users\V\AppData\Local\Programs\Python\Python311\Lib\site-packages\torch\serialization.py", line 1357, in _legacy_load typed_storage._untyped_storage._set_from_file( RuntimeError: unexpected EOF, expected 39359 more bytes. The file might be corrupted.

[dscarmo]

Thanks for the outputs, It looks like you are not using a separate environment for medpseg, I recommend you create a new environment and install the requirements listed in requirements.txt. If the model weights are not corrupted in download, I suspect the model weights are not loading correctly due to library versions.

What version is your PyTorch and PyTorch Lightning?

Could you install miniconda and create an specific environment as detailed on the repository readme, installing torch and subsequently installing the requirements with pip install -r requirements.txt?

[dscarmo]

I have provided an alternative link to download the weights, in case they are being corrupted when downloaded to your specific location. I have tested these today and they should work!

https://drive.google.com/file/d/1biSkeLhDNpxUA6xqOVEOhpaElJ1Y8mWt/view?usp=sharing

[puallee]

torch 2.4.0 torchinfo 1.8.0 torchmetrics 1.4.0.post0 torchvision 0.19.0

[puallee]

My python version is 3.11

[puallee]

pytools 2024.1.11 pytorch-lightning 2.3.3

[puallee]

SimpleITK==2.2.1

[dscarmo]

@puallee Hello! were you able to solve the problem using the alternative link to the weights I provided?

[puallee]

sorry，the same error， I think the weight maybe not the reason

[dscarmo]

@puallee were you able to solve this? I suspect your EfficientNet weights might be corrupted given your previous error message. Have you tried what I mentioned before: "Could you install miniconda and create an specific environment as detailed on the repository readme, installing torch and subsequently installing the requirements with pip install -r requirements.txt?"?

[dscarmo]

@puallee were you able to solve this issue? I suspect the EfficientNet weights being corrupt in your torch cache, maybe try what I mentioned in the last comment.