Layout-Parser / layout-parser

A Unified Toolkit for Deep Learning Based Document Image Analysis
https://layout-parser.github.io/
Apache License 2.0
4.82k stars 464 forks source link

_pickle.UnpicklingError: invalid load key, '<'. #180

Open cullen122 opened 1 year ago

cullen122 commented 1 year ago

I am facing error while building the detectron2 base model from the example given in github and I receive the following error. Couldnt find any solution. Is there anyone facing this issue as well?

PLATFORM : Windows 10

model_final.pth: 942kB [00:02, 436kB/s] Traceback (most recent call last): File "C:\Users\Alexander\layoutTest\main.py", line 3, in model = lp.Detectron2LayoutModel( File "C:\Users\Alexander\anaconda3\envs\detectron_env\lib\site-packages\layoutparser\models\detectron2\layoutmodel.py", line 119, in init self._create_model() File "C:\Users\Alexander\anaconda3\envs\detectron_env\lib\site-packages\layoutparser\models\detectron2\layoutmodel.py", line 122, in _create_model self.model = detectron2.engine.DefaultPredictor(self.cfg) File "C:\Users\Alexander\layoutTest\detectron2\detectron2\engine\defaults.py", line 288, in init checkpointer.load(cfg.MODEL.WEIGHTS) File "C:\Users\Alexander\layoutTest\detectron2\detectron2\checkpoint\detection_checkpoint.py", line 62, in load ret = super().load(path, *args, kwargs) File "C:\Users\Alexander\anaconda3\envs\detectron_env\lib\site-packages\fvcore\common\checkpoint.py", line 155, in load checkpoint = self._load_file(path) File "C:\Users\Alexander\layoutTest\detectron2\detectron2\checkpoint\detection_checkpoint.py", line 99, in _load_file loaded = self._torch_load(filename) File "C:\Users\Alexander\layoutTest\detectron2\detectron2\checkpoint\detection_checkpoint.py", line 114, in _torch_load return super()._load_file(f) File "C:\Users\Alexander\anaconda3\envs\detectron_env\lib\site-packages\fvcore\common\checkpoint.py", line 252, in _load_file return torch.load(f, map_location=torch.device("cpu")) File "C:\Users\Alexander\anaconda3\envs\detectron_env\lib\site-packages\torch\serialization.py", line 815, in load return _legacy_load(opened_file, map_location, pickle_module, pickle_load_args) File "C:\Users\Alexander\anaconda3\envs\detectron_env\lib\site-packages\torch\serialization.py", line 1033, in _legacy_load magic_number = pickle_module.load(f, **pickle_load_args) _pickle.UnpicklingError: invalid load key, '<'.

Process finished with exit code 1

ashrewdmint commented 1 year ago

I had this issue, but I fixed it by using the version of detectron2 specified in the dev-requirements.txt file: git+https://github.com/facebookresearch/detectron2.git@v0.4#egg=detectron2

This version differs with the version specified in the installation.md file, which uses v0.5. Fortunately, using that version also works for me.

I'm on linux and not using anaconda, so hopefully this fix will still work for you.

Balthus1989 commented 1 year ago

Hey @ashrewdmint , despite the fact that in dev-requirements.txt is present the fix, I still got the _pickle.UnpicklingError

pedrohavay commented 12 months ago

Hey @Balthus1989 and @ashrewdmint ! This has nothing to do with the version of detectron2 or torch. Basically, the error occurs because pickle does not load the model file due to its incorrect format, which in this case is an HTML returned by DropBox in the download attempt made based on the configuration passed to layoutparser like, for example, lp://PubLayNet/mask_rcnn_X_101_32x8d_FPN_3x/config.

I recommend you download the desired configuration file at this link and change the Dropbox URL to the address of the model on your computer. And then provide the address of your config.yaml file when initializing the layoutparser.

Ex.: model = lp.Detectron2LayoutModel( "/Users/[youuser]/Downloads/config.yaml", label_map={ 0: "Text", 1: "Title", 2: "List", 3: "Table", 4: "Figure", })

junxi-liu commented 11 months ago

Hey @pedrohavay , unfortunately this method still doesn't work for me.

Edit: installing detectron2==0.5 solved the issue. Using detectron2==0.6 always fails.

fabmeyer commented 10 months ago

@junxi-liu is it possible to install an older version from source? like python -m pip install 'git+https://github.com/facebookresearch/detectron2.git' but with version 0.5 instead of the latest version? Else it doesn't match my torch version.

Edit: I found it out by myself. Download the zip file (https://github.com/facebookresearch/detectron2/releases), unzip it and then run python -m pip install detectron2-0.5/ in the given directory.

BiboyQG commented 6 months ago

@junxi-liu is it possible to install an older version from source? like python -m pip install 'git+https://github.com/facebookresearch/detectron2.git' but with version 0.5 instead of the latest version? Else it doesn't match my torch version.

Edit: I found it out by myself. Download the zip file (https://github.com/facebookresearch/detectron2/releases), unzip it and then run python -m pip install detectron2-0.5/ in the given directory.

Really appreciate the help!