search

emanjavacas / pie

A fully-fledge PyTorch package for Morphological Analysis, tailored to morphologically rich and historical languages.
MIT License
22 stars 10 forks source link

Error during tagging #51

Closed Malichot closed 4 years ago

Malichot commented 4 years ago

Dear all,

I'm encountering an error when running the tagger since yesterday evening whereas it was working two minutes before. I got the following error :

Traceback (most recent call last):

File "/home/malichot/env/bin/pie", line 8, in sys.exit(pie_cli())

File "/home/malichot/env/lib/python3.6/site-packages/click/core.py", line 764, in call return self.main(*args, **kwargs)

File "/home/malichot/env/lib/python3.6/site-packages/click/core.py", line 717, in main rv = self.invoke(ctx)

File "/home/malichot/env/lib/python3.6/site-packages/click/core.py", line 1137, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx))

File "/home/malichot/env/lib/python3.6/site-packages/click/core.py", line 956, in invoke return ctx.invoke(self.callback, **ctx.params)

File "/home/malichot/env/lib/python3.6/site-packages/click/core.py", line 555, in invoke return callback(*args, **kwargs)

File "/home/malichot/env/lib/python3.6/site-packages/pie/scripts/group.py", line 35, in tag model_spec, input_path, device, batch_size, lower, beam_width, use_beam)

File "/home/malichot/env/lib/python3.6/site-packages/pie/scripts/tag.py", line 11, in run tagger.add_model(model, *tasks)

File "/home/malichot/env/lib/python3.6/site-packages/pie/tagger.py", line 53, in add_model model = BaseModel.load(model_path)

File "/home/malichot/env/lib/python3.6/site-packages/pie/models/base_model.py", line 189, in load utils.get_gzip_from_tar(tar, 'label_encoder.zip'))

File "/home/malichot/env/lib/python3.6/site-packages/pie/utils.py", line 136, in get_gzip_from_tar return gzip.open(tar.extractfile(fpath)).read().decode().strip()

File "/usr/lib/python3.6/gzip.py", line 276, in read return self._buffer.read(size)

File "/usr/lib/python3.6/gzip.py", line 454, in read self._read_eof()

File "/usr/lib/python3.6/gzip.py", line 501, in _read_eof hex(self._crc)))

OSError: CRC check failed 0x18d5a8d4 != 0x4ab18f5b

Thanks in advance !

emanjavacas commented 4 years ago

Hey!

You got a CRC error while the model was being serialized. From what I can gather it seems the underlying zip file got corrupted during training and it failed to be serialized. Could you try to load the model as it is now and see if that's working? This is quite a rare case I haven't encountered before... Was the model already running for a long time?

On Wed, Mar 4, 2020 at 10:38 AM Malichot notifications@github.com wrote:

Dear all,

I'm encountering an error when running the tagger since yesterday evening whereas it was working two minutes before. I got the following error :

Traceback (most recent call last):

File "/home/malichot/env/bin/pie", line 8, in sys.exit(pie_cli())

File "/home/malichot/env/lib/python3.6/site-packages/click/core.py", line 764, in call return self.main(*args, **kwargs)

File "/home/malichot/env/lib/python3.6/site-packages/click/core.py", line 717, in main rv = self.invoke(ctx)

File "/home/malichot/env/lib/python3.6/site-packages/click/core.py", line 1137, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx))

File "/home/malichot/env/lib/python3.6/site-packages/click/core.py", line 956, in invoke return ctx.invoke(self.callback, **ctx.params)

File "/home/malichot/env/lib/python3.6/site-packages/click/core.py", line 555, in invoke return callback(*args, **kwargs)

File "/home/malichot/env/lib/python3.6/site-packages/pie/scripts/group.py", line 35, in tag model_spec, input_path, device, batch_size, lower, beam_width, use_beam)

File "/home/malichot/env/lib/python3.6/site-packages/pie/scripts/tag.py", line 11, in run tagger.add_model(model, *tasks)

File "/home/malichot/env/lib/python3.6/site-packages/pie/tagger.py", line 53, in add_model model = BaseModel.load(model_path)

File "/home/malichot/env/lib/python3.6/site-packages/pie/models/base_model.py", line 189, in load utils.get_gzip_from_tar(tar, 'label_encoder.zip'))

File "/home/malichot/env/lib/python3.6/site-packages/pie/utils.py", line 136, in get_gzip_from_tar return gzip.open(tar.extractfile(fpath)).read().decode().strip()

File "/usr/lib/python3.6/gzip.py", line 276, in read return self._buffer.read(size)

File "/usr/lib/python3.6/gzip.py", line 454, in read self._read_eof()

File "/usr/lib/python3.6/gzip.py", line 501, in _read_eof hex(self._crc)))

OSError: CRC check failed 0x18d5a8d4 != 0x4ab18f5b

Thanks in advance !

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/emanjavacas/pie/issues/51?email_source=notifications&email_token=ABPIPI6WWOG3LQ3QMM7LWITRFYOSNA5CNFSM4LBAVUY2YY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4ISJYVAQ, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABPIPI4HGVLTQDVAZI35A3TRFYOSNANCNFSM4LBAVUYQ .

-- Enrique Manjavacas

Malichot commented 4 years ago

Dear emanjavacas,

Thanks a lot for your prompt response ! It worked, seems the model got corrupted indeed, don't know why but it's working again greatly since I followed your advices.

Thanks again and have a good day.