How to evaluate a model

[HariG123]

Hi I am trying to run allennlp evaluate command for scierc dataset. However the following error came:

Traceback (most recent call last):
  File "/usr/local/bin/allennlp", line 8, in <module>
    sys.exit(run())
  File "/usr/local/lib/python3.7/dist-packages/allennlp/__main__.py", line 34, in run
    main(prog="allennlp")
  File "/usr/local/lib/python3.7/dist-packages/allennlp/commands/__init__.py", line 92, in main
    args.func(args)
  File "/usr/local/lib/python3.7/dist-packages/allennlp/commands/evaluate.py", line 117, in evaluate_from_args
    overrides=args.overrides,
  File "/usr/local/lib/python3.7/dist-packages/allennlp/models/archival.py", line 168, in load_archive
    archive.extractall(tempdir)
  File "/usr/lib/python3.7/tarfile.py", line 2002, in extractall
    numeric_owner=numeric_owner)
  File "/usr/lib/python3.7/tarfile.py", line 2044, in extract
    numeric_owner=numeric_owner)
  File "/usr/lib/python3.7/tarfile.py", line 2114, in _extract_member
    self.makefile(tarinfo, targetpath)
  File "/usr/lib/python3.7/tarfile.py", line 2163, in makefile
    copyfileobj(source, target, tarinfo.size, ReadError, bufsize)
  File "/usr/lib/python3.7/tarfile.py", line 247, in copyfileobj
    buf = src.read(bufsize)
  File "/usr/lib/python3.7/gzip.py", line 287, in read
    return self._buffer.read(size)
  File "/usr/lib/python3.7/_compression.py", line 68, in readinto
    data = self.read(len(byte_view))
  File "/usr/lib/python3.7/gzip.py", line 482, in read
    uncompress = self._decompressor.decompress(buf, size)
zlib.error: Error -3 while decompressing data: invalid code lengths set

The code than i ran on googel colab was:

allennlp evaluate "dygiepp/models/scierc/model.tar.gz" \
                  "dygiepp/scripts/data/scierc/normalized_data/json/test.json" \
               --output-file "dygiepp/models/scierc/metrics_test.json" \
               --cuda-device -1  \
               --include-package dygie 

What can I do to resolve this ?Thanks.

[dwadden]

I'll take a look this weekend.

[HariG123]

Ok, thanks.

[dwadden]

It looks like you're getting an error associated with decompressing the model archive? I've never run into this. Can you confirm that this isn't Colab-related by running the same command in a normal shell environment?

[dwadden]

Closing due to inactivity.