search

lucianopaz / compress_pickle

Standard python pickle, thinly wrapped with standard compression libraries
MIT License
42 stars 11 forks source link

dumping fails with the .lzma extension #32

Closed jerabaul29 closed 3 years ago

jerabaul29 commented 3 years ago

I have a piece of code that looks like:

import compress_pickle as cpkl

def dump_object(filename, to_dump):
    """A minor helper to dump the object into filename."""
    os.makedirs(os.path.dirname(filename), exist_ok=True)
    with open(filename, "bw") as fh:
        cpkl.dump(to_dump, fh)

When running it, I get the error:

  File "script_logging.py", line 59, in dump_object
    cpkl.dump(to_dump, fh)
  File "/home/jrmet/.local/lib/python3.8/site-packages/compress_pickle/compress_pickle.py", line 96, in dump
    compresser = instantiate_compresser(
  File "/home/jrmet/.local/lib/python3.8/site-packages/compress_pickle/utils.py", line 112, in instantiate_compresser
    compression = _infer_compression_from_path(_path)
UnboundLocalError: local variable '_path' referenced before assignment

Is this a user or package problem? :)

jerabaul29 commented 3 years ago

PS: the filename is actually PATH/SOME_NAME.pkl.lzma; is it possible the .pkl.lzma is what confuses compress_pickle?

lucianopaz commented 3 years ago

@jerabaul29, thanks for reporting. The problem is that you can only infer the compression when you supply a string as the path. You are supplying the file object fh. Due to a minor bug, the wrong error message is raised. I'll fix it when I get some time next week.

lucianopaz commented 3 years ago

This was fixed by #35