ValueError: Unpack failed: incomplete input

[jchia]

Got this error on Ubuntu 20.04 with Python 3.8.10.

$ python image_from_text.py --text 'alien life' --seed 7                                                                                 ✔  14:15:01 
Namespace(image_path='generated', image_token_count=256, mega=False, seed=7, text='alien life', torch=False)
parsing metadata from ./pretrained/dalle_bart_mini
tokenizing text
['Ġalien']
['Ġlife']
text tokens [0, 8925, 742, 2]
Traceback (most recent call last):
  File "image_from_text.py", line 44, in <module>
    image = generate_image_from_text(
  File "/home/jchia/gh/min-dalle/min_dalle/generate_image.py", line 54, in generate_image_from_text
    params_dalle_bart = load_dalle_bart_flax_params(model_path)
  File "/home/jchia/gh/min-dalle/min_dalle/load_params.py", line 44, in load_dalle_bart_flax_params
    params = serialization.msgpack_restore(f.read())
  File "/home/jchia/venv/pt/lib/python3.8/site-packages/flax/serialization.py", line 350, in msgpack_restore
    state_dict = msgpack.unpackb(
  File "msgpack/_unpacker.pyx", line 205, in msgpack._cmsgpack.unpackb
ValueError: Unpack failed: incomplete input

pretrained/dalle_bart_mini/flax_model.msgpack was somehow not deserializing properly.

[jwenjian]

same, macos 12.2, python 3.8

[georgemorgan]

+1

[kuprel]

I think this is a duplicate of #1

[jchia]

@kuprel Not the same file. This one is for dalle_bart_mini and being downloaded with wandb.

[kuprel]

does the suggestion in #31 work?

[jchia]

There are 2 problems causing undetected bad downloads:

1. wandb artifact get fails silently. I did the wandb artifact get download for mini twice, each time obtaining 0 exit code (indicating success) but got different sizes for flax_model.msgpack. Probably network errors were being silently ignored.
2. The lack of set -e in setup.sh. This was added in 63a65372b0e50fa09633d900cd2ef3e1a54f2679 but later removed, perhaps accidentally. This means if a big download fails resulting in partially-downloaded files, set -e helps failure detection if the downloading program fails as it should (not wandb in this case).

The problem went away after I manually downloaded flax_model.msgpack with a web browser.

I suspect a lot of people getting msgpack deserializing errors are getting them from silently-failed downloads caused by the above problems, so fixing the silent failure would prevent further similar issues from being reported and wasting time. I'm not familiar with wandb artifact get but this kind of silent failure is surprising.

[kuprel]

I just added back set -e

[jchia]

I suppose the main issue now is the silent failure in wandb artifact get. I wonder if others can repro.

[kuprel]

Closing this since the flax model was moved