google-deepmind / gemma

Open weights LLM from Google DeepMind.
http://ai.google.dev/gemma
Apache License 2.0
2.41k stars 305 forks source link

7b errors on deserilization #6

Closed lgyStoic closed 7 months ago

lgyStoic commented 7 months ago

in m1 mac os..fellowed the read me ,and passed pytest but when run

python3.11 examples/sampling.py  --path_checkpoint=/Users/garryling/Downloads/archive_test/7b/ --path_tokenizer=/Users/garryling/Downloads/archive_test/tokenizer.model

cause error.

Traceback (most recent call last):
  File "/Users/garryling/workspace/gemma/examples/sampling.py", line 112, in <module>
    app.run(main)
  File "/Users/garryling/gemma-demo/lib/python3.11/site-packages/absl/app.py", line 308, in run
    _run_main(main, args)
  File "/Users/garryling/gemma-demo/lib/python3.11/site-packages/absl/app.py", line 254, in _run_main
    sys.exit(main(argv))
             ^^^^^^^^^^
  File "/Users/garryling/workspace/gemma/examples/sampling.py", line 102, in main
    _load_and_sample(
  File "/Users/garryling/workspace/gemma/examples/sampling.py", line 73, in _load_and_sample
    parameters = params_lib.load_and_format_params(path_checkpoint)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/garryling/gemma-demo/lib/python3.11/site-packages/gemma/params.py", line 29, in load_and_format_params
    params = load_params(path)
             ^^^^^^^^^^^^^^^^^
  File "/Users/garryling/gemma-demo/lib/python3.11/site-packages/gemma/params.py", line 40, in load_params
    params = checkpointer.restore(path)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/garryling/gemma-demo/lib/python3.11/site-packages/orbax/checkpoint/checkpointer.py", line 166, in restore
    restored = self._handler.restore(directory, args=ckpt_args)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/garryling/gemma-demo/lib/python3.11/site-packages/orbax/checkpoint/pytree_checkpoint_handler.py", line 1073, in restore
    restored_item = asyncio.run(
                    ^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.11/3.11.6_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/runners.py", line 190, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.11/3.11.6_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.11/3.11.6_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/base_events.py", line 653, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/Users/garryling/gemma-demo/lib/python3.11/site-packages/orbax/checkpoint/pytree_checkpoint_handler.py", line 903, in _maybe_deserialize
    deserialized_batches += await asyncio.gather(*deserialized_batches_ops)
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/garryling/gemma-demo/lib/python3.11/site-packages/orbax/checkpoint/type_handlers.py", line 1134, in deserialize
    ret = await asyncio.gather(*read_ops)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.11/3.11.6_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/tasks.py", line 694, in _wrap_awaitable
    return (yield from awaitable.__await__())
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: FAILED_PRECONDITION: Error reading "transformer/layer_26/pre_attention_norm.scale/0" in OCDBT database at local file "/Users/garryling/Downloads/archive_test/7b/": Error reading file: /Users/garryling/Downloads/archive_test/7b/ocdbt.process_0/d/400042b385de57e9fbec46e8f38bf642 [OS error: Invalid argument] [source locations='tensorstore/kvstore/file/file_key_value_store.cc:330\ntensorstore/kvstore/kvstore.cc:377']