ValueError: Error opening "zarr" driver

[DelinQu]

Hi, Thanks for sharing your codes. I'd tried to read a buffer with the provided API flashback.vault.Vault but encountered the following error. I'm sure all the data exists under my DATA directory, but catch a ValueError exception. Could you provide any suggestions? Many thanks :)

• code

  
  from flashbax.vault import Vault
  Vault(rel_dir="../DATA/og_marl/smac_v1", vault_name="3m.vlt", vault_uid="Good")

errors

Loading vault found at ~/zhangyang2/Projects/CausalMAD/og-marl-custom/../DATA/og_marl/smac_v1/3m.vlt/Good *** ValueError: Error opening "zarr" driver: Error reading "actions./.zarray" in OCDBT database at local file "~/zhangyang2/Projects/CausalMAD/og-marl-custom/../DATA/og_marl/smac_v1/3m.vlt/Good/": Error reading local file "~/zhangyang2/Projects/CausalMAD/og-marl-custom/../DATA/og_marl/smac_v1/3m.vlt/Good/manifest.ocdbt": Invalid key: "~/zhangyang2/Projects/CausalMAD/og-marl-custom/../DATA/og_marl/smac_v1/3m.vlt/Good/manifest.ocdbt" [tensorstore_spec='{\"context\":{\"cache_pool\":{},\"data_copy_concurrency\":{},\"file_io_concurrency\":{},\"file_io_sync\":true,\"ocdbt_coordinator\":{}},\"driver\":\"zarr\",\"kvstore\":{\"base\":{\"driver\":\"file\",\"path\":\"~/zhangyang2/Projects/CausalMAD/og-marl-custom/../DATA/og_marl/smac_v1/3m.vlt/Good/\"},\"driver\":\"ocdbt\",\"path\":\"actions./\"}}'] [source locations='tensorstore/kvstore/file/file_key_value_store.cc:619\ntensorstore/kvstore/ocdbt/io/manifest_cache.cc:122\ntensorstore/internal/cache/kvs_backed_cache.h:208\ntensorstore/driv

- datas
```python
ls ../DATA/og_marl/smac_v1/
2s3z.zip  3m.vlt  3m.zip  3s5z_vs_3s6z.zip  5m_vs_6m.zip  8m.zip

ls ../DATA/og_marl/smac_v1/3m.vlt/*
../DATA/og_marl/smac_v1/3m.vlt/Good:
d  manifest.ocdbt  metadata.json

../DATA/og_marl/smac_v1/3m.vlt/Medium:
d  manifest.ocdbt  metadata.json

../DATA/og_marl/smac_v1/3m.vlt/Poor:
d  manifest.ocdbt  metadata.json

[jcformanek]

Thanks for opening this issue, we are happy to help. This looks like an issue with tensorstore. @callumtilbury can you please help.

[callumtilbury]

Hi @DelinQu, thanks for flagging this! Unfortunately this is a known limitation of our current implementation, where your rel_dir argument cannot traverse backwards in your folder directory—i.e. you cannot point to the parent folder ... I will revisit the issue to see if we can solve this problem; but in the interim, could you perhaps run your script such that you access the vaults from the current directory? Thanks!

[callumtilbury]

Update: simple fix here, hopefully merged in soon!

[DelinQu]

Thanks for your reply @callumtilbury 🤗

[callumtilbury]

The bug (https://github.com/instadeepai/flashbax/pull/32) in Flashbax has been merged, and a new release will be cut soon. We will then update the OG-MARL dependency!