audeering / audb

Manage audio and video databases
https://audeering.github.io/audb/
Other
23 stars 1 forks source link

Dependency file error reported when trying to build the documentation locally #402

Closed hagenw closed 2 months ago

hagenw commented 2 months ago

When I run the following:

$ python -m sphinx -W docs/ build/html -b html

It fails when executing docs/publish.rst with

...
RuntimeError: You want to depend on '1.0.0' of age-test, but the dependency file 'db.parquet' in ./age-test-1.1.0 does not match the dependency file for the requested version in the repository. Did you forgot to call 'audb.load_to(./age-test-1.1.0, age-test, version='1.0.0') or modified the file manually?

This error does not happen here in the CI pipeline, e.g. https://github.com/audeering/audb/pull/401. It is most likely related to https://github.com/audeering/audb/pull/398

/cc @ChristianGeng

hagenw commented 2 months ago

When publishing the first version of the dataset used in the documentation example, the dependency table in age-test-1.0.0/db.parquet contains the following entries for me:

                                            archive  bit_depth  channels                          checksum  duration format  removed  sampling_rate  type version
db.age.csv                                      age          0         0  d321a6718d7129005bf0931dad51a6b1       0.0    csv        0              0     0   1.0.0
audio/001.wav  436c65ec-1e42-f9de-2708-ecafe07e827e         16         1  ed7622a12b7c2858792814880ba73b75      60.0    wav        0          16000     1   1.0.0
audio/002.wav  fda7e4d6-f2b2-4cff-cab5-906ef5d57607         16         1  ed7622a12b7c2858792814880ba73b75      60.0    wav        0          16000     1   1.0.0
audio/003.wav  e26ef45d-bdc1-6153-bdc4-852d83806e4a         16         1  ed7622a12b7c2858792814880ba73b75      60.0    wav        0          16000     1   1.0.0

When storing the same dependency table with audb.load_to() to age-test-1.1.0/db.parquet, the resulting file contains:

                                            archive  bit_depth  channels                          checksum  duration format  removed  sampling_rate  type version
db.age.csv                                      age          0         0  f4d6fb3cca8f4361586e44eec638296c       0.0    csv        0              0     0   1.0.0
audio/001.wav  436c65ec-1e42-f9de-2708-ecafe07e827e         16         1  ed7622a12b7c2858792814880ba73b75      60.0    wav        0          16000     1   1.0.0
audio/002.wav  fda7e4d6-f2b2-4cff-cab5-906ef5d57607         16         1  ed7622a12b7c2858792814880ba73b75      60.0    wav        0          16000     1   1.0.0
audio/003.wav  e26ef45d-bdc1-6153-bdc4-852d83806e4a         16         1  ed7622a12b7c2858792814880ba73b75      60.0    wav        0          16000     1   1.0.0

showing a different checksum for the table. The checksum of the published dataset is identical with the first table:

$ parquet-tools csv data/data-local/age-test/1.0.0/db.parquet
file,archive,bit_depth,channels,checksum,duration,format,removed,sampling_rate,type,version
db.age.csv,age,0,0,d321a6718d7129005bf0931dad51a6b1,0.0,csv,0,0,0,1.0.0
audio/001.wav,436c65ec-1e42-f9de-2708-ecafe07e827e,16,1,ed7622a12b7c2858792814880ba73b75,60.0,wav,0,16000,1,1.0.0
audio/002.wav,fda7e4d6-f2b2-4cff-cab5-906ef5d57607,16,1,ed7622a12b7c2858792814880ba73b75,60.0,wav,0,16000,1,1.0.0
audio/003.wav,e26ef45d-bdc1-6153-bdc4-852d83806e4a,16,1,ed7622a12b7c2858792814880ba73b75,60.0,wav,0,16000,1,1.0.0
hagenw commented 2 months ago

The problem happens, as audb.dependencies() does first look for a of the dataset in the shared cache root, in which I had an old version of it stored. We need to set audb.config.SHARED_CACHE_ROOT inside docs/publish.rst as well.

ChristianGeng commented 2 months ago

The problem happens, as audb.dependencies() does first look for a of the dataset in the shared cache root, in which I had an old version of it stored. We need to set audb.config.SHARED_CACHE_ROOT inside docs/publish.rst as well.

This should explain why it builds for me: I didn't have the age-test database downloaded.