search

VisionEval / VisionEval-Dev

Development version of VisionEval framework
https://visioneval.github.io/
Apache License 2.0
6 stars 32 forks source link

When using VE_EXPRESS=YES (default for ve.build), not checking out-of-date data #137

Closed jrawbits closed 2 years ago

jrawbits commented 3 years ago

The VE_EXPRESS environment variable (set by using ve.build(express=TRUE) (the default) is designed to not re-build data packages if they already exist. During package development that re-estimates models or repackages data, that default means that existing data files will not be updated.

The data file handling should be more nuanced so that the source data and the data target files are checked for relative file dates and updated if the source has changed.

The hard part of this issue is that the data files are generated in code, so it is not always externally apparent that the source data has changed (if we compare code files to the data files, they will always get rebuilt).

The workaround is to use ve.build(express=FALSE) if you need to make sure that data files get rebuilt (it will rebuild ALL the data files in your package if any of the source files are post-dated).

jrawbits commented 2 years ago

The data generation happens properly in the current build process