Open lhz opened 1 month ago
Need to implement this in Python:
find models -newer ~/.config/dbtwiz/models-cache.json
from pathlib import Path
cache_mtime = (Path.home() / ".config" / "dbtwiz" / "models-cache.json").stat().st_mtime
all_models = (Path.cwd() / "models").glob("**/*.sql")
modified_models = list(filter(lambda p: p.stat().st_mtime > cache_mtime, all_models))
We skip running
dbt parse
before reading the manifest, since the parsing is quite slow.When new models have been added, we need to update the manifest before they will be available in interactive selectors and other parts of dbtwiz, so we should do a fast recursive scan of the model folder to check for files that have been updated since the manifest was last built and automatically do a
dbt parse
before reading the manifest when needed.