Closed fvankrieken closed 1 year ago
@alexrichey this is pretty limited in scope but is an example of adding new data sources via data library - two new yaml files that specify that the dataset "source" is from the added scripts
I think I get the gist of the changes. Looks good to me, though maybe we want consider a linter/formatter to flag unused imports and enforce style?
We use black often for linting/formatting but could definitely stand to incorporate it with a tad more rigor/norms (commit hooks or something along those lines).
I don't think it complains about unused imports (or removes them)
(started typing this before I saw Finn's reply lol)
@alexrichey agree on the value of linters/formatters. we haven't taken a shot at implementing them as a CI check on all mono repo PRs, but would love to here sometime soon
black
has been our go to for python formatting (it may flag unused imports though) and here's where it was being used by some datasets that are now in the mono repo
I had some fun exploring the use of trunk
to lint many languages at once, but the overhead of the VS Code extension has me turning away from it
Two things.
script
-sourced dataset that are still pulling from just one endpoint). I missed bpl_libraries back then so added it now (and in the one test yml file that uses it)