search

NYCPlanning / data-engineering

Primary repository for NYC DCP's Data Engineering team
14 stars 0 forks source link

add all files from EDDE repo #755

Closed damonmcc closed 2 months ago

damonmcc commented 2 months ago

related to #520

this copies nearly all files from the EDDE repo db-equitable-development-tool. I left out jupyter notebooks and R scripts

will use and revise files in follow-up PRs

fvankrieken commented 2 months ago

Let's maybe take the tests out for now

fvankrieken commented 2 months ago

My gut is that "internal review" and "external review" folders should not be included, they seem to be output data only?

damonmcc commented 2 months ago

My gut is that "internal review" and "external review" folders should not be included, they seem to be output data only?

external_review/export_DO.sh is used to export data in the two github actions

and I'm seeing from internal_review. ... import ... show up in 34 files in aggregate/

damonmcc commented 2 months ago

our python formatting test fails (example), will try formatting those files

damonmcc commented 2 months ago

@fvankrieken should be good to go

would rather keep the tests. they aren't being run yet and I'll see how broken they are in a follow-up

fvankrieken commented 2 months ago

@fvankrieken should be good to go

would rather keep the tests. they aren't being run yet and I'll see how broken they are in a follow-up

Oh yeah definitely keep them, my language was a bit too harsh. Just don't run them in our main test action is all I meant