NYCPlanning / db-developments

🏠 🏘️ 🏗️ Developments Database
https://nycplanning.github.io/db-developments
8 stars 2 forks source link

Refactor dataloading function #577

Closed SashaWeinstein closed 1 year ago

SashaWeinstein commented 1 year ago

When I ran ./devdb.sh dataloading weekly from the root directory I didn't get the expected behavior. The first issue is that it didn't pass the weekly parameter to the dataloading script. The second was that it automatically cleared the .library folder. This is the code in question (you can see the file here)

function dataloading { 
    shift;
    MODE="${1:-edm}"
    CLEAR="${2:-true}"
    echo "mode: $MODE"
    ./bash/01_dataloading.sh $1
    if [ "${CLEAR}" == "true" ]; then
        rm -rf .library
    fi
}

Passing the weekly parameter is just a bug fix, no issue with the design there. We could just default CLEAR to false, but I'm skeptical we should keep it in at all. Other pipelines have a clear function, and it's quite easy to just delete .library manually.