search

AtlasOfLivingAustralia / data-management

Data management issue tracking
7 stars 0 forks source link

Delete records from DwCA DAG #894

Open sadeghim opened 1 year ago

sadeghim commented 1 year ago

This is the delete records job in jenkins:

#!/bin/bash

cd /data/dmgt/scripts/utils/py
source ./.venv/bin/activate

#python3 delete-records.py -f /data/jenkins/dr368/dr368.zip  -d /data/jenkins/temp/testCatNumbers.csv -w /data/tmp  --chunksize 500000
python3 delete-records.py -f ${sourceDWCAFile} -d ${deleteListFile} -w ${tmpDir}  --chunksize ${chunksize}