Implement Delete a DR DAG

[sadeghim]

here is the jenkins job script:

solrWs="http://solr.ala:8983/solr/biocache/update?commit=true"

for dr in ${datasetIds[@]}
do
    echo "Deleting from solr and pipelines for: $dr"
    #echo "Deleting from pipelines for: $dr"

    curl "$solrWs" -H "Content-Type: text/xml"  --data-binary "<delete><query>dataResourceUid:$dr</query></delete>"

    sudo -u spark /data/hadoop/bin/hdfs dfs -rm -r -f /pipelines-all-datasets/index-record/$dr
    sudo -u spark /data/hadoop/bin/hdfs dfs -rm -r -f /dwca-exports/$dr
    sudo -u spark /data/hadoop/bin/hdfs dfs -rm -r -f /pipelines-data/$dr    
done

[peggynewman]

In prod