Open patkyn opened 2 months ago
2 issues: 1) A need to implement a 'Timedout' in the request if the server is not responding with the response. Currently, the request request.get seems to be stucked forever causing the cluster to hang. https://data.bionet.nsw.gov.au/biosvcapp/odata/SpeciesSightings_CoreData?%24filter=dcterms_modified+ge+2024-08-12+and+dcterms_modified+lt+2024-08-19&%24top=50000&%24count=true&%24skip=0
Refer to EMR cluster: https://ap-southeast-2.console.aws.amazon.com/emr/home?region=ap-southeast-2#/clusterDetails/j-QK821F6BZCHM
2) A bug in the deletion, possibly from the latest release of dwcahandler v0.2.0
https://ap-southeast-2.console.aws.amazon.com/emr/home?region=ap-southeast-2#/clusterDetails/j-2CCVVO9JJKP6L
2 issues: 1) A need to implement a 'Timedout' in the request if the server is not responding with the response. Currently, the request request.get seems to be stucked forever causing the cluster to hang. https://data.bionet.nsw.gov.au/biosvcapp/odata/SpeciesSightings_CoreData?%24filter=dcterms_modified+ge+2024-08-12+and+dcterms_modified+lt+2024-08-19&%24top=50000&%24count=true&%24skip=0
Refer to EMR cluster: https://ap-southeast-2.console.aws.amazon.com/emr/home?region=ap-southeast-2#/clusterDetails/j-QK821F6BZCHM
2) A bug in the deletion, possibly from the latest release of dwcahandler v0.2.0
https://ap-southeast-2.console.aws.amazon.com/emr/home?region=ap-southeast-2#/clusterDetails/j-2CCVVO9JJKP6L