jataware / dojo

5 stars 0 forks source link

Dataset Append hanging #175

Closed bagoodman closed 1 year ago

bagoodman commented 1 year ago

When I tried to append 2021 data to the ND-GAIN dataset (https://causemos-analyst.dojo-modeling.com/datasets/append/register/bac91444-31ed-42cd-9550-7bd681daa792), after I load the new data to add and click on the 'Next' button, nothing happens. The 'Next' button stays clickable and no processing or movement from the Dataset Append front-page for the dataset pccirs/

brandomr commented 1 year ago

@bagoodman my suspicion is that it has to do with the new logic around data transformations breaking the append capability. We'll investigate and report back.

adamgilg commented 1 year ago

I confirmed that appending a dataset worked for me locally, so this may be an S3 data path issue on the deployed instance. We're continuing to look into this further.

brandomr commented 1 year ago

@bagoodman the issue seemed to be that the original data was in a different S3 bucket due to changes in our deployment in the year since that data was registered. I copied over the data to the new bucket and append seems to be working alright.

Screenshot 2023-10-11 at 8 46 45 PM

https://causemos-analyst.dojo-modeling.com/datasets/append/preview/bac91444-31ed-42cd-9550-7bd681daa792?filename=raw_data_1.xlsx

However, it looks like we're having connectivity issues to Causemos at the moment--our server had some problem today and now has a new IP address. I'll notify Uncharted to add this new IP to their whitelist.

@scotthaleen can we use an Elastic IP so we don't have to have them re-add new IPs when we have to swap out the server?

brandomr commented 1 year ago

Finished this process and the data is now appended since we have connectivity with Causemos again.