datadryad / dryad-product-roadmap

Repository of issues for Dryad project boards
https://github.com/orgs/datadryad/projects
8 stars 0 forks source link

Dataset stuck in "Processing" status for 24+hrs (10.5061/dryad.c59zw3rj0) #3675

Open jleighherzog opened 1 month ago

jleighherzog commented 1 month ago

10.5061/dryad.c59zw3rj0 Data from: Photosynthetic responses to instrinsic water use efficiency depend on atmospheric feedbacks and modify the magnitude of response to elevated CO2

Can you investigate this one? It's ~15GB in size and it's been stuck in "Processing" status for over a day. https://datadryad.org/stash/admin_dashboard?page=

ahamelers commented 1 month ago

This one has been erroring repeatedly. I've not looked into it fully (@alinvetian is meant to be handling submission errors this sprint), but from the error message it may be a problem with one of the files uploaded by URL

jleighherzog commented 1 month ago

@ahamelers : Ok.

alinvetian commented 1 month ago

@jleighherzog I checked the 15GB file for this dataset and it seems that the URL for it is not available. Image

jleighherzog commented 1 month ago

@alinvetian : What's a UPR?

alinvetian commented 1 month ago

@jleighherzog sorry for that, I ment to write URL

jleighherzog commented 1 month ago

@jleighherzog : Ok, can you guide me on next steps? What do I need to tell the author? They are unable to edit the submission.

ahamelers commented 1 month ago

@jleighherzog It's easiest if they can give us a new URL for the file in question. Then the dev team can set the URL in the database and try processing again. If they cannot provide another URL, we can set it back to 'in progress' for the user to upload the file another way.

jleighherzog commented 1 month ago

Sent request for new URL to author, waiting for reply.

jleighherzog commented 1 month ago

Would I be able to add files to my dryad repo if I give an URL? If yes,please use this URL (https://m-ca0ddc.7a577b.6fbd.data.globus.org/glade/derecho/scratch/amyliu/stomatalslope_archive/model_output/landonly_SP_LOWmedslope_2xCO2CPL_2xCO2_v3_lnd.tar.gz)

Please set it back to "Action Required" status

ryscher commented 1 month ago

Hi Jess, this link requires a user from NCAR. The author will have to open the permissions so we have access to download the file.

jleighherzog commented 3 weeks ago

From the author:

"I've updated the Globus permissions to the collection to public read and write. I've also updated the permissions for the folder on NCAR servers to be able to be read by anyone, but I think "anyone" in this case would refer to those with NCAR accounts. Hopefully the Globus permissions fix the issue. C/ping the link here again in case it's changed: https://mca0ddc.7a577b.6fbd.data.globus.org/glade/derecho/scratch/amyliu/stomatalslope_archive/model_output/landonly_SP_LOWmedslope_2xCO2CPL_2xCO2_v3_lnd.tar.gz"

ryscher commented 1 week ago

@jleighherzog Following up on this.... we're not getting the NCAR error any longer. But we still get "Access Denied".

jleighherzog commented 22 hours ago

Following up with author