search

leap-stc / data-management

Collection of code to manually populate the persistent cloud bucket with data
https://catalog.leap.columbia.edu/
Apache License 2.0
0 stars 5 forks source link

Adding Feedstock structure #19

Closed jbusecke closed 1 year ago

jbusecke commented 1 year ago

Towards #13

jbusecke commented 1 year ago

I am stuck here. The jobs were successfully submitted but crashed due to this error in dataflow:

Workflow failed. Causes: Unable to create directory: gs://leap-scratch/data-library/temp/gh-leap-stc-data-management-3107bcc-1683762190.1683762212.140912/dax-tmp-2023-05-10_16_43_37-1120618972252613873-S05-0-1419016091a5051e.

I thought the jobname might be the culprit -> Hence https://github.com/pangeo-forge/deploy-recipe-action/pull/2

With that PR, I am failing at the job submission stage.

I think having https://github.com/pangeo-forge/deploy-recipe-action/issues/3 would be helpful to debug what is going on.

jbusecke commented 1 year ago

Oh hold up. The latest iteration just submitted a job. Ill wait for a few minutes if the issue with the folder creation is fixed and then call it a night. Huuuge thanks to @cisaacstern for all the help here.