search

NCAR / CUPiD

CUPiD is a “one stop shop” that enables and integrates timeseries file generation, data standardization, diagnostics, and metrics from all CESM components.
https://ncar.github.io/CUPiD/
Apache License 2.0
21 stars 19 forks source link

Testing out external scripts #99

Open dabail10 opened 1 month ago

dabail10 commented 1 month ago

Testing out calls to external scripts here.

mnlevy1981 commented 1 month ago

From the 89b8c5e commit log:

ploomber.exceptions.DAGBuildError:
============================================= DAG build failed =============================================
--------------------------- CUPiDScriptRunner: divergence -> File('vector.png') ----------------------------
-------------------- /glade/work/mlevy/codes/CUPiD/examples/nblibrary/ice/divergence.py --------------------
ploomber.exceptions.TaskBuildError: Error building task 'divergence': the task ran successfully but the product File('vector.png') is missing. Ensure your code is generating it.
ploomber.exceptions.TaskBuildError: Exception when running on_finish for task 'divergence'
============================================= Summary (1 task) =============================================
CUPiDScriptRunner: divergence -> File('vector.png')
============================================= DAG build failed =============================================

@rmshkv it looks like it is creating examples/nblibrary/ice/vector.png, rather than examples/coupled_model/computed_notebooks/ice/vector.png (or maybe we want examples/coupled_model/computed_scripts/ice/vector.png?)

dabail10 commented 1 month ago

Perhaps I need a complete path here?

mnlevy1981 commented 1 month ago

Perhaps I need a complete path here?

I don't think so -- the problem is that ploomber is running your script in examples/nblibrary/ so that's where the output is going... but we should be able to set up CUPiD so that ploomber executes it in a different location (that's what we do for notebooks)

rmshkv commented 1 month ago

I dug into this a little and it seems like by default local_execution is set to false for both notebooks and scripts, so I wouldn't expect the output to be saved in the nblibrary directory at first glance. When I run it myself with the current configuration I get environment errors, so not able to test myself without doing some more work on that functionality first.