search

nodet / dowml

A library and command line client to use Decision Optimization on IBM Watson Machine Learning (WML). NOT SUPPORTED BY IBM.
Apache License 2.0
3 stars 0 forks source link

'log' doesn't work if the jobs use data-assets as output #31

Closed nodet closed 2 years ago

nodet commented 2 years ago

'log' calls DOWMLLib.get_job_details with with_content == 'log'. This uses output_data as a filter. But what if the log was stored as a data-asset? Is the log always inline? Does output_data as a filter also gets output_data_references?

nodet commented 2 years ago

Indeed, when 'outputs assets' is used, the log is stored in output_data_references and that is not fetched when the filter list only has output_data.

dowml> log 22e7237b-d7e8-4f66-b82a-07ebfd65424a
2021-10-05 18:20:57,647 Fetching output...
           GET https://eu-de.ml.cloud.ibm.com/ml/v4/deployment_jobs/22e7237b-d7e8-4f66-b82a-07ebfd65424a?version=2021-06-24&space_id=4645be25-e08e-4a34-a475-ff3cd0dc9635&include=output_data
2021-10-05 18:20:58,675 Done.
None
nodet commented 2 years ago

APIClient.data_assets.download requires a file name because it will necessarily store the asset on disk. In order to allow the use of multiple instances of the library concurrently, I will have to create temporary directories using https://docs.python.org/3/library/tempfile.html#tempfile.TemporaryDirectory.