search

ari-apc-lab / croupier

Cloudify plugin for HPCs and batch applications
https://hub.docker.com/repository/docker/marangiop/cloudify-croupier-ari-apc-lab
Apache License 2.0
6 stars 4 forks source link

Unable to pass job specific parameters inside inputs for jobs submitted to CESGA #10

Open marangiop opened 3 years ago

marangiop commented 3 years ago

Cloudify Version 20.02.23~community (Community)

Croupier Version Commit 2abb8b63821ae228af621482a750aba3e6f4a50f of branch permedcoe

In the EUXDAT project we implemented a way to pass job specific inputs as additional arguments inside the inputs file. Basically the orchestrator would create a .py file inside HPC before execution started, then this file containing parameter values would be parsed by the main computational script

https://github.com/ari-apc-lab/croupier/blob/euxdat/croupier_plugin/tests/integration/inputs/agroclimate-zones-pilot-inputs_template.yaml

I was trying to do something similar by adding the following lines to an inputs yaml to be used for executing a blueprint from the AGROAPPS partner:

job_config_content: | 
  vineyard='Somontano'\n
  year= 2019\n
  month= 5\n
  day= 4\n
job_config_pathname: ~/$STORE/a3_disease_model_workflow/config.py

And I got this error 

2021-06-03 10:24:26.443  CFY <test_single> [gfs1_e2chcl] State changed to PENDING
2021-06-03 10:24:30.048  LOG <test_single> WARNING: 'CANCELLED by 13416'

I have not had the time to investigate this. The hypothesis proposed by Yosu is that the CESGA console is not compatible with this implementation that was tested with Hawk. So somebody would need to check by hand to execute the same ssh command that is being executed by Croupier to create the config file to check the reported errors. Probably this config_bootstrap.sh needs to be amended to be CESGA compatible.