radical-collaboration / hpc-workflows

NSF16514 EarthCube Project - Award Number:1639694
5 stars 0 forks source link

ENTK running example in doc error #108

Closed wjlei1990 closed 4 years ago

wjlei1990 commented 4 years ago

Hi,

I am trying to run the entk example on summit localhost here https://radicalentk.readthedocs.io/en/stable/user_guide/adding_data.html

However, there is error shown up when I run the code:

radical.entk.exceptions.ValueError: Value for attribute placeholder of object task incorrect. Expected value $Pipeline_(pipeline_name)_Stage_(stage_name)_Task_(task_name) or $SHARED, but got ['$Pipeline', 'None', 'Stage', 'None', 'Task', 'None'].

I think it is coming from this line:

    t2.copy_input_data = ['$Pipeline_%s_Stage_%s_Task_%s/output.txt'%(p.uid, s1.uid, t1.uid)]

I tried to print the p.uid, s1.uid, and t1.uid. They are all None.

Do you have any idea?

My env:

(summit-entk) lei@login5 /autofs/nccs-svm1_home1/lei/software/summit/conda/summit-entk/test $ 
radical-stack 

  python               : 3.7.6
  pythonpath           : /sw/summit/xalt/1.2.0/site:/sw/summit/xalt/1.2.0/libexec
  virtualenv           : summit-entk

  radical.entk         : 1.0.2
  radical.pilot        : 1.1.1
  radical.saga         : 1.1.2
  radical.utils        : 1.1.1
lee212 commented 4 years ago

It looks like a documentation error. Could you try name instead p.uid, s1.uid, t1.uid? like:

p.name, s1.name, t1.name

I will get read the docs updated about this section.

wjlei1990 commented 4 years ago

As documented in the API docs here, for example: https://radicalentk.readthedocs.io/en/stable/api/app_create/task.html

Is uid a attribute in the Task class?

andre-merzky commented 4 years ago

uid is a valid attribute, but its value is assigned at a very late point in time, during task submission, and thus it is still None when you try to use it. name is assigned by the user immediately upon creation and is always valid, but it is the user's responsibility to keep it unique. Last but not least, the resolution of data references (as for you input staging dependency) is done based on the name attribute, not on uids.

wjlei1990 commented 4 years ago

Got it. Thanks for clarifying it.

So is the example in the doc still valid?

t2.copy_input_data = ['$Pipeline_%s_Stage_%s_Task_%s/output.txt'%(p.uid, s1.uid, t1.uid)]