radical-cybertools / radical.pilot

RADICAL-Pilot
http://radical-cybertools.github.io/radical-pilot/index.html
Other
54 stars 23 forks source link

Fix/tutorial task description #3030

Closed mtitov closed 1 year ago

mtitov commented 1 year ago

Original tutorial was with outputs, this one is without outputs

review-notebook-app[bot] commented 1 year ago

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

mtitov commented 1 year ago

@andre-merzky I think this is what you mentioned earlier regarding RPC calls - this tutorial hangs after calling the following method (for the last task in this tutorial) - added traceback into "Details" after the manual stop

pilot.prepare_env(env_name='test_env', 
                  env_spec={'type' :  'venv', 
                            'setup': ['psutil']})
``` --------------------------------------------------------------------------- KeyboardInterrupt Traceback (most recent call last) Cell In [14], line 1 ----> 1 pilot.prepare_env(env_name='test_env', 2 env_spec={'type' : 'venv', 3 'setup': ['psutil']}) 5 td = rp.TaskDescription({'executable' : '/bin/sh', 6 'arguments' : ['-c', 'which python3; pip list | grep psutil'], 7 'named_env' : 'test_env' 8 }) 9 task = tmgr.submit_tasks(td) File /opt/conda/lib/python3.9/site-packages/radical/pilot/pilot.py:706, in Pilot.prepare_env(self, env_name, env_spec) 667 def prepare_env(self, env_name, env_spec): 668 """Prepare a virtual environment. 669 670 Request the preparation of a task or worker environment on the target (...) 703 704 """ --> 706 self.rpc('prepare_env', {'env_name': env_name, 707 'env_spec': env_spec}) File /opt/conda/lib/python3.9/site-packages/radical/pilot/pilot.py:772, in Pilot.rpc(self, cmd, args) 763 rpc_req = {'uid' : rpc_id, 764 'rpc' : cmd, 765 'tgt' : self._uid, 766 'arg' : args} 769 self._ctrl_pub.put(rpc.CONTROL_PUBSUB, {'cmd': 'rpc_req', 770 'arg': rpc_req, 771 'fwd': True}) --> 772 rpc_res = self._rpc_queue.get() 773 self._log.debug('rpc result: %s', rpc_res['ret']) 775 if rpc_res['ret']: File /opt/conda/lib/python3.9/queue.py:171, in Queue.get(self, block, timeout) 169 elif timeout is None: 170 while not self._qsize(): --> 171 self.not_empty.wait() 172 elif timeout < 0: 173 raise ValueError("'timeout' must be a non-negative number") File /opt/conda/lib/python3.9/threading.py:312, in Condition.wait(self, timeout) 310 try: # restore state no matter what (e.g., KeyboardInterrupt) 311 if timeout is None: --> 312 waiter.acquire() 313 gotit = True 314 else: KeyboardInterrupt: ```

p.s. after RPC calls are resolved, then this PR can be pushed further

codecov[bot] commented 1 year ago

Codecov Report

Merging #3030 (a501bbf) into devel_nodb_2 (03e464f) will increase coverage by 0.19%. Report is 19 commits behind head on devel_nodb_2. The diff coverage is 44.04%.

:exclamation: Current head a501bbf differs from pull request most recent head 1aa3b8f. Consider uploading reports for the commit 1aa3b8f to get more accurate results

@@               Coverage Diff                @@
##           devel_nodb_2    #3030      +/-   ##
================================================
+ Coverage         42.22%   42.42%   +0.19%     
================================================
  Files                99       99              
  Lines             10796    10825      +29     
================================================
+ Hits               4559     4592      +33     
+ Misses             6237     6233       -4     
Files Coverage Δ
src/radical/pilot/agent/executing/base.py 74.48% <ø> (-0.52%) :arrow_down:
src/radical/pilot/agent/executing/sleep.py 21.95% <ø> (ø)
src/radical/pilot/messages.py 100.00% <ø> (+21.73%) :arrow_up:
src/radical/pilot/pilot_manager.py 19.37% <100.00%> (+0.05%) :arrow_up:
src/radical/pilot/pmgr/launching/base.py 47.24% <100.00%> (+0.27%) :arrow_up:
src/radical/pilot/proxy.py 16.96% <ø> (ø)
src/radical/pilot/tmgr/staging_input/default.py 28.64% <100.00%> (+0.97%) :arrow_up:
src/radical/pilot/raptor/master.py 29.06% <50.00%> (+0.16%) :arrow_up:
src/radical/pilot/task_manager.py 23.93% <50.00%> (+0.10%) :arrow_up:
src/radical/pilot/tmgr/scheduler/base.py 16.12% <33.33%> (+0.17%) :arrow_up:
... and 8 more

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more