search

radical-cybertools / radical.analytics

Analytics for RADICAL-Cybertools
Other
1 stars 1 forks source link

NoneType object is not subscriptable #114

Closed euhruska closed 4 years ago

euhruska commented 4 years ago

I would like to measure the duration of one stage using analytics in py3 to compare actual consumption versus requested walltime, but I get errors like

radical-analytics-inspect re.session.login4.eh22.018283.0004
re.session.login4.eh22.018283.0004 cache read failed: [Errno 2] No such file or
 directory: '/ccs/home/eh22/.radical/analytics/cache//re.session.login4.eh22.018283.0004.pickle'                                                              
WARNING: profile "re.session.login4.eh22.018283.0004/umgr_reschedule_pubsub.prof" not correctly closed.                                                       
WARNING: profile "re.session.login4.eh22.018283.0004/umgr.0000.prof" not correctly closed.                                                                    
WARNING: profile "re.session.login4.eh22.018283.0004/umgr_staging_output_queue. prof" not correctly closed.                                                    
WARNING: profile "re.session.login4.eh22.018283.0004/umgr_unschedule_pubsub.prof" not correctly closed.                                                       
WARNING: profile "re.session.login4.eh22.018283.0004/log_pubsub.prof" not correctly closed.                                                                   
WARNING: profile "re.session.login4.eh22.018283.0004/radical.entk.resource_manager.0000.prof" not correctly closed.                                           
WARNING: profile "re.session.login4.eh22.018283.0004/cmgr.0000.hb.prof" not correctly closed.                                                                 
WARNING: profile "re.session.login4.eh22.018283.0004/pmgr_launching.0000.prof" ot correctly closed.                                                          
WARNING: profile "re.session.login4.eh22.018283.0004/cmgr.0001.prof" not correctly closed.                                                                    
WARNING: profile "re.session.login4.eh22.018283.0004/cmgr.0002.prof" not correctly closed.                                                                    
+ re.session.login4.eh22.018283.0004 (json)                                    
session loaded                                                                 
Traceback (most recent call last):                                             
  File "/autofs/nccs-svm1_home1/eh22/.conda/envs/vamp10/bin/rp_inspect/plot_stte.py", line 77, in <module>                                                   
    in thing.description.get('name', '').split()]                              
  File "/autofs/nccs-svm1_home1/eh22/.conda/envs/vamp10/bin/rp_inspect/plot_stte.py", line 76, in <listcomp>                                                 
    pipe[thing.uid] = [int(x) for x                                            
ValueError: invalid literal for int() with base 10: 'task.0000,None,stage.0000None,pipeline.0000,None'                                                       
...Traceback (most recent call last):                                          
  File "/autofs/nccs-svm1_home1/eh22/.conda/envs/vamp10/bin/rp_inspect/plot_util.py", line 116, in <module>                                                   
    prov, cons, stats_abs, stats_rel, info = session.utilization(metrics)      
  File "/ccs/home/eh22/.conda/envs/vamp10/lib/python3.7/site-packages/radical/analytics/session.py", line 961, in utilization                                 
    provided  = rp.utils.get_provided_resources(self)                          
  File "/ccs/home/eh22/.conda/envs/vamp10/lib/python3.7/site-packages/radical/pilot/utils/prof_utils.py", line 479, in get_provided_resources                 
    data = _get_pilot_provision(session, p)                                    
  File "/ccs/home/eh22/.conda/envs/vamp10/lib/python3.7/site-packages/radical/pilot/utils/prof_utils.py", line 437, in _get_pilot_provision                   
    cpn   = pilot.cfg['resource_details']['rm_info']['cores_per_node']         
TypeError: 'NoneType' object is not subscriptable                              
 done

and

radical-analytics-inspect re.session.login4.eh22.018284.0000
re.session.login4.eh22.018284.0000 Traceback (most recent call last):
  File "/autofs/nccs-svm1_home1/eh22/.conda/envs/vamp10/bin/rp_inspect/plot_state.py", line 61, in <module>
    session = ra.Session.create(src, stype)
  File "/ccs/home/eh22/.conda/envs/vamp10/lib/python3.7/site-packages/radical/analytics/session.py", line 258, in create
    sid, src, tgt, ext = Session._get_sid(sid, src)
  File "/ccs/home/eh22/.conda/envs/vamp10/lib/python3.7/site-packages/radical/analytics/session.py", line 161, in _get_sid
    raise ValueError('src [%s] does not exist' % src)
ValueError: src [re.session.login4.eh22.018284.0000] does not exist

possibly connected with https://github.com/radical-cybertools/radical.analytics/issues/108 radical-stack

python               : 3.7.6
  pythonpath           : /sw/summit/xalt/1.1.4/site:/sw/summit/xalt/1.1.4/libexec
  virtualenv           : vamp10
  radical.analytics    : 0.90.7-v0.72.0-34-g545607e@devel
  radical.entk         : 1.0.0-v1.0.0-7-gd6c4c29@devel
  radical.pilot        : 1.0.0
  radical.saga         : 1.0.0-v1.0.0-3-gb44efbd@devel
  radical.utils        : 1.0.0
andre-merzky commented 4 years ago

I don't think I can help with src [re.session.login4.eh22.018284.0000] does not exist - I would suspect that re.session.login4.eh22.018284.0000 is indeed not a valid session directory?

As for the None-type problem: that is indeed related to #108, which though should be resolved via radical-cybertools/radical.pilot/pull/2032. The new RP release goes out today and contains this fix, otherwise please give the RP devel branch a try.

euhruska commented 4 years ago

the session re.session.login4.eh22.018284.0000 exists I will try the RP release

andre-merzky commented 4 years ago

the session re.session.login4.eh22.018284.0000 exists

I am not sure about that error then... :-/ If you want me to look into it, can you please create a tarball of that session and attach it?

mturilli commented 4 years ago

@euhruska is this still an issue? Is there a tarball we can use or should we close the ticket?

euhruska commented 4 years ago

not a current issue