Same input data, different plots

[vrpascuzzi]

Problem

Resource utilization (RU) plots of the same input data are different when they should be identical.

Version

❯ radical-stack 

  python               : /<...>/miniconda3/envs/rct/bin/python3
  pythonpath           : 
  version              : 3.8.13
  virtualenv           : rct

  radical.analytics    : 1.13.0
  radical.entk         : 1.13.0
  radical.gtod         : 1.13.0
  radical.pilot        : 1.13.0
  radical.saga         : 1.13.0
  radical.utils        : 1.13.0

Observation

Attached figure shows four RU plots. All use the exact same input data to be plotted. The issue is that (a), (b), (c) and (d) should be identical; however, there are subtle differences in their GPU utilization.

[andre-merzky]

Thanks for reporting! Can you please attach the session you are plotting?

[vrpascuzzi]

re.session.login3.pascuzzi.019079.0001.zip

[andre-merzky]

Thanks @vrpascuzzi ! Did you use a custom script to plot, or how did you invoke the plotter? Sorry for asking piecewise :-)

[vrpascuzzi]

Sorry for asking piecewise :-)

No problemo. I knew to add these extra bits but was in a rush.

resource_utilization-summit.ipynb.zip

[andre-merzky]

@vrpascuzzi : if you add this to your 6th cell:

# Add axes labels
fig.text(0.5, -0.2, 'Time (s)', ha='center')
fig.savefig('tmp.pdf', bbox_inches="tight")

and then check the resulting tmp.pdf, you will see that the plots are in fact consistent. What you see above is a rendering artifact of png I'm afraid...

As discussed online, we should keep the ticket open though as the contributions other than schedule seem to be missing...

[vrpascuzzi]

Finally got all the dependencies in order (backward-compatibility, anyone?). Using:

$ radical-stack 

  python               : /ccs/home/pascuzzi/conda/bin/python3
  pythonpath           : /sw/summit/xalt/1.2.1/site:/sw/summit/xalt/1.2.1/libexec
  version              : 3.8.12
  virtualenv           : /ccs/home/pascuzzi/conda

  radical.entk         : 1.11.0
  radical.gtod         : 1.6.7
  radical.pilot        : 1.9.2
  radical.saga         : 1.11.1
  radical.utils        : 1.11.0

the same run looks as expected (same input data in each plot):

[andre-merzky]

Great!

Yeah, backward compatibility is a problem - Matteo can tell you a thing or two about it... :-( Alas, some changes will likely break that again in the near future as the raptor changes introduced yet another set of profile events. We should document the interface between RP/RE and RA after that to stabilize this. I opened radical-cybertools/radical.pilot/issues/2566 for that.

Can this issue be closed though?

[vrpascuzzi]

Can this issue be closed though?

Given the title, yes. However, I think there are still issues with plotting in 1.13.0. Another ticket?

[andre-merzky]

@vrpascuzzi: I lost track of the open issues in this ticket - can you please help me to clarify what is still open? You mention 'issues with plotting in 1.13.0.' - does the version apply to both the experiment and the analytics stack? If so, please do open a new ticket indeed and attach an example session, please. Thanks!

[andre-merzky]

@vrpascuzzi : ping

[vrpascuzzi]

Sorry for missing this.

I will follow-up in another ticket if necessary. Thanks.