Timing load of IMAS data

[orso82]

see https://github.com/gafusion/omas/commit/0f7285ac9da71cf8982e427a15e3a95e8c280c83#commitcomment-32969860

[orso82]

@jsferreira I opened this issue to see what we can do to speedup the loading from IMAS

Could you please share the piece of code that you have used to generate your timings? This way I can test it myself and perhaps find if there are bottlenecks on the OMAS side that I can iron out.

[jsferreira]

@orso82

With omas/0.35.0 the loading times are similar. Still need to test the last commit.

The tests are very simple. At the WPCD gateway with omas/0.35.0:

• core_profiles, 1 time-slice

  %time ods=omas.load_omas_imas(machine='test',pulse=54118,run=2,paths=[['core_profiles']],verbose=False, imas_version=u'3.20.0')
  CPU times: user 64.6 ms, sys: 30.6 ms, total: 95.2 ms
  Wall time: 84.7 ms

• core_profiles, 821 time-slices

  %time ods=omas.load_omas_imas(machine='test',pulse=54118,run=1,paths=[['core_profiles']],verbose=False, imas_version=u'3.20.0')
  CPU times: user 30 min 19 s, sys: 10.7 s, total: 30 min 30 s
  Wall time: 30 min 35 s

• equilibrium, 1 time-slice without GGD

  %time ods=omas.load_omas_imas(machine='test',pulse=54118,run=3,paths=[['equilibrium']],verbose=False, imas_version=u'3.20.0')
  CPU times: user 660 ms, sys: 53.5 ms, total: 714 ms
  Wall time: 677 ms

• equilibrium, 1 time-slice with GGD

  %time ods=omas.load_omas_imas(machine='test',pulse=54118,run=1,paths=[['equilibrium']],verbose=False, imas_version=u'3.20.0')
  ~ 50 min

[orso82]

Thank you @jsferreira . The speedup that I saw were measured when accessing data that was already within an ODS, for example when using the .from_omas() and .to_omas() in the OMFIT classes. I'll see if there is something that I can do to improve the performance of load_omas_imas()

[orso82]

@jsferreira I have to remind myself how I used to connect to the gateway server. Latest OMAS version 0.37.0 should do a lot better. Please let me know once you get the chance to test it.

[orso82]

@jsferreira with commit a3d86707868eaff6a81e815b9311d525f98846ca you should now be able to skip loading of ggd structures. I have not field tested this so please let me know if you find any issues.

[jsferreira]

Another useful thing to have would be a way to import into OMAS only a slice of the IDS, eg. by using the getSlice

[github-actions[bot]]

Stale issue message