ToFuProject / tofu

Project for an open-source python library for synthetic diagnostics and tomography for Fusion devices
https://tofuproject.github.io/tofu/index.html
MIT License
72 stars 11 forks source link

Plasma2D extraction from MultiIDSLoader on ITER failing #218

Closed munechika-koyo closed 4 years ago

munechika-koyo commented 5 years ago

Extracting a Plasma2D from a MultiIDSLoader does not seem to work in my case:

In [2]: multi = tf.imas2tofu.MultiIDSLoader(user='public',tokamak='iter',shot=134000,run=28,ids=['core_sources', 'equilibrium'])
Getting ids   [occ]  tokamak  user    version  shot    run  refshot  refrun
------------  -----  -------  ------  -------  ------  ---  -------  ------
core_sources  [0]    iter     public  3        134000  28   -1       -1
equilibrium   [0]    "        "       "        "       "    "        "

In [4]: _dshort = {'core_sources': {'1drhotn':'source[identifier.name=radiation].profiles_1d[time].grid.rho_tor_norm',
   ...:                             '1deEnergy':'source[identifier.name=radiation].profiles_1d[time].electrons.energy'
   ...:                             },
   ...:            'equilibrium': {'2dpsi': 'time_slice[time].profiles_2d[0].psi',
   ...:                            '2dR': 'time_slice[time].profiles_2d[0].r',
   ...:                            '2dZ': 'time_slice[time].profiles_2d[0].z'}}

In [5]: multi.set_shortcuts(dshort=_dshort)

In [7]: plasma = multi.to_Plasma2D()

Then the following error message appears:

/home/ITER/munechk/.conda/envs/tofu2/lib/python3.6/site-packages/tofu/imas2tofu/_core.py:1912: UserWarning:
No / several matching signals for:
    - source[]['identifier', 'name'] = bremsstrahlung
    - nb.of matches: 0
        In ids core_sources, signal 1dbrem not loaded !
  warnings.warn(msg)
/home/ITER/munechk/.conda/envs/tofu2/lib/python3.6/site-packages/tofu/imas2tofu/_core.py:1912: UserWarning:
No / several matching signals for:
    - source[]['identifier', 'name'] = lineradiation
    - nb.of matches: 0
        In ids core_sources, signal 1dline not loaded !
  warnings.warn(msg)
/home/ITER/munechk/.conda/envs/tofu2/lib/python3.6/site-packages/tofu/imas2tofu/_core.py:1912: UserWarning:
'1dbrem'
        In ids core_sources, signal 1dprad not loaded !
  warnings.warn(msg)
/home/ITER/munechk/.conda/envs/tofu2/lib/python3.6/site-packages/tofu/imas2tofu/_core.py:1912: UserWarning:
No / several matching signals for:
    - source[]['identifier', 'name'] = lineradiation
    - nb.of matches: 0
        In ids core_sources, signal 1dpsi not loaded !
  warnings.warn(msg)
/home/ITER/munechk/.conda/envs/tofu2/lib/python3.6/site-packages/tofu/imas2tofu/_core.py:1912: UserWarning:
'1dpsi'
        In ids core_sources, signal 1drhopn not loaded !
  warnings.warn(msg)
/home/ITER/munechk/.conda/envs/tofu2/lib/python3.6/site-packages/tofu/imas2tofu/_core.py:1912: UserWarning:
list index out of range
        In ids equilibrium, signal 2dBT not loaded !
  warnings.warn(msg)
/home/ITER/munechk/.conda/envs/tofu2/lib/python3.6/site-packages/tofu/imas2tofu/_core.py:1912: UserWarning:
list index out of range
        In ids equilibrium, signal 2dBR not loaded !
  warnings.warn(msg)
/home/ITER/munechk/.conda/envs/tofu2/lib/python3.6/site-packages/tofu/imas2tofu/_core.py:1912: UserWarning:
list index out of range
        In ids equilibrium, signal 2dBZ not loaded !
  warnings.warn(msg)
/home/ITER/munechk/.conda/envs/tofu2/lib/python3.6/site-packages/tofu/imas2tofu/_core.py:1912: UserWarning:
'2dBT'
        In ids equilibrium, signal 2dB not loaded !
  warnings.warn(msg)
/home/ITER/munechk/.conda/envs/tofu2/lib/python3.6/site-packages/tofu/imas2tofu/_core.py:1912: UserWarning:
list index out of range
        In ids equilibrium, signal 2djT not loaded !
  warnings.warn(msg)
/home/ITER/munechk/.conda/envs/tofu2/lib/python3.6/site-packages/tofu/imas2tofu/_core.py:1912: UserWarning:
list index out of range
        In ids equilibrium, signal 2dmeshFaces not loaded !
  warnings.warn(msg)
/home/ITER/munechk/.conda/envs/tofu2/lib/python3.6/site-packages/tofu/imas2tofu/_core.py:1912: UserWarning:
list index out of range
        In ids equilibrium, signal 2dmeshNodes not loaded !
  warnings.warn(msg)
/home/ITER/munechk/.conda/envs/tofu2/lib/python3.6/site-packages/tofu/imas2tofu/_core.py:1912: UserWarning:
list index out of range
        In ids equilibrium, signal 2dphi not loaded !
  warnings.warn(msg)
/home/ITER/munechk/.conda/envs/tofu2/lib/python3.6/site-packages/tofu/imas2tofu/_core.py:1844: RuntimeWarning: invalid value encountered i                                                                      n greater
  out[ii][np.abs(out[ii]) > 1.e30] = np.nan
/home/ITER/munechk/.conda/envs/tofu2/lib/python3.6/site-packages/tofu/imas2tofu/_core.py:1912: UserWarning:
'2dphi'
        In ids equilibrium, signal 2drhotn not loaded !
  warnings.warn(msg)
/home/ITER/munechk/.conda/envs/tofu2/lib/python3.6/site-packages/tofu/imas2tofu/_core.py:1912: UserWarning:
'2dmeshNodes'
        In ids equilibrium, signal 2dtheta not loaded !
  warnings.warn(msg)
/home/ITER/munechk/.conda/envs/tofu2/lib/python3.6/site-packages/tofu/imas2tofu/_core.py:1912: UserWarning:
list index out of range
        In ids equilibrium, signal strike0R not loaded !
  warnings.warn(msg)
/home/ITER/munechk/.conda/envs/tofu2/lib/python3.6/site-packages/tofu/imas2tofu/_core.py:1912: UserWarning:
list index out of range
        In ids equilibrium, signal strike0Z not loaded !
  warnings.warn(msg)
/home/ITER/munechk/.conda/envs/tofu2/lib/python3.6/site-packages/tofu/imas2tofu/_core.py:1912: UserWarning:
'strike0R'
        In ids equilibrium, signal strike0 not loaded !
  warnings.warn(msg)
/home/ITER/munechk/.conda/envs/tofu2/lib/python3.6/site-packages/tofu/imas2tofu/_core.py:1912: UserWarning:
list index out of range
        In ids equilibrium, signal strike1R not loaded !
  warnings.warn(msg)
/home/ITER/munechk/.conda/envs/tofu2/lib/python3.6/site-packages/tofu/imas2tofu/_core.py:1912: UserWarning:
list index out of range
        In ids equilibrium, signal strike1Z not loaded !
  warnings.warn(msg)
/home/ITER/munechk/.conda/envs/tofu2/lib/python3.6/site-packages/tofu/imas2tofu/_core.py:1912: UserWarning:
'strike1R'
        In ids equilibrium, signal strike1 not loaded !
  warnings.warn(msg)
/home/ITER/munechk/.conda/envs/tofu2/lib/python3.6/site-packages/tofu/imas2tofu/_core.py:1912: UserWarning:
list index out of range
        In ids equilibrium, signal x0R not loaded !
  warnings.warn(msg)
/home/ITER/munechk/.conda/envs/tofu2/lib/python3.6/site-packages/tofu/imas2tofu/_core.py:1912: UserWarning:
list index out of range
        In ids equilibrium, signal x0Z not loaded !
  warnings.warn(msg)
/home/ITER/munechk/.conda/envs/tofu2/lib/python3.6/site-packages/tofu/imas2tofu/_core.py:1912: UserWarning:
'x0R'
        In ids equilibrium, signal x0 not loaded !
  warnings.warn(msg)
/home/ITER/munechk/.conda/envs/tofu2/lib/python3.6/site-packages/tofu/imas2tofu/_core.py:1912: UserWarning:
list index out of range
        In ids equilibrium, signal x1R not loaded !
  warnings.warn(msg)
/home/ITER/munechk/.conda/envs/tofu2/lib/python3.6/site-packages/tofu/imas2tofu/_core.py:1912: UserWarning:
list index out of range
        In ids equilibrium, signal x1Z not loaded !
  warnings.warn(msg)
/home/ITER/munechk/.conda/envs/tofu2/lib/python3.6/site-packages/tofu/imas2tofu/_core.py:1912: UserWarning:
'x1R'
        In ids equilibrium, signal x1 not loaded !
  warnings.warn(msg)
---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
<ipython-input-7-51881e127524> in <module>
----> 1 plasma = multi.to_Plasma2D()

~/.conda/envs/tofu2/lib/python3.6/site-packages/tofu/imas2tofu/_core.py in to_Plasma2D(self, tlim, dsig, t0, Name, occ, config, out, plot,                                                                       plot_sig, plot_X, bck, dextra, nan, pos)
   2657         if out == object or plot == True:
   2658             import tofu.data as tfd
-> 2659             plasma = tfd.Plasma2D( **plasma )
   2660             if plot == True:
   2661                 plasma.plot(plot_sig, X=plot_X, bck=bck)

~/.conda/envs/tofu2/lib/python3.6/site-packages/tofu/data/_core.py in __init__(self, dtime, dradius, d0d, d1d, d2d, dmesh, config, Id, Nam                                                                      e, Exp, shot, fromdict, SavePath, SavePath_Include)
   2271         del kwdargs['self']
   2272         # super()
-> 2273         super(Plasma2D,self).__init__(**kwdargs)
   2274
   2275     def _reset(self):

~/.conda/envs/tofu2/lib/python3.6/site-packages/tofu/utils.py in __init__(self, fromdict, **kwdargs)
   1279             self._reset()
   1280             self._set_Id(**kwdargs)
-> 1281             self._init(**kwdargs)
   1282         self._Done = True
   1283

~/.conda/envs/tofu2/lib/python3.6/site-packages/tofu/data/_core.py in _init(self, dtime, dradius, dmesh, d0d, d1d, d2d, config, **kwargs)
   2528         largs = self._get_largs_dgeom()
   2529         kwdgeom = self._extract_kwdargs(kwdargs, largs)
-> 2530         self._set_dindrefdatagroup(**kwdindrefdatagroup)
   2531         self.set_dgeom(**kwdgeom)
   2532         self._dstrip['strip'] = 0

~/.conda/envs/tofu2/lib/python3.6/site-packages/tofu/data/_core.py in _set_dindrefdatagroup(self, dtime, dradius, dmesh, d0d, d1d, d2d)
   2628                     assert len(lkt) == 1
   2629                     axist = dradius[k0]['depend'].index(lkt[0])
-> 2630                     size = data.shape[1-axist]
   2631                 dindref[k0] = {'size':size,
   2632                                'group':'radius'}

IndexError: tuple index out of range

The configuration:

And the loaded modules are:

[ munechk hpc-login04 ~ ] module list
Currently Loaded Modulefiles:
  1) GCCcore/6.4.0                                       16) ncurses/6.0-GCCcore-6.4.0                           31) Tk/8.6.8-GCCcore-6.4.0
  2) binutils/2.28-GCCcore-6.4.0                         17) libreadline/7.0-GCCcore-6.4.0                       32) Tkinter/3.6.4-intel-2018a-Python-3.6.4
  3) GCC/6.4.0-2.28                                      18) Tcl/8.6.8-GCCcore-6.4.0                             33) Szip/2.1.1-GCCcore-6.4.0
  4) icc/2018.1.163-GCC-6.4.0-2.28                       19) SQLite/3.21.0-GCCcore-6.4.0                         34) HDF5/1.10.1-intel-2018a
  5) ifort/2018.1.163-GCC-6.4.0-2.28                     20) XZ/5.2.3-GCCcore-6.4.0                              35) libxml2/2.9.7-GCCcore-6.4.0
  6) iccifort/2018.1.163-GCC-6.4.0-2.28                  21) GMP/6.1.2-GCCcore-6.4.0                             36) MDSplus/7.46.1-intel-2018a
  7) impi/2018.1.163-iccifort-2018.1.163-GCC-6.4.0-2.28  22) libffi/3.2.1-GCCcore-6.4.0                          37) MDSplus-Java/7.46.1-intel-2018a-Java-1.8.0_162
  8) iimpi/2018a                                         23) Python/3.6.4-intel-2018a                            38) Blitz++/0.10-GCCcore-6.4.0
  9) imkl/2018.1.163-iimpi-2018a                         24) libyaml/0.1.7-GCCcore-6.4.0                         39) libMemcached/1.0.18-GCCcore-6.4.0
 10) intel/2018a                                         25) PyYAML/3.12-intel-2018a-Python-3.6.4                40) Boost/1.66.0-GCCcore-6.4.0-no_mpi
 11) numactl/2.0.11-GCCcore-6.4.0                        26) libpng/1.6.34-GCCcore-6.4.0                         41) UDA/2.2.6-GCCcore-6.4.0
 12) PGI/18.4-GCC-6.4.0-2.28                             27) freetype/2.9-GCCcore-6.4.0                          42) ant/1.10.1-Java-1.8.0_162
 13) Java/1.8.0_162(1.8)                                 28) expat/2.2.5-GCCcore-6.4.0                           43) IDStools/1.2.2-Python-3.6.4
 14) bzip2/1.0.6-GCCcore-6.4.0                           29) fontconfig/2.12.6-GCCcore-6.4.0                     44) IMAS/3.24.0-4.1.5
 15) zlib/1.2.11-GCCcore-6.4.0                           30) X11/20180131-GCCcore-6.4.0                          45) Anaconda3/5.0.1
Didou09 commented 4 years ago

Error due to the fact tofu is trying to extract some radial profiles. It assumes that radial profiles are always 2d, with time shape (nt, nr) where:

But in this case, there is only one time step, and the extracted array is flat (1d of shape (nr,)), instead of being of shape (1, nr).

=> solution: will implement a safety check, with a more explicit error message

Didou09 commented 4 years ago

Should be fixed by PR #280 (available in 1.4.2 as soon as the Pull Request is validated and merged). See the PR for more details.

Please note, @munechika-koyo , that you should:

An example is provided below based on your needs:

In [1]: import tofu as tf
/home/ITER/vezined/ToFu_All/tofu/tofu/__init__.py:95: UserWarning: 
The following subpackages are not available:
    - tofu.mag
  => see tofu.dsub[<subpackage>] for details.
  warnings.warn(msg)

In [2]: tf.__version__
Out[2]: '1.4.2-alpha2-28-g806afc5'

In [3]: multi = tf.imas2tofu.MultiIDSLoader(user='public',tokamak='iter',shot=134000,run=28,ids=['core_sources', 'equilibrium'])
Getting ids   [occ]  tokamak  user    version  shot    run  refshot  refrun
------------  -----  -------  ------  -------  ------  ---  -------  ------
core_sources  [0]    iter     public  3        134000  28   -1       -1    
equilibrium   [0]    "        "       "        "       "    "        "     

In [4]: _dshort = {'core_sources': {'1drhotn':'source[identifier.name=radiation].profiles_1d[time].grid.rho_tor_norm',
   ...:    ...:                             '1deEnergy':'source[identifier.name=radiation].profiles_1d[time].electrons.energy'
   ...:    ...:                             },
   ...:    ...:            'equilibrium': {'2dpsi': 'time_slice[time].profiles_2d[0].psi',
   ...:    ...:                            '2dmeshR': 'time_slice[time].profiles_2d[0].r',
   ...:    ...:                            '2dmeshZ': 'time_slice[time].profiles_2d[0].z'}}
   ...:                            

In [5]: multi.set_shortcuts(dshort=_dshort)

In [6]: plasma = multi.to_Plasma2D(shapeRZ=('R', 'Z'))
/home/ITER/vezined/ToFu_All/tofu/tofu/imas2tofu/_core.py:484: RuntimeWarning: invalid value encountered in sqrt
  return np.sqrt( (psi - psi0[:,None]) / (psisep[:,None] - psi0[:,None]) )
/home/ITER/vezined/ToFu_All/tofu/tofu/imas2tofu/_core.py:1846: RuntimeWarning: invalid value encountered in greater
  out[ii][np.abs(out[ii]) > 1.e30] = np.nan
/home/ITER/vezined/ToFu_All/tofu/tofu/imas2tofu/_core.py:1962: UserWarning: The following data could not be retrieved:
    - equilibrium:
        2dB  : '2dBT'
        2dBR  : list index out of range
        2dBT  : list index out of range
        2dBZ  : list index out of range
        2djT  : list index out of range
        2dmeshFaces  : list index out of range
        2dmeshNodes  : list index out of range
        2dphi  : list index out of range
        2drhotn  : '2dphi'
        2dtheta  : '2dmeshNodes'
        strike0  : 'strike0R'
        strike0R  : list index out of range
        strike0Z  : list index out of range
        strike1  : 'strike1R'
        strike1R  : list index out of range
        strike1Z  : list index out of range
        x0  : 'x0R'
        x0R  : list index out of range
        x0Z  : list index out of range
        x1  : 'x1R'
        x1R  : list index out of range
        x1Z  : list index out of range
    - core_sources:
        1dbrem  : No / several matching signals for:     - source[]['identifier', 'name'] = bremsstrahlung     - nb.of matches: 0
        1dline  : No / several matching signals for:     - source[]['identifier', 'name'] = lineradiation     - nb.of matches: 0
        1dprad  : '1dbrem'
        1dpsi  : No / several matching signals for:     - source[]['identifier', 'name'] = lineradiation     - nb.of matches: 0
        1drhopn  : '1dpsi'
  warnings.warn(msg)

In [7]: npts = 100

In [8]: pts = np.array([np.linspace(4, 8, npts), np.zeros((npts)), np.zeros((npts,))])

In [9]: val, t = plasma.interp_pts2profile(pts, quant='2drhopn')

In [10]: val
Out[10]: 
array([[1.00000001, 1.00000001, 1.00000001, 1.00000001, 1.00000001,
        1.00000001, 1.00000001, 0.99519379, 0.9823119 , 0.9693463 ,
        0.95623018, 0.94737822, 0.93391692, 0.92022385, 0.90629477,
        0.89212509, 0.87770741, 0.86303376, 0.84809764, 0.83799228,
        0.82261002, 0.8069563 , 0.79102977, 0.77483118, 0.75836272,
        0.74162666, 0.72462484, 0.71314399, 0.69570641, 0.67801283,
        0.66006675, 0.64187223, 0.62343606, 0.60476769, 0.58587839,
        0.57316944, 0.55394466, 0.53454563, 0.51499928, 0.4953385 ,
        0.47560473, 0.45585194, 0.44270668, 0.42308562, 0.40366962,
        0.3845765 , 0.36591491, 0.34778222, 0.33029443, 0.31358574,
        0.30295342, 0.28788942, 0.27405435, 0.26165837, 0.25093208,
        0.2421167 , 0.23544763, 0.23113242, 0.2296427 , 0.22955887,
        0.23207063, 0.23711526, 0.2445543 , 0.25419447, 0.26581205,
        0.27917483, 0.28894178, 0.30472475, 0.32169771, 0.33968939,
        0.35854621, 0.37813249, 0.39833154, 0.41905435, 0.43309765,
        0.45441556, 0.47594585, 0.497604  , 0.51932443, 0.54105608,
        0.56275732, 0.58439388, 0.59876878, 0.62024028, 0.64158581,
        0.66278883, 0.68383636, 0.7047192 , 0.72542845, 0.74595545,
        0.7595356 , 0.77974593, 0.79976103, 0.81957919, 0.83920331,
        0.85864043, 0.87790024, 0.89699693, 0.9096461 , 0.92851068]])