search

MIT-PSFC / disruption-py

An open-source physics-based Scientific Framework for Disruption Analysis of Fusion Plasmas for AI/ML applications
https://mit-psfc.github.io/disruption-py/
MIT License
11 stars 0 forks source link

DIII-D unaccessible PTDATA pointnames #261

Open yumouwei opened 3 months ago

yumouwei commented 3 months ago

Description of the problem

The get_n1_bradial_parameters tries to access PTDATA DUSBRADIAL and (if it fails) ONSBRADIAL to fetch data for its computation. Currently both of these get data calls fail. https://github.com/MIT-PSFC/disruption-py/blob/7284810809e9347b7244d3b4cab1b7dcaf0e7115/disruption_py/machine/d3d/physics.py#L705-L725

Corresponding lines in the MATLAB script https://github.com/MIT-PSFC/disruption-py/blob/5a13906aba52b22fc4119e7f867443d7936f3d03/DIII-D/get_n1_bradial_d3d.m#L84-L99

I also tried checking these pointnames in ReviewPlus but it gave ERROR %TREE-W-NNF, Node Not Found error as well. image

Also, on Omega I tried to check jTraverser by using tree name = "d3d" or "efit01" but neither of them worked. What's the proper tree name I should use?

gtrevisan commented 3 months ago

watch out, PTDATA is not MDSplus, although I believe lots of the queries are made to be MDSplus-compatible. have you tried treename="PTDATA"?

yumouwei commented 3 months ago

params.mds_conn.get_data_with_dims(f"ptdata('dusbradial', {params.shot_id})",tree_name="ptdata") failed as well.

gtrevisan commented 3 months ago

have you tried with normal python and mdsplus calls? outside of disrpy?

yumouwei commented 3 months ago

Doesn't seem to work either

import MDSplus as mds

pointname = 'onsbradial'   # MDSplus.mdsExceptions.TdiABORT: %TDI-E-ABORT, Program requested abort
shotno = 161228

conn = mds.Connection('atlas.gat.com')
# conn.openTree('PTDATA', shotno)   # TREE-E-FOPENR, Error opening file read-only
conn.openTree('d3d', shotno)

data = conn.get(f"ptdata('{pointname}', {shotno})").data()
time = conn.get("dim_of("+f"ptdata('{pointname}', {shotno})"+")").data()

conn.closeAllTrees()
conn.disconnect()

print(f"Time: {time}")
print(f"data: {data}")

Setting pointname = 'ip' succeeded so the MDSplus calls should be correct.

gtrevisan commented 3 months ago

could you also try with treename=None?

yumouwei commented 3 months ago

I tried None, 'efit01' & '_efit_tree' but neither worked.

yumouwei commented 3 months ago

Cannot find PTDATA point names dpsradcva and dpsradxdiv

yumouwei commented 2 months ago

Get_n1_bradial_parameters

There are 4 potential data sources for the bradial (locked mode) signal. These are:

  1. DUSBRADIAL, which points to the DUD legacy warning algorithm ("category") in the PCS.
  2. ONSBRADIAL, which points to the more recent ONFR system in the PCS. In general this point name should be checked first before checking DUSBRADIAL.
  3. On IRIS, the /fusion/projects/disruption_warning/software/recalc_bradial/bradial/... folder (and the compressed bradial.tar.gz file). This folder stores data for shots 156199--172430 (3814 shots taken between 2014--2017) which were missing in the "standard" atlas tree path and were recomputed by B. Sammuli. The data are stored as MDS tree structures by shot.
  4. On IRIS, /fusion/projects/disruption_warning/software/recalc_bradial/recalc.nc. This NetCDF file stores data for shots 176030--176912 (880 shots) that were recalculated also by B. Sammuli due to an error in hardware calibration.
Shot range Data source Tested shot
--156198 Check ONSBRADIAL -> DUSBRADIAL 156100: DUSBRADIAL available, ONSBRADIAL unavailable.
156199--172430 Get data from custom tree structures (3) Note: all 4 testing shots are within this range.
172431--176029 Check ONSBRADIAL -> DUSBRADIAL 172435: DUSBRADIAL available, ONSBRADIAL unavailable.
176030--176912 Read data from recalc.nc (4)
176913-- Check ONSBRADIAL -> DUSBRADIAL 176913: ONSBRADIAL available.

TODO:

Bolometer peaking factors

The two point names DPSRRDCVA & DPSRRDXDIV (be aware of the typo in the signal names) point to the DPRF (Disruption Prediction via Random Forest) module in the PCS. These signals are archived as the data source of the DPRF algorithm. The DPRF module was implemented around 2019, therefore only data after that date would be available. We tested shot 180808 and it worked.

The DPRF module has not been actively maintained. For this reason it would be more preferable to compute these two signals natively in disruption-py (fixed in #265) instead of relying on the stored data.

gtrevisan commented 2 months ago

what's the relationship between Brian's two netcdf files, and this one already in the repo? https://github.com/MIT-PSFC/disruption-py/blob/3c81c17f7783ed3c32c422cc370447d0cadc8116/disruption_py/machine/d3d/physics.py#L845-L848 EDIT: cannot read that path, but I do find /fusion/projects/disruption_warning/media/recalc.nc

yumouwei commented 2 months ago

I don't have permission to access /media so I can't check, but I assume it's the same as (4) because it covers the same shot numbers

(3) is not a netcdf file but a folder of MDS data structures. This range of shot numbers was mentioned in the MATLAB script but isn't included in the function's logic, so I assume it was done manually some time afterward. For us the problem is how to read these files because of that thick vs thin clients problem.

image
gtrevisan commented 2 months ago

hmm, okay. I don't see a quick way to implement this fallback in a nice, robust, and sustainable manner -- we'll have to think about it some more.

just FYI, it seems that these two files are different:

cd /fusion/projects/disruption_warning
ls -l {media,software/recalc_bradial}/recalc.nc
[snip] 3351859235 Aug 20  2018 media/recalc.nc
[snip] 3351848992 May  3  2018 software/recalc_bradial/recalc.nc