Getting density, power, and magnetics columns for cmod fails for large shotlist

[andrew-maris]

When attempting to get data using the cmod_adm branch for this shotlist: cmod_non_disruptions_ids_not_blacklist.txt

I find that most shots have missing columns related to density, power, and magnetics. These columns appear consistently for the first 83 shots, but it suddenly becomes inconsistent after that. For example, shot 1160913022 has all columns (except for peaking, which is expected), while 1160913023 is missing density, power, and magnetics columns. When I run just shot 1160913023, however, all the columns (except for peaking, as expected) are successfully grabbed.

These groups of columns are only sporadically available after 1160913022.

The .csv file and logbook files are too big to attach here, but I'll send them to you. The error message is "MDSplus.mdsExceptions.TreeFOPENR: %TREE-E-FOPENR, Error opening file read-only." This is thrown when it tries to grab the magnetics or electrons tree.

I will note that something similar happened when I tried this on the 17th. The first 70ish shots had all columns available but after that the presence became spotty.

For the record, I will run this again today to see if I can replicate the problem again. In the meantime, it would be great if you could try this out on your own and let me know what you think the issue is.

[hturner08]

Some short snippets from the log file as examples: [Shot 1160505014]:Generated shot object, 628 of 3148 (19.9% percent complete)' [Shot 1160505015]:Populating _get_EFIT_parameters [Shot 1160505015]:Populating _get_H98 [Shot 1160505015]:Failed to populate _get_H98 Traceback (most recent call last): File "/home/maris/disruption-py/disruption_py/shots/shot.py", line 213, in populate_method return method() File "/home/maris/disruption-py/disruption_py/shots/shot.py", line 31, in wrapper result = func(*args, kwargs) File "/home/maris/disruption-py/disruption_py/shots/cmod_shot.py", line 1504, in _get_H98 powers_df = self._get_power() File "/home/maris/disruption-py/disruption_py/shots/shot.py", line 31, in wrapper result = func(*args, *kwargs) File "/home/maris/disruption-py/disruption_py/shots/cmod_shot.py", line 544, in _get_power p_oh = self._get_ohmic_parameters()['p_oh'] File "/home/maris/disruption-py/disruption_py/shots/shot.py", line 31, in wrapper result = func(args, kwargs) File "/home/maris/disruption-py/disruption_py/shots/cmod_shot.py", line 495, in _get_ohmic_parameters ip_parameters = self._get_ip_parameters() File "/home/maris/disruption-py/disruption_py/shots/shot.py", line 31, in wrapper result = func(*args, **kwargs) File "/home/maris/disruption-py/disruption_py/shots/cmod_shot.py", line 262, in _get_ip_parameters magnetics_tree = Tree('magnetics', self._shot_id) File "/usr/local/mdsplus/python/MDSplus/tree.py", line 569, in init self.open(mode) File "/usr/local/mdsplus/python/MDSplus/tree.py", line 541, in open _exc.checkStatus(status) File "/usr/local/mdsplus/python/MDSplus/mdsExceptions.py", line 94, in checkStatus raise exception MDSplus.mdsExceptions.TreeFOPENR: %TREE-E-FOPENR, Error opening file read-only.

[Shot 1160505015]:Populating _get_Ts_parameters [Shot 1160505015]:Failed to populate _get_Ts_parameters Traceback (most recent call last): File "/home/maris/disruption-py/disruption_py/shots/shot.py", line 213, in populate_method return method() File "/home/maris/disruption-py/disruption_py/shots/shot.py", line 31, in wrapper result = func(*args, **kwargs) File "/home/maris/disruption-py/disruption_py/shots/cmod_shot.py", line 848, in _get_Ts_parameters electron_tree = Tree("electrons", self._shot_id) File "/usr/local/mdsplus/python/MDSplus/tree.py", line 569, in init self.open(mode) File "/usr/local/mdsplus/python/MDSplus/tree.py", line 541, in open _exc.checkStatus(status) File "/usr/local/mdsplus/python/MDSplus/mdsExceptions.py", line 94, in checkStatus raise exception MDSplus.mdsExceptions.TreeFOPENR: %TREE-E-FOPENR, Error opening file read-only.

[Shot 1160505015]:Populating _get_densities [Shot 1160505015] %TREE-E-FOPENR, Error opening file read-only. [Shot 1160505015] No density data [Shot 1160505015]:Failed to populate _get_densities Traceback (most recent call last): File "/home/maris/disruption-py/disruption_py/shots/cmod_shot.py", line 791, in _get_densities e_tree = Tree('electrons', self._shot_id) File "/usr/local/mdsplus/python/MDSplus/tree.py", line 569, in init self.open(mode) File "/usr/local/mdsplus/python/MDSplus/tree.py", line 541, in open _exc.checkStatus(status) File "/usr/local/mdsplus/python/MDSplus/mdsExceptions.py", line 94, in checkStatus raise exception MDSplus.mdsExceptions.TreeFOPENR: %TREE-E-FOPENR, Error opening file read-only.

During handling of the above exception, another exception occurred:

[andrew-maris]

I have tried this again after starting a new branch (andrew-maris/cmod) from the most recent version of main. A similar problem has emerged: all shots after 1140605004 (including 1140605006 which comes right after) are missing all the density, magnetics, edge_quantities, plasma current, and power columns. Looking at the logs, these all fail due being unable to access either the magnetics or electrons tree. It doesn't have a problem accessing any of the other trees (such as the efit tree), which seems to be and important clue...

@hturner08 did you ever hear back from the MDSplus team? I'm curious if you think we should try wrapping the script in some sort of loop as you suggested or if we can figure out

I will attach an abridge version of the logs because the full logs are too long. 230809_cmod_stable.log

[mwinkel-dev]

Hi @andrew-maris -- I'm a member of the MDSplus software team. Back on August 1, @hturner08 sent a Slack message to me about this disruptions-py issue #33. Unfortunately, I got busy with some priority MDSplus issues filed by DIII-D, so dropped the ball on this disruptions-py issue.

Today, I will do a quick investigation to see if the root cause is an MDSplus issue. And if so, will see to it that an issue is submitted to the MDSplus GitHub.