Add more general parser for fort.44 files which include labels

[jacklovell]

Versions of fort.44 >= 20160829 have a comment line before each data block which includes the name of the field and the size of the data block. This makes the file close enough to self-describing that the block can be parsed without knowledge of the exact format. This should enable greater flexibility for the number of formats we support.

This change means we essentially have two types of parsers, unlabelled and labelled. The unlabelled ones are further split into pre-2007 and 2013 at the moment, due to non-self-describing differences in these formats. But the availability of at least a couple of general parsers leads to a reworking of the logic to choose a parser: it's no longer tied to the exact version but is more of a boundary match.

This has been tested on fort.44 files with versions 20160829, 20170328 and 20180323 for SOLPS simulations carried out in DIII-D, ITER and AUG geometries. I also tested it on a WEST simulation which only included He atoms, and it failed loudly because of missing dmb2 etc. entries: I think this is a good thing compared with silently reading garbage. I'm in 2 minds as to whether to add extra complexity to deal with this: should we automatically populate missing fields with zeros, or error out on the basis that these files do not comply with the specification in the SOLPS manual for fort.44? I think it's fair to say we only support compliant files for the purposes of code simplicity.

Here is a script which can be used for quickly plotting all the quantities we read into the Eirene object, to help with testing other files. It requires the b2fstate, b2fgmtry and fort.44 files to all live in the same directory (so some copying may be required for runs which reference a baserun).

@Mateasek @vsnever could you check that this works with suitable SOLPS outputs you have access to? Other suggestions for improvement are also welcome.

[vsnever]

I also tested it on a WEST simulation which only included He atoms, and it failed loudly because of missing dmb2 etc. entries: I think this is a good thing compared with silently reading garbage. I'm in 2 minds as to whether to add extra complexity to deal with this: should we automatically populate missing fields with zeros, or error out on the basis that these files do not comply with the specification in the SOLPS manual for fort.44?

I don't think that processing the case when the number of molecules nm is 0 adds any complexity, you can just add if statement each time you assign the molecular data:

if eirene.nm:
    eirene.dm = raw_data['dmb2'].reshape(eirene.nm, eirene.ny, eirene.nx)

Actually, the same should be done for the parser of unlabelled files. I forgot that there are cases without molecules.

[jacklovell]

I also tested it on a WEST simulation which only included He atoms, and it failed loudly because of missing dmb2 etc. entries: I think this is a good thing compared with silently reading garbage. I'm in 2 minds as to whether to add extra complexity to deal with this: should we automatically populate missing fields with zeros, or error out on the basis that these files do not comply with the specification in the SOLPS manual for fort.44?

I don't think that processing the case when the number of molecules nm is 0 adds any complexity, you can just add if statement each time you assign the molecular data:

if eirene.nm:
    eirene.dm = raw_data['dmb2'].reshape(eirene.nm, eirene.ny, eirene.nx)

Actually, the same should be done for the parser of unlabelled files. I forgot that there are cases without molecules.

Is it also possible to have cases without atoms or ions too? Perhaps all assignments should be checked with if eirene.na, if eirene.nm or if eirene.ni as appropriate.

[vsnever]

Is it also possible to have cases without atoms or ions too? Perhaps all assignments should be checked with if eirene.na, if eirene.nm or if eirene.ni as appropriate.

I thought about this too. Even if a simulation without atoms or ions is technically possible, it will not be a plasma simulation. So I do not think we should care about these cases because they are too exotic (if even exist).

[jacklovell]

Is it also possible to have cases without atoms or ions too? Perhaps all assignments should be checked with if eirene.na, if eirene.nm or if eirene.ni as appropriate.

I thought about this too. Even if a simulation without atoms or ions is technically possible, it will not be a plasma simulation. So I do not think we should care about these cases because they are too exotic (if even exist).

I've found a run which just uses He: the fort.44 file has na=1, nm=0, ni=0. There will still be He-II and He-III in the B2 plasma though, just like in the generomak example the plasma has all 7 carbon ionisation stages even though the fort.44 file does not list C among the ions (only H2+ is included in fort.44).

So it's perhaps not unreasonable to have these fort.44s without any molecules or ions: molecules may simply not be present in the SOLPS simulation at all, while the only ionised species will be in the B2 data. And B2 treats ions self-consistently, whereas its fluid-neutrals model is not self consistent which is why we need the Eirene data in the first place.

[vsnever]

I've found a run which just uses He: the fort.44 file has na=1, nm=0, ni=0. There will still be He-II and He-III in the B2 plasma though, just like in the generomak example the plasma has all 7 carbon ionisation stages even though the fort.44 file does not list C among the ions (only H2+ is included in fort.44).

So it's perhaps not unreasonable to have these fort.44s without any molecules or ions: molecules may simply not be present in the SOLPS simulation at all, while the only ionised species will be in the B2 data. And B2 treats ions self-consistently, whereas its fluid-neutrals model is not self consistent which is why we need the Eirene data in the first place.

I see. In terms of Eirene, the ions are the molecular ions. I forgot about that, sorry. So, yes, checking for ni == 0 is necessary, it's not an exotic case at all.

[jacklovell]

Is it also possible to have cases without atoms or ions too? Perhaps all assignments should be checked with if eirene.na, if eirene.nm or if eirene.ni as appropriate.

I thought about this too. Even if a simulation without atoms or ions is technically possible, it will not be a plasma simulation. So I do not think we should care about these cases because they are too exotic (if even exist).

I've found a run which just uses He: the fort.44 file has na=1, nm=0, ni=0. There will still be He-II and He-III in the B2 plasma though, just like in the generomak example the plasma has all 7 carbon ionisation stages even though the fort.44 file does not list C among the ions (only H2+ is included in fort.44).

So it's perhaps not unreasonable to have these fort.44s without any molecules or ions: molecules may simply not be present in the SOLPS simulation at all, while the only ionised species will be in the B2 data. And B2 treats ions self-consistently, whereas its fluid-neutrals model is not self consistent which is why we need the Eirene data in the first place.

[jacklovell]

I've pushed a couple more changes, and now the code can deal with simulations with no molecules or ions. I've tested it on a He-only WEST simulation and it runs without error and produces reasonable-looking plots.

[vsnever]

Just tested this parser against the entire set of SOLPS-ITER example cases, and it works fine with all of them. However, in all these cases, the fort.44 files have the same version, 20170328.

[jacklovell]

OK, great. Once @Mateasek has confirmed he's happy with it I'll merge.

[Mateasek]

Hi @jacklovell and @vsnever , sorry for my late reply. I forgot about this PR. I looked into it and I have no suggestions, I think your knowledge about the sopls fomats is deeper than mine anyway. I tested loading of 20180323 and 20170328 fort.44 files and it seems it loaded both correctly, however it is difficult for me to check thoroughly. If there are no other matters you would like me to check with these two versions I think this PR should be accepted.