fluorescence spectra in files for RSM3D

[prjemian]

• From: "Zhang, Zhan" ZhanZhang@anl.gov
• Date: 6/12/18 3:09 PM (GMT-06:00)
• To: "Jemian, Pete R." jemian@anl.gov, "Hammonds, John P." jphammonds@anl.gov (@JPHammonds )
• Subject: Reading spec file in RSM3D

Hi Pete & John,

Last week we have a user doing sort of a powder scan. When I tried to analyze the data with RSM3D, I was surprised to saw an error message. It took me a while to realize that the reason was that I have fluorescence spectrum saved within the spec file (ESRF standard?) with a ‘@’ leading the line and breaking the whole spectrum (2048 channels) into 64 lines of 32 numbers.

Pymca handles that properly, of course. I did not realize that the reader we are using in RSM3D consider that an ‘incomplete’ scan and could not index the scan numbers properly as soon as such data appears in a data file. Could that capability be included?

Thanks!

Zhan

[prjemian]

example file in commit 527507d1540c3e006b0997e6a243c2f1049820bc

[prjemian]

looking at the file "startup_1.spec", it shows (in scan 16) there are lines such as these:

#UXML </group>
#@SCA0    vorsca1    -1    0
#@SCA1    vorsca2    -2    0
#@SCA2    vorsca3    -3    0
#@SCA3    vortrlt    -109    0
#@SCA4    vorcalt    -101    0
#@SCA5    vorrt    -102    0
#@SCA6    voricr    -103    0
#@SCA7    vorocr    -104    0
#@SCA8    vortrig    -105    0
#@SCA9    vorevnt    -106    0
#@SCA10    vorovrf    -107    0
#@SCA11    vorundf    -108    0
@0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\

spec2nexus does not yet have a plugin to support these lines. Won't be hard to add but that's why it has problems reading the SPEC file.

The comparable handler already in spec2nexus is for MCA data which uses similar but not identical line. The @ data lines might present some trouble since they have a more generic pattern match than the @A (tha added letter is a big help) lines used by the MCA support. There's conflict only if you use both SCA & MCA in a scan. Keep fingers crossed.

[prjemian]

@JPHammonds : Is there anything to be parsed from the #SCAn lines? The @ lines would be easier (unique) to parse if they had a letter and that it was other than A as used by the MCA plugin.

[prjemian]

What about the existing files?

• [ ] don't change them and don't try to read them with spec2nexus?
• [ ] change them to "@"

Pete

On 2018-06-13 11:10 AM, Zhang, Zhan wrote:
> The change is easy to make from my end.  Hope I get it right this time.  Thanks!
> Zhan
>
> -----Original Message-----
> From: Jemian, Pete R.
> Sent: Wednesday, June 13, 2018 10:37 AM
> To: Zhang, Zhan <ZhanZhang@anl.gov>; Hammonds, John P. <jphammonds@anl.gov>
> Subject: Re: Reading spec file in RSM3D
>
> SPEC control lines info : not so easy to find dug through the standard macros (mostly) for this
>
> My docs here:
> http://spec2nexus.readthedocs.io/en/latest/supplied_plugins/spec_common_spec2nexus.html#spec2nexus.plugins.spec_common_spec2nexus.SPEC_MCA_Array
>
> SPEC's docs not so informative
> https://certif.com/spec_manual/user_1_4_1.html
> Might be in there somewhere but my attention is elsewhere and patience is not so good now
>
> Can you switch to "@A"?  Handling for that already exists.
>
> Pete
>
> On 2018-06-13 10:25 AM, Zhang, Zhan wrote:
>> Hi Pete,
>>     I assume that the lines started with '#' would be safely ignored.  These are my add-ons for recording the spectrum info when needed.  It was impletemented quite some time ago per SPEC's suggestion, I think.  But I am not so sure now.  I switched to another route last week by saving the spectrum to an external file. Keeping the data together would be preferred but the work-around is ok, too.
>>     Thanks!
>>     I am not aware the MCA option you mentioned.  What's its format?
>> Zhan

[zhanzhang3]

I'd say don't do anything. I don't have many file with MCA's in it and they don't need to be read by spec2nexus, I think. I did a bit google search and the only thing I can find now about the MCA format in SPEC file is from this specpy page. But on the same page there are two statement:

1. Special data lines containing MCA data begin with an @ character followed by a row of numbers.
2. Data in the lines (specila data line) starting with @A will contain the MCA data.

I am not sure how the two cases used. Apparently I simply used the first one and never bother checked. Now that there are some other information there, I can update my macro to be more compatible.
Thanks!

Zhan

[prjemian]

@vasole, @vrey01 : Any comment on how to store spectra in SPEC files? This page provides this info:

Special data lines containing MCA data begin with an @ character followed by a row of numbers.

Data in the lines starting with @A will be stuffed into the y (2D) or z (3D) data array.

Is it just a typo in the first quote that should read @A?

[vasole]

For the data is @A

I am positively surprised by PyMca handling properly 32 numbers per line (at the ESRF we only use 16)

[vasole]

For the special information in the scan header, the line starts by #@ and the use of #@ followed by new words (different from CHANN, CALIB, CTIME) as your user seems to be introducing can only lead to problems with existing software.

If you need user defined entries in the header, it is advisable that you use #U or #C followed by whatever you may want instead of #@

[zhanzhang3]

@vasole Yeah, PyMca could be slow but it did manage to pull the data out and plot it, even though we used '@' instead of '@A' to lead the spectrum and with 32 integers per line. Going forward, I will change the header lines. Hopefully that would make things smoother. Thanks!

[vasole]

PyMca slow???? Fitting 40000 spectra per second on a four years old CPU is not something I would consider slow.

[zhanzhang3]

Oh, it should be 'slowed down'. I meant it got choked on the data I had. I think the wrong format took its toll. In spite of that, the spectra was loaded and plotted out.

[prjemian]

@vasole : Thanks for your comments. I'll mark this as not a problem to be fixed and close the issue.