SurfaceLab7 .itax & .itm file support

[veeezy]

Hi PySPM Team,

Thank you for this useful approach to analysing SIMS data.

I am a new user of python and surface lab, so I apologise if there are any obvious oversights from me.

The method works great for studying .ita files from SurfaceLab 6 however I am having difficulty opening .itax and .itm files from Surfacelab 7

Error for opening surfacelab7 .itm file UnicodeDecodeError: 'ascii' codec can't decode byte 0xc2 in position 39: ordinal not in range(128)

If possible please could you inform me if there is support for IonToF Surfacelab 7 file formats in particular processing of .itax and .itm file formats.

Many thanks,

V

[scholi]

Hi, If someone wish to integrate the support for SurfaceLab7 .itax file, he is welcome. I don't have access to such file and cannot do the integration myself, but I can support a developper that knows python to guide him how the surfacelab6 fileformat work.

You can start also by posting the full error (which tell which function generate the error) and the code you execute that create the error.

[teolombardo]

I would be very interested to use pySPM with .itax files. Did anything happen from the creation of this issue (any programmer working on this topic or similar)?

Thanks in advance for the update,

Teo Lombardo

[scholi]

One can open the file manualy and start to analyse it. Here is an example with the data toy ITM version 6, not 7

import pySPM
from pySPM_data import get_data

f=open(get_data("BigSmiley.itm"),"rb")
print("Header",f.read(8)) # Skip header
B = pySPM.Block.Block(f)
B.show()

That will print a huge list of the datastructure inside the ITM. Something like:

Header b'ITStrF01'
versionheader (0) @567
    Footprint (0) @870
    Version (0) @906
StorageInfo (0) @940
    Type (0) @1506
    VersionMajor (0) @1571
    VersionMinor (0) @1609
Analyses (0) @1647
LateralShiftCorrection (0) @4330
MassScale (0) @7027
    clsid (0) @203693
    sf (0) @203739
    k0 (0) @203774
    polarity (0) @203809
    channelwidth (0) @203846
    establ (0) @203891
    calibtype (0) @203923
    calib (0) @203961
        assign (0) @205051
        mcp (0) @205086
    totalshots (0) @211661
Meta (0) @9711
    SI Image (0) @301624248
        clsid (0) @301624811
        zoomfactor (0) @301624857
        zoomcenter_x (0) @301624900
        zoomcenter_y (0) @301624945
        stageposition_x (0) @301624990
        stageposition_y (0) @301625038
        stageposition_z (0) @301625086
        stageposition_r (0) @301625134
        stageposition_t (0) @301625182
        fieldofview (0) @301625230
        resolution (0) @301625274
        shotsperpixel (0) @301625886
        res_x (0) @301625928
        res_y (0) @301625962
        bpp (0) @301625996
        description (0) @301626028
        name (0) @301626080
        intensdata (0) @301626109
        comment (0) @301976323

If one wants to read one of the information, for e.g. "Meta/SI Image we can get it with:

    v = B.goto("Meta/SI Image").show_list()

This will print something like:

List of 19
clsid (0) <16> @301624811, value = b'be8a9975fb391248...' (hex) = "誾疙㧻䠒皓ꄢ" (UTF-16)= ??? (???)
zoomfactor (0) <8> @301624857, value = b'0000000000001440' (hex) = "䀔" (UTF-16)= 5.0 (double) = 4617315517961601024 (long64)
zoomcenter_x (0) <8> @301624900, value = b'0000000000000000' (hex) = "" (UTF-16)= 0.0 (double) = 0 (long64)
zoomcenter_y (0) <8> @301624945, value = b'0000000000000000' (hex) = "" (UTF-16)= 0.0 (double) = 0 (long64)
stageposition_x (0) <8> @301624990, value = b'5d830f6b58103540' (hex) = "荝欏ၘ䀵" (UTF-16)= 21.063849154746425 (double) = 4626622164132397917 (long64)
stageposition_y (0) <8> @301625038, value = b'72057f4525b92240' (hex) = "ղ䕿뤥䀢" (UTF-16)= 9.361612483745123 (double) = 4621459737366431090 (long64)
stageposition_z (0) <8> @301625086, value = b'f6ec05bb9bf9dfbf' (hex) = "묅鍊뿟" (UTF-16)= -0.4996098829648895 (double) = -4620700245405864714 (long64)
stageposition_r (0) <8> @301625134, value = b'382bcf98a01d813f' (hex) = "⬸飏ᶠ㾁" (UTF-16)= 0.008357290900581668 (double) = 4575971271980821304 (long64)
stageposition_t (0) <8> @301625182, value = b'bcc1e3b07c4c70bf' (hex) = "솼냣䱼뽰" (UTF-16)= -0.003979193758127438 (double) = -4652134316646350404 (long64)
fieldofview (0) <8> @301625230, value = b'2d431cebe2361a3f' (hex) = "䌭㛢㼚" (UTF-16)= 0.0001 (double) = 4547007122018943789 (long64)
resolution (0) <4> @301625274, value = b'00020000' (hex) = "Ȁ" (UTF-16)= 512 (long)
shotsperpixel (0) <4> @301625886, value = b'01000000' (hex) = "
" (UTF-16)= 1 (long)
res_x (0) <4> @301625928, value = b'00020000' (hex) = "Ȁ" (UTF-16)= 512 (long)
res_y (0) <4> @301625962, value = b'00020000' (hex) = "Ȁ" (UTF-16)= 512 (long)
bpp (0) <4> @301625996, value = b'20000000' (hex) = " " (UTF-16)= 32 (long)
description (0) <16> @301626028, value = b'54006f0074006100...' (hex) = "Total SI" (UTF-16)= ??? (???)
name (0) <0> @301626080, value = b'' (hex) = "" (UTF-16)= ??? (???)
intensdata (0) [128] <350179> @301626109
comment (0) <44> @301976323, value = b'54006f0074006100...' (hex) = "Total SI (Measuremen..." (UTF-16)= ??? (???)

As the cell don't tell which is the encoding all various encoding is shown to the user (hex value, UTF-16 string and long). We can see here that the image was recorded with 512×512 pixels (res_x × res_y)

You can start with this analysis to understand the new fileformat which really close to the old one

[teolombardo]

Thanks for the answer! This procedure works also with my data, but from this can I also get information on the spectra and associated peaks images? And even more, perform analysis like PCA?

Just as a matter of example, following your procedure I tried the add the command:

"B.show_masses()"

And I got the error:

"AttributeError: 'Block' object has no attribute 'show_masses'"

That kind of makes sense of course, but it also makes me think that this procedure allows getting only the instrumental information (like stage position, etc.). Is there a way to access the data associated with the measurement by reading the .itm file? (data that should also be contained in the .itm file, as far as I know. But I could be wrong on this).

Thanks a lot in advance for your answer and your help with this :)

Teo

[scholi]

OK, so you should not mix ITM/ITA and Block classes. The Block class let you read the raw data inide ITA, ITM and ITAX files. Now the ITA and ITM classes have a lot of methods to help you to get the valuable informations without having to dive into the raw format.

Now I remember that there is a draft for ITAX file that I created long time ago (see https://github.com/scholi/pySPM/blob/master/pySPM/ITAX.py). You can start to play around with it and test if get_profile works. I have no itx file so I cannot test.

[teolombardo]

Yeah, I had the impression that block class was not the right way to go for me.

I have just given a test with a .itax file, but it does not seem to work, unfortunately.

Here the simple script I have used:

AI = pySPM.ITAX("test.itax") AI.getSpectrum()

And I get the following error:

error Traceback (most recent call last) Input In [19], in <cell line: 1>() ----> 1 AI.getSpectrum()

File ~\AppData\Local\Programs\Python\Python310\lib\site-packages\pySPM\ITAX.py:57, in ITAX.getSpectrum(self, sf, k0, time, *kargs) 55 slen = self.root.goto("CommonDataObjects/DataViewCollection//sizeSpectrum").getLong() 56 raw = self.root.goto("CommonDataObjects/DataViewCollection//dataSource/simsDataCache/spectrum/correctedData").value ---> 57 spectrum = np.array(struct.unpack("<"+str(slen)+"d", raw)) 58 CH = 2np.arange(slen)
59 if time:

error: unpack requires a buffer of 8000000 bytes

If you are still willing to give it a try (I refer to your message in the other issue), I can send you an example of .itm and .itax files obtained by SurfaceLab7, for you to see if there is a procedure to extract/analyse the spectra/images into those files through PySPM, or not. It would be very very useful :)

Thanks again in advance!

Teo

[scholi]

It seems that I missed your answer.... Sorry. Could you manage to read ITAX files? Do you have some advises to share for the other users?

[teolombardo]

Nope, unfortunately I did not manage to read ITAX files.