TeaxGrabber with ThermalGrabber USB outputs a scrambled image

[navotoz]

Hello, First , thanks for all the work! Really nice package.

I'm trying to grab images with the Tau2 with ThermalCapture Grabber USB. I cloned the latest version of flirpy (0.2.3), setup a venv, installed using requirements.txt and tried running:

from flirpy.camera.tau import TeaxGrabber

camera = TeaxGrabber()
image = camera.grab()
camera.close()
np.save('test', image)

The resultent image is scrambled. I think this is sync issue of some sort. When I use ThermalCaputre Viewer the output is OK, so I'm positive the camera and grabber are working.

I attached the output of the code. When looking in the ThermalCaputre Viewer it can be seen perfectly. test_TeaxGrabber_0.2.3.zip

Thank you again, with regards, Navot

[jveitchmichaelis]

Heya, yes there are sometimes some sync problems. Particularly if you mix commands and streaming. I have some ideas how to solve it, but I think it may occur when the code that looks for the start of a frame goes completely out of phase (e.g. it never finds the start word). But usually it'll work a few times if you unplug/restart - does it work ever? Or does it fall over?

Is also may be a resolution issue? Do you have the 640 or the 320?

Can you try making the following modification here: https://github.com/LJMUAstroecology/flirpy/blob/ada3e02d61d8baa5d45698e6172ed2369bde9490/flirpy/camera/tau.py#L616

    def _sync_uart(self, allow_timeout=False):
        data = self._read()

        magic = b"UART"

        t = time.time()
        while data.find(magic) == -1:
            data += self._read()

            if not allow_timeout and time.time()-t > 0.2:
                log.warn("Timeout in command sync")
                break
            elif time.time() -t > 0.2:
                break

        return data[data.find(magic):]

This may start to fill up RAM if it fails though, as the data buffer gets bigger, but it might solve the problem.

[navotoz]

Thanks for the fast replay.

The resolution is 640x512, and the results are consistent with it.

The magic word is found even without the fix, but I fixed tit anyway. Still no cigar.

I tried restarts/replugging, still no luck..

[jveitchmichaelis]

Hmm so you're actually getting correct frame sizes, they're just scrambled? I didn't get a chance to look at your file yet, I'll do that when I get a chance. Could it be some config setting on the camera?

[navotoz]

Regarding the config - I don't know, but the camera does work when using the ThermalCapture software.

[jveitchmichaelis]

It's possible that TeAx's software handles some extra configuration that Flirpy doesn't. The way their API works is that the camera stream is constantly processed and there's a big switch statement to deal with it (it's a state machine).

To be honest I'm not sure, as we've tested this with two USB Grabbers without issues (beyond the occasional sync error).

Can you send the raw bytes between the two magic "TEAX" words, rather than the converted frame? It looks like the rows in the image are the wrong length.

Also what temperature are you expecting/what are you looking at? One issue could be your gain settings (high/low), but that should only affect the temp conversion, it shouldn't cause the artifacts you're seeing.

[navotoz]

I'm currently just taking pictures of my office. Attached is the data from _convert_frame under TeaxGrabber. raw_data.zip

[jveitchmichaelis]

Ok success 👍

import re, pickle, numpy as np

with open('../raw_data.pkl', 'rb') as f:
    data = pickle.load(f)

"""
Load a single frame
"""
data_slice = b""
prev_start, prev_end = None, None
r = re.compile(b"TEAX")
for i, match in enumerate(re.finditer(r, data)):
    current_start, current_end = match.span()

    if prev_end is not None:
        data_slice += data[prev_start:current_start]
        break

    prev_start = current_start
    prev_end = current_end

data = data_slice

def _convert_frame(data, to_temperature=False, width=336):
    raw_image = np.frombuffer(data[10:], dtype='uint8').reshape((-1,2*(width+2)))

    print(raw_image.shape)

    raw_image = 0x3FFF & raw_image.view('uint16')[:,1:-1]

    if to_temperature:
        return 0.04*raw_image - 273
    else:
        return raw_image

imshow(_convert_frame(data))
colorbar()

So it looks like your camera is defaulting to a much smaller frame size. I may need to add a command to send to the camera to fix this. If you grep in your image for "TEAX" you'll actually find there's about four copies. For now can you try:

from flirpy.camera.tau import TeaxGrabber

camera = TeaxGrabber(width=336, height=256)
image = camera.grab()
camera.close()
np.save('test', image)

I'll add a warning in the image decoder for now. @navotoz please check in the Teax config utility to see if you can set the default resolution to 640x512, this should be possible.

https://github.com/LJMUAstroecology/flirpy/commit/603d0c6f7338da542457db4d2e786b9f250065fa#diff-d1ab8f73bdd91dd06368cbe9a82f90e3 please pull from master and give that a go.

[jveitchmichaelis]

It would be worth figuring out where this resolution is being set, if indeed you have a 640 model camera and Teax's software gives you the correct resolution. I can't see anything in the Tau software IDD that mentions changing the size of the image that the camera core outputs, it should always be full res. There is a function to window the image (0xED), but I don't see why that would be used.

[navotoz]

When running

from flirpy.camera.tau import TeaxGrabber

camera = TeaxGrabber(width=336, height=256)
image = camera.grab()
camera.close()
np.save('test', image)

This line in TeaxGrabber failes, so I added @staticmethod before _convert_frame.

Anyway, in _convert_frame the frame_width recived is 338 for some reason, and the image is still scrambled. I'm attaching all the outputs of the camera. __syncoutput is the output of _sync inside grab. _while_loopoutput is the final output of grab, _raw_image_first/secondline are the first and second raw_image. runs.zip

Interestingly, if I use width=336, disregard the frame_width from data, and write: data = data[:-2*512] before the _rawimage lines, I get the correct image (albit with the smaller resolution).

I'll also write an email to ThermalCapture about setting the resolution in TeaxGrabber.

Thank you.

[jveitchmichaelis]

Oh I think we're just missing a self. Fixed that, it should be a class method.

Frame width should be 338 because it includes the two words at either end of each line. The conversion code automatically strips those and reports the effective width (and also adds 2 to the width that's passed in).

[jveitchmichaelis]

Almost certainly this is wrong

https://github.com/LJMUAstroecology/flirpy/blob/2605e504da92c396e2071741daa891c548a2929d/flirpy/camera/tau.py#L551

You should try changing 2058 to something like (10+4*height). Ie account for the 10 bytes in the header plus 4 pad bytes per row.

[navotoz]

You should try changing 2058 to something like (10+4*height). Ie account for the 10 bytes in the header plus 4 pad bytes per row.

Yes!! Thank you, works perfectly now.

[navotoz]

OK, I got it, the camera is actually 336x256. I checked the part number \<palm face>. Well, thank you very much @jveitchmichaelis. It works perfectly now.

[jveitchmichaelis]

OK let me push this and I'll bump the version on pypi for you