Cannot get Map Strings for Strobe Mode in python

[kmcgrathgenerate]

I am trying to use the function:

int AC IC_GetPropertyMapStrings(HGRABBER hGrabber, char* Property, char *Element, int *StringCount, char **Strings );

but I am getting an error on parameter 5, char **Strings, here is what I have in python:

        num = ctypes.c_int()
        strings = ctypes.c_char_p()
        strings = ctypes.POINTER(ctypes.c_char_p)
        self.ic.IC_GetPropertyMapStrings(self.hGrabber, tis.T("Strobe"), tis.T("Mode"), num, strings)

The error I get is: "ctypes.ArgumentError: argument 5: <class 'TypeError'>: Don't know how to convert parameter 5"

Is there something I am doing wrong?

I am trying to set the strobe on, with positive polarity, and setting it to be a fixed duration of a few microseconds. That is my actual task but I see no example code using strobe, at least not in python. So if someone knows the strings for the modes that I can use, that is my end goal. I am only trying to call this function because I do not see that information anywhere around.

Thank you!

[TIS-Stefan]

Hello

There are a few issues in your code... However, the main issue is the signature of the function, IC_GetPropertyMapStrings, which does not match my. The correct one is:

ic.IC_GetPropertyMapStrings.argtypes = (ctypes.POINTER(tis.HGRABBER),
                                        ctypes.c_char_p,
                                        ctypes.c_char_p,
                                        ctypes.POINTER(ctypes.c_long),
                                        ctypes.POINTER(ctypes.c_long),
                                        ctypes.POINTER(ctypes.c_char_p)                                                
                                        )

Parameters are grabberhandle, propertyid, propertyelement, long pointer number of strings, long pointer maximum string length , char pointer to char* The other issue the char** handling, for which we have to allocate memory.

You need to declare two variables first:

    modecount = ctypes.c_long()
    modecount.value = 0

    maxlength = ctypes.c_long()
    maxlength.value = 0`

Then call the IC_GetPropertyMapStrings() function in order to get the amount of strings for this property and the maximum length of them

    ic.IC_GetPropertyMapStrings(hGrabber, tis.T("Strobe"), tis.T("Mode"), modecount, maxlength, None )

Now create the memory for the strings

    string_buffers = [ctypes.create_string_buffer(maxlength.value + 1) for i in range(modecount.value)]
    pointers = (ctypes.c_char_p*modecount.value)(*map(ctypes.addressof, string_buffers))

Next get the strings

    ic.IC_GetPropertyMapStrings(hGrabber, tis.T("Strobe"), tis.T("Mode"), modecount, maxlength, pointers )

Make a nice list of them and show them

    modelist = [s.value for s in string_buffers]
    print("Available Modes")
    for mode in modelist:
        print(tis.D(mode))

The complete function for copy & paste:

def getStrobeModes(hGrabber):
    modecount = ctypes.c_long()
    modecount.value = 0

    maxlength = ctypes.c_long()
    maxlength.value = 0

    ic.IC_GetPropertyMapStrings(hGrabber, tis.T("Strobe"), tis.T("Mode"),
                                 modecount, maxlength, None )

    string_buffers = [ctypes.create_string_buffer(maxlength.value + 1)
                       for i in range(modecount.value)]
    pointers = (ctypes.c_char_p*modecount.value)(*map(ctypes.addressof, string_buffers))
    ic.IC_GetPropertyMapStrings(hGrabber, tis.T("Strobe"), tis.T("Mode"),
                                 modecount, maxlength, pointers )

    return [s.value for s in string_buffers]

In case your version of the tisgrabber.dll does not match mine (which I do not think): Here it is: tisgrabber_x64.zip

Best regards Stefan

[krixen55]

Hello Stefan,

Thanks for getting back to me. I first used the function on the DMK 37BUX250 Camera and 0 strobe modes came back, which might be right. I then tried it on a DMK 37BUX287 and I get an error every time I use the function you wrote, it fails at the first get property map strings function:

        ic.IC_GetPropertyMapStrings(
            hGrabber, tis.T("Strobe"), tis.T("Mode"), modecount, maxlength, None
        )

    ic.IC_GetPropertyMapStrings(
OSError: exception: access violation reading 0x0000000000000000

Here is the signature I see in the tisgrabber.h file for the get map strings function:

    @param hGrabber Handle to a grabber object.
    @param Property  The name of the property, e.g. "Strobe"
    @param Element  The type of the interface, e.g. "Mode" 
    @param StringCount  Receives the count of strings, that is modes, availble
    @Param StringMaxLength Receives the maximum length of th strings
    @param Strings pointer to an array of char*, that will contain the mode strings. The array size should be StringCount * 20. Parameter can be null in order to query the number of strings

    @retval IC_SUCCESS          Success
    @retval IC_NO_HANDLE        Invalid grabber handle
    @retval IC_NO_DEVICE        No video capture device opened
    @retval IC_PROPERTY_ITEM_NOT_AVAILABLE      A requested property item is not available
    @retval IC_PROPERTY_ELEMENT_NOT_AVAILABLE       A requested element of a given property item is not available
    @retval IC_PROPERTY_ELEMENT_WRONG_INTERFACE     requested element has not the interface, which is needed.
*/
int AC IC_GetPropertyMapStrings(HGRABBER hGrabber, char* Property, char *Element, int *StringCount, int *StringMaxLength, char **Strings);

Any idea where the mismatch is?

Also since I might be trying to get this working but my base idea is not smart. Can you use the strobe output as an indicator that an image was taken? We have an independent timing system that is triggering the camera but sometimes suddenly the camera appears to not take pictures. So we were hoping the strobe output could be feedback to the timing system that a picture was indeed taken. Does this work how I think it will?

[TIS-Stefan]

Hello did you exchange the tisgrabber DLL, as I mentioned? Stefan

[krixen55]

@TIS-Stefan

Yes I did swap them out. Is the DLL the only thing that needs to change? I removed the old file and added in the new one. I am loading it in as:

ic = ctypes.cdll.LoadLibrary("./TIS/tisgrabber_x64.dll")

Does the signature I posted to you look different than what you expect? Is there an accompanying .h file?

[TIS-Stefan]

Hello Yes there is is: tisgrabber.zip Also here is my complete sample include binaries: mapstrings.zip

I am sorry for confusing versions. I started to make a wheel including a nice camera class. Due to this, I also made changes on the DLL.

Stefan

[krixen55]

Your utility function worked great! I finally got the original function to at least finish without error but they came back with 0 modes available, but your dialog let me see there is 'constant' 'fixed duration' and 'exposure'.

Just to give some context and hopefully you can give some insight. When we are triggering the camera at high frame-rates, we will get 5+ minutes of great captures and then suddenly everything seems to freeze for about a second and we miss 50-100 images back to back. We are still triggering the camera, we confirmed, but the images are not getting to the callback function.

The reason I am asking about strobe modes is because I was hoping the strobe output would be feedback that an image was taken. Is this actually the case? If the strobe output goes high, can that guarantee that an image was taken and will find its way to the frame ready callback?

Also, is there some way for me to know if images that are triggered have failed to produce an image? some sort of error callback?

[krixen55]

Also can you show an example of how you would set a strobe to be enabled with the mode set to fixed duration, and how to set the duration? This is what I have

  ic.SetPropertySwitch(
      hGrabber, tis.T("Strobe"), tis.T("Enable"), 1
  )
  ic.IC_SetPropertyValue(
      hGrabber,
      tis.T("Strobe"),
      tis.T("Mode"),
      tis.T("fixed duration"),
  )

[TIS-Stefan]

Hello Images are taken and sent by the camera in any case, regardless whether your software got stuck or not. So you may check, what happens in your computer. In case you save the images to hard disc: The Windows disc cache blocks saving, when it is full and thus flushed.

The "IC_SetPropertyValue()" can not be used, because the fourth parameter is expected to be an integer type, not a char pointer. Thus you need to declare the matching function C

int AC  IC_SetPropertyMapString(HGRABBER hGrabber, char* Property, char *Element,  char *String );

Python

ic.IC_SetPropertyMapString.argtypes = (ctypes.POINTER(HGRABBER),
                                ctypes.c_char_p,
                                ctypes.c_char_p,
                                ctypes.c_char_p)

It is used

ic.IC_SetPropertyMapString(
      hGrabber,
      tis.T("Strobe"),
      tis.T("Mode"),
      tis.T("fixed duration"),
  )

Stefan

[krixen55]

Thanks for the update. I think I may have found something that can help me with dropped frames. We are triggering the camera very quickly and sometimes we dont get an image in the 'framereadycallback', but what I noticed is that the framenumber jumps in this case.

Does the framenumber in the framreadycallback count the # of triggers it receives even if it cant process the image? Can I use this number to find out how many images the camera was unable to take?

[TIS-Stefan]

Hello

The frame number you get is made by the driver and counts the received frames. If the callback lasts longer than the time between two frames, then a frame can be skipped. Python is not really the fastest programming language, so you should make sure, the code in your callback is processed fast enough. I am guessing a little bit, because I do not know, what you do in the callback.

Stefan

[krixen55]

Sorry this has gotten a bit off-topic. I have a microcontroller timer system that is taking pictures (triggering) on the camera every (500 hz) 2 ms or slower. We are using a very small resolution (200x720) which seems to be able to handle up to 800+ fps when we are in free running mode.

Every once in awihle we have an issue where things get out of sync, there are images that were not taken. What I did notice yesterday is your new driver code you sent me seems to have minimized that effect greatly.

I know that my framereadycallback code is very fast, much faster than 2 ms. what we will notice is that the framenumber jumps about 3-6 in-between images very periodically at very high speeds. We might get this behavior once every 100k images, but we need a way to recover and keep things back in-line. Here is my framereadycallback

   def frameReadyCallback(self, hGrabber, pBuffer, framenumber, pData):
        if self.trigger_mode == False:
            return
        framedifference = 0
        if self.prev_frame_number == -1:   # first image taken, set to framenumber
            self.prev_frame_number  = framenumber
        else:
            if not self.prev_frame_number + 1 == framenumber:
                framedifference = framenumber - self.prev_frame_number  + 1
                print("Dropped Frames", framedifference )
            self.prev_frame_number  = framenumber

        Width = ctypes.c_long()
        Height = ctypes.c_long()
        BitsPerPixel = ctypes.c_int()
        colorformat = ctypes.c_int()

        # Query the image description values
        self.ic.IC_GetImageDescription(
            hGrabber, Width, Height, BitsPerPixel, colorformat
        )

        # Calculate the buffer size
        bpp = int(BitsPerPixel.value / 8.0)
        buffer_size = Width.value * Height.value * BitsPerPixel.value

        if buffer_size > 0:
            image = ctypes.cast(pBuffer, ctypes.POINTER(ctypes.c_ubyte * buffer_size))

            cvMat = np.ndarray(
                buffer=image.contents,
                dtype=np.uint8,
                shape=(Height.value, Width.value, bpp),
            )

            cv_img = np.squeeze(cvMat, axis=2)
            try:
                self.trigger_mode_queue.put((self.device_name, cv_img, framedifference))
            except Exception as e:
                print(e)

I keep track of the last image that we got with the 'self.previous_frame_number' and check to see if every framenumber that comes in from the callback is equal to that number plus one. When a mismatch does occur, its never just 1 frame, its usually several.

Have you seen this behavior before?

[TIS-Stefan]

Hello

Yes, frame drops are very common. Usually they happen, because there is something on the computer, e.g. idle states for power saving or too high CPU load which disturbs communication. However, you may remove all the image processing from your callback and see, how it behaves. Make sure you do not run out of physical memory, because then Windows starts swapping. And that really blocks things.

So it is step by step analysis for finding the bottle neck.

If your goal is to capture a specified amount of frames into memory, then this is possible without a big callback and the tisgrabber.dll gave to you.

import ctypes
from operator import truediv
import tisgrabber as tis
import cv2
import numpy as np
import sys
import time

ic = ctypes.cdll.LoadLibrary("./tisgrabber_x64.dll")
tis.declareFunctions(ic)

###############
#
###############
class CallbackUserdata(ctypes.Structure):
    def __init__(self):
        self.Value1 = 0
        self.Value2 = 0
        self.end = False
        self.imagedata=[] 
        self.firstindex = -1

def FrameCallback(hGrabber, hMemBuffer, framenumber, pData):
    # capture n images only
    if pData.Value1 >= 9:  return
    pData.Value1 = pData.Value1 + 1
    # Get the index of the first image in the ring buffer
    if( pData.firstindex == -1 ):
        Index = ctypes.c_long()
        ic.IC_MemBufferGetIndex(hMemBuffer, Index)
        print(Index.value)
        pData.firstindex = Index.value

def saveimage(hGrabber, i):
    """
    Retrieve the image at index i and save it.
    """
    membuffer = ctypes.c_void_p()
    ic.IC_GetMemBuffer(hGrabber,i, membuffer)

    Width = ctypes.c_long()
    Height = ctypes.c_long()
    BitsPerPixel = ctypes.c_int()

    ic.IC_GetMemBufferDescription( membuffer,Width, Height, BitsPerPixel)

    bpp = int(BitsPerPixel.value / 8.0)
    buffer_size = Width.value * Height.value * BitsPerPixel.value

    # Get the image data
    imagePtr = ctypes.c_char_p()
    ic.IC_MemBufferGetDataPtr(membuffer, imagePtr)

    imagedata = ctypes.cast(imagePtr,
                            ctypes.POINTER(ctypes.c_ubyte *
                                            buffer_size))

    # Create the numpy array
    image = np.ndarray(buffer=imagedata.contents,
                        dtype=np.uint8,
                        shape=(Height.value,
                               Width.value,
                               bpp))
    image = cv2.flip(image, 0) 
    cv2.imwrite("test{0}.png".format(i),image) 

    # Free memory, we do not need it any longer.
    # The image is still in the ring buffer
    ic.IC_ReleaseMemBuffer( membuffer )

ic.IC_InitLibrary(0)
Userdata = CallbackUserdata()
Callbackfuncptr = ic.FRAMEREADYCALLBACKEX(FrameCallback)
hGrabber = tis.openDevice(ic)

if(ic.IC_IsDevValid(hGrabber)):
    ic.IC_SetFrameReadyCallbackEx(hGrabber, Callbackfuncptr, Userdata)

    ic.IC_SetRingBufferSize(hGrabber,500)

    ic.IC_SetContinuousMode(hGrabber, 0) 
    ic.IC_StartLive(hGrabber, 0) 

    ic.IC_MsgBox(tis.T("OK to end"), tis.T("Callback"))
    ic.IC_StopLive(hGrabber)

    # now handle the captured images
    for i in range(Userdata.firstindex, Userdata.firstindex+ Userdata.Value1):
        saveimage(hGrabber, i)

else:
    ic.IC_MsgBox(tis.T("No device opened"), tis.T("Callback"))

ic.IC_ReleaseGrabber(hGrabber)

The trick is to let IC Imaging Control saving all images into its ringbuffer. Its size is set to 500 here as example. Maybe this is an approach for you.

Stefan

[kmcgrathgenerate]

Thank you for the thorough response, my goal is actually strict timing based. Each image needs to quickly move through my system and generate an output YES/NO at the end. So I cannot wait for n frames, I need to pass a single frame through immediately after its been taken. I do not save any images in memory, I simply pass the image through the callback into a processing queue.

I think that your code might give me some ideas to make the callback faster.

What I do notice with this issue is that the framenumber difference in my code is always off by a lot from the number of frames it thinks it dropped. i.e., my code will say that 6 frames were dropped (always back-to-back), but the microcontroller is always off by about double that.

Is there anything in the camera library that would cause it to not increment the framenumber?

[TIS-Stefan]

Hello There is nothing in the camera library, that causes this issue. Sounds for a computer issue. And if so, it is expected, that something is somwhere too slow.

IC Capture 2.5 shows the frame delivery statistics in the lower right corner Can you please let me know these numbers, in case the issue shows up? Meaning of the four numbers is Packet Loss during transmission Driver Transform issue, e.g. computer to slow DirectShow Graph issue, e.g. computer to slow Unknown

IC Capture is available at https://www.theimagingsource.com/en-de/support/download/iccapture-2.5.1557.4007/

Which computer do you use? Stefan

[kmcgrathgenerate]

Hello @TIS-Stefan , so as it turns out, the dropped frames I am measuring from the callback does exactly line up with the number of missed frames my timing system thinks. So the way I implemented it above is working perfectly! I do think you are right that windows is having issues where it lags and the callback is still running on the previous image.

I am going to take your advice and make my callback faster. We are also getting a much needed PC upgrade shortly which should minimize this issue. Thank you for all the help!!

[TIS-Stefan]

Hello

I thought about your issue last night and I think, I know, what happens. The good news is, you do not have frame drops. The bad news is, it wont help. The callback mechanism of IC Imaging Control is implemented in a way, you do not loose frames. The callback is called under two conditions:

1. There is a new frame
2. A previous callback execution has ended IC Imaging Control uses a ring buffer for saving frames. All frames are copied into this ring buffer and if possible, the callback is called. I the callback was not called, the new frame is at least in the ring buffer, until it is overwritten by a new frame.

Therefore, if your callback lasts too long, you have a jump between these frame numbers.

Now we have to explain, why the script runs fine for a time but later it will not get callbacks for all frames. In your callback you always create a new numpy array. That means, memory is allocated. Also you put this numpy array into a trigger_mode_queue, for which I am not sure, whether new memory is allocated or whether it is a pointer operation only. However, memory is allocated and marked for deletion later every 1/800 second. Sooner or later the garbage collector will turn on and this is, where the callback lasts longer than expected. And then you get these missing callbacks in regular intervals.

In short: It is because the way we have to create the numpy array for OpenCV.

This is also a good portion of guessing about how Python works.

I have no solution for that.

If this is a time critical task you have to perform, I would change to C++, because there is no need to allocate and copy images to a cv::Mat. You can simply create the cv::Mat once and pass the ImageBuffer.getPtr() data pointer to the cv::Mat::Data and then you can use OpenCV functions. No copy needed. No new memory needed.

I you change to C++ you can use Visual Studio Community Edition with IC Imaging Control Classlibrary or MinGW/Gnu C++ with the tisgrabber.dll as you do in Python (only easier). I will try to help you as much as I can.

Or you find a way to avoid the allocation of memory the callback.

(I assume next version of IC Imaging Control 4 will has a real Python port, which avoids this issue at all.)

Stefan

[krixen55]

Hello Stefan, thank you for giving my issue so much thought!

What is most important for my project is maintaining order of images collected. Since the # of images that I miss, from a slow callback, are counted, I can maintain perfect order.

Another thing to minimize this issue is the computer I am using now is pretty weak and I am building a custom built that should be the fastest reasonably priced PC possible as of today (i9 13th gen). This will be a big help.

But that is good to know that the next IC imaging control will have a real python port. Is there a rough timeline of when that comes out?

[TIS-Stefan]

Hello

ETA is January 2024. Hopefully we meet that date.

Stefan