Thorlabs CCS spectrometer "Unsupported constant type 'long int' error"

[LewisBarltrop]

Hi,

I am having an issue loading drivers for the thorlabs CCS spectrometer when calling list_instruments()

Initially the return was empty, so I enabled logging and I received the following related to the attempted thorlabs driver install:

[    INFO]2020-09-01 16:50:57,261 instrumental.drivers: Importing driver module 'spectrometers.thorlabs_ccs'
[    INFO]2020-09-01 16:50:57,261 nicelib: Loading lib tlccs...
[    INFO]2020-09-01 16:50:57,261 nicelib: Loading ._tlccslib from instrumental.drivers.spectrometers...
[    INFO]2020-09-01 16:50:57,261 nicelib: Loading build module ._build_tlccs from instrumental.drivers.spectrometers...
[    INFO]2020-09-01 16:50:57,261 nicelib.build: Module _tlccslib does not yet exist, building it now. This may take a minute...
[    INFO]2020-09-01 16:50:57,272 nicelib.build: Searching for headers...
[    INFO]2020-09-01 16:50:57,272 nicelib.util: Looking for ('TLCCS.h',) in ('{PROGRAMFILES}\\IVI Foundation\\VISA\\Win64\\Include', '{PROGRAMFILES(X86)}\\IVI Foundation\\VISA\\Win64\\Include')
[    INFO]2020-09-01 16:50:57,272 nicelib.build: Found ['C:\\Program Files\\IVI Foundation\\VISA\\Win64\\Include\\TLCCS.h']
[    INFO]2020-09-01 16:50:57,272 nicelib.build: Parsing and cleaning headers...
[    INFO]2020-09-01 16:50:57,312 instrumental.drivers: Error when importing driver module spectrometers.thorlabs_ccs: <<Unsupported constant type 'long int'>>

I have installed the github version of Instrumental, along with the python dependencies for the CCS driver (pip install instrumental-lib[spectrometers.thorlabs_ccs]) and I have the thorlabs spectrometer software installed.

Any help would be much appreciated.

Thanks, Lewis

[natezb]

Hi Lewis, thanks for the informative bug report! I am able to reproduce this on my system as well. I'll try to take a look tonight after work to see if I can fix it.

This appears to be a NiceLib issue. We may need to add a missing platform constant (LONG_MAX) or a new case to the constant handling code. If you're at all curious about digging into what's going on, you can enable debug logging to get very detailed info:

from instrumental import log
log.log_to_screen(log.DEBUG)
import instrumental.drivers.spectrometers.thorlabs_ccs as ccs

[natezb]

So, I haven't had time to investigate the correctness of any platform constants, but I have tracked down the proximate cause of the issue. It turns out that NiceLib was failing to convert the C expression 0 & (0 > 0x7FFFFFL into a Python expression it could evaluate.

I've pushed a fix to NiceLib for this issue, so you can install it from GitHub to see if everything works ok. After the change, my copy of Instrumental can successfully parse the header and generate everything needed for the CCS module. I don't have a spectrometer to test with, so I can't verify that everything works.

I'll leave this issue open for now, in case you encounter more difficulties, and until I double-check the platform constants.

[LewisBarltrop]

Hi Nate, I tested the patch and this seems to have solved the issue for me too! I am yet to test it with the spectrometer again, but I will let you know how I go. Thanks again.

[natezb]

I pushed a change to the CCS driver that will instruct NiceLib to choose the right set of typedefs in visatype.h. I'd suggest pulling this new code and deleting your Instrumental\instrumental\drivers\spectrometers\_tlccslib.py file so it gets regenerated.

[LewisBarltrop]

Hi, I have followed your above advice about replacing the driver. When testing with the spectrometer, it is found as an instrument and I can interact with it however it appears I cannot extract data consistently. The commands that appear to work consistently are:

ccs200.start_single_scan() ccs200.get_integration_time() ccs200.start_continuous_scan() ccs200.is_data_ready() ccs200.cont_scan_in_progress() ccs200.start_continuous_scan() ccs200.stop_scan() ccs200.reset()

These following commands do not work consistently (I have been able to get them to work once or twice and receive the data, but this is very rare)

ccs200.stop_and_clear() ccs200.take_data() ccs200.get_scan_data()

They give the following error:

ccs200.take_data()
C:\Users\lewis\.conda\envs\PHYS3900python3Conda\lib\site-packages\instrumental_lib-0.6-py3.8.egg\instrumental\drivers\spectrometers\thorlabs_ccs.py:331: Warning: Data was not successfully acquired: Retrying now
  warn(Warning("Data was not successfully acquired: Retrying now"))
Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "C:\Users\lewis\.conda\envs\PHYS3900python3Conda\lib\site-packages\instrumental_lib-0.6-py3.8.egg\instrumental\drivers\spectrometers\thorlabs_ccs.py", line 387, in take_data
    self.stop_and_clear(max_attempts)
  File "C:\Users\lewis\.conda\envs\PHYS3900python3Conda\lib\site-packages\instrumental_lib-0.6-py3.8.egg\instrumental\drivers\spectrometers\thorlabs_ccs.py", line 335, in stop_and_clear
    self.get_scan_data()
  File "C:\Users\lewis\.conda\envs\PHYS3900python3Conda\lib\site-packages\instrumental_lib-0.6-py3.8.egg\instrumental\drivers\spectrometers\thorlabs_ccs.py", line 295, in get_scan_data
    data = self._NiceCCS.getScanData()
  File "C:\Users\lewis\.conda\envs\PHYS3900python3Conda\lib\site-packages\nicelib\nicelib.py", line 700, in __call__
    return self._libfunc._call(args, kwds, niceobj=self._niceobj)
  File "C:\Users\lewis\.conda\envs\PHYS3900python3Conda\lib\site-packages\nicelib\nicelib.py", line 1170, in _call
    return self.sig.extract_outputs(c_args, retval, ret_handler_args)
  File "C:\Users\lewis\.conda\envs\PHYS3900python3Conda\lib\site-packages\nicelib\nicelib.py", line 219, in extract_outputs
    retval = self.ret_handler.handle(retval, ret_handler_kwargs)
  File "C:\Users\lewis\.conda\envs\PHYS3900python3Conda\lib\site-packages\nicelib\nicelib.py", line 542, in handle
    return self.__func__(retval, **kwargs)
  File "C:\Users\lewis\.conda\envs\PHYS3900python3Conda\lib\site-packages\instrumental_lib-0.6-py3.8.egg\instrumental\drivers\spectrometers\thorlabs_ccs.py", line 64, in ccs_errcheck
    raise ThorlabsCCSError(niceobj.error_message(error_code))
instrumental.drivers.spectrometers.thorlabs_ccs.ThorlabsCCSError: b'Timeout expired before operation completed.'

Any help would be much appreciated,

Thanks, Lewis

[natezb]

It looks like all three of these methods eventually call tlccs_getScanData(). I don't one of these spectrometers, so I can't actually test anything, but the header TLCCS.h does have this note under a few of the functions:

Note: When you issue a read command 'Get Scan Data' before the CCS was triggered you will get a timeout error. Use 'Get Device Status' to check the scan status.

You may have to poke around and use some of the low-level functions (e.g. myccs._NiceCCS.getScanData()) to investigate what's going on. If you report back your findings we can hopefully fix whatever bugs you've encountered in the high-level code.

[LewisBarltrop]

Hi, after some struggle with this, I have found that in my case this issue is resolved by plugging my laptop into power. This was difficult to discover, as the Thorlabs software works fine on battery power. I can only assume that the issue is caused by some low power USB state that the laptop enters when on battery power, which the Thorlabs software overrides.

[natezb]

Thanks for the update, sounds like a tricky issue to diagnose!

[jlozanol]

Hi Nate, I tested the patch and this seems to have solved the issue for me too! I am yet to test it with the spectrometer again, but I will let you know how I go. Thanks again.

Hi @LewisBarltrop & @natezb... sorry if this is the wrong to place this query. I'm just facing the same issue after installing the CCS driver (pip install instrumental-lib[spectrometers.thorlabs_ccs])

[    INFO]2021-04-29 10:41:00,486 nicelib.util: Looking for ('TLCCS.h',) in ('{PROGRAMFILES}\\IVI Foundation\\VISA\\Win64\\Include', '{PROGRAMFILES(X86)}\\IVI Foundation\\VISA\\Win64\\Include')
[    INFO]2021-04-29 10:41:00,488 nicelib.build: Found ['C:\\Program Files\\IVI Foundation\\VISA\\Win64\\Include\\TLCCS.h']
[    INFO]2021-04-29 10:41:00,494 nicelib.build: Parsing and cleaning headers...
[    INFO]2021-04-29 10:41:00,554 instrumental.drivers: Error when importing driver module spectrometers.thorlabs_ccs: <<Unsupported constant type 'long int'>>

Unfortunately I'm still a newbie in programing and even in github! So just wanted to ask where I can have the patch that you mentioned before and where I need to install it on my computer as I don't know where to find it in github:

I pushed a change to the CCS driver that will instruct NiceLib to choose the right set of typedefs in visatype.h. I'd suggest pulling this new code and deleting your Instrumental\instrumental\drivers\spectrometers_tlccslib.py file so it gets regenerated.

Thank you very much for your help and again apologies if this is not the right way to make this request.

Regards,