[COM] Thorlabs CCS200 spectrometer on Linux?

[FilipDominec]

Instrument details

• Model: Thorlabs CCS200 spectrometer
• Communication: USB [VID: 0x1313 = 4883, PID 0x8088 = 32904], Thorlabs support suggests it communicates over VISA and that it could work on Linux, too
• Link to the documentation (if available):

Output of pyvisa-info

Machine Details:
   Platform ID:    Linux-4.15.0-124-generic-x86_64-with-Ubuntu-18.04-bionic
   Processor:      x86_64

Python:
   Implementation: CPython
   Executable:     /usr/bin/python3
   Version:        3.6.9
   Compiler:       GCC 8.4.0
   Bits:           64bit
   Build:          Oct  8 2020 12:12:24 (#default)
   Unicode:        UCS4

PyVISA Version: 1.11.3

Backends:
   ivi:
      Version: 1.11.3 (bundled with PyVISA)
      Binary library: Not found
   py:
      Version: 0.5.2
      ASRL INSTR: Available via PySerial (3.5)
      USB INSTR: Available via PyUSB (1.1.1). Backend: libusb1
      USB RAW: Available via PyUSB (1.1.1). Backend: libusb1
      TCPIP INSTR: Available 
      TCPIP SOCKET: Available 
      GPIB INSTR:
         Please install linux-gpib (Linux) or gpib-ctypes (Windows, Linux) to use this resource type. Note that installing gpib-ctypes will give you access to a broader range of funcionality.
         No module named 'gpib'

What I tried first

import pyvisa
pyvisa.util.get_debug_info()
pyvisa.log_to_screen()

rm = pyvisa.ResourceManager('@py')
print(rm)
    # gives: Resource Manager of Visa Library at py

res = rm.list_resources('?*::?*')
print(res)
    # when CCS200 spectrometer is connected, this gives: ('USB0::4883::32904::None::0::RAW',) 

rm.list_resources_info(res[0])
    # gives: {'USB0::4883::32904::None::0::RAW': ResourceInfo(interface_type=<InterfaceType.usb: 7>, interface_board_number=0, resource_class='RAW', resource_name='USB0::4883::32904::None::0::RAW', alias=None)}

ccs200 = rm.open_resource(res[0])

And this fails with the following message, as if the spectrometer returned no information about its serial number, in/out endpoints etc.:

...
2021-02-12 10:16:43,947 - pyvisa - DEBUG - Created library wrapper for py
2021-02-12 10:16:43,948 - pyvisa - DEBUG - Created ResourceManager with session 1285874
Resource Manager of Visa Library at py
('USB0::4883::32904::None::0::RAW',)
2021-02-12 10:16:43,975 - pyvisa - DEBUG - USB0::4883::32904::None::0::RAW - opening ...
Traceback (most recent call last):
  File "./test_visa_raw.py", line 20, in <module>
    ccs200 = rm.open_resource(res[0])
  File "/home/dominecf/.local/lib/python3.6/site-packages/pyvisa/highlevel.py", line 3304, in open_resource
    res.open(access_mode, open_timeout)
  File "/home/dominecf/.local/lib/python3.6/site-packages/pyvisa/resources/resource.py", line 298, in open
    self._resource_name, access_mode, open_timeout
  File "/home/dominecf/.local/lib/python3.6/site-packages/pyvisa/highlevel.py", line 3232, in open_bare_resource
    return self.visalib.open(self.session, resource_name, access_mode, open_timeout)
  File "/home/dominecf/.local/lib/python3.6/site-packages/pyvisa_py/highlevel.py", line 167, in open
    sess = cls(session, resource_name, parsed, open_timeout)
  File "/home/dominecf/.local/lib/python3.6/site-packages/pyvisa_py/sessions.py", line 323, in __init__
    self.after_parsing()
  File "/home/dominecf/.local/lib/python3.6/site-packages/pyvisa_py/usb.py", line 84, in after_parsing
    self.parsed.serial_number,
  File "/home/dominecf/.local/lib/python3.6/site-packages/pyvisa_py/protocols/usbraw.py", line 37, in __init__
    super(USBRawDevice, self).__init__(vendor, product, serial_number, **kwargs)
  File "/home/dominecf/.local/lib/python3.6/site-packages/pyvisa_py/protocols/usbtmc.py", line 195, in __init__
    self.find_devices(vendor, product, serial_number, None, **device_filters)
  File "/home/dominecf/.local/lib/python3.6/site-packages/usb/core.py", line 1285, in device_iter
    if _interop._all(tests) and (custom_match is None or custom_match(d)):
  File "/home/dominecf/.local/lib/python3.6/site-packages/pyvisa_py/protocols/usbutil.py", line 209, in cm
    if not fnmatch(value.lower(), pattern.lower()):
AttributeError: 'NoneType' object has no attribute 'lower'
2021-02-12 10:16:44,043 - pyvisa - DEBUG - Closing ResourceManager (session: 1285874)

What I tried second

I experimentally edited the pyvisa_py/protocols/usbutil.py so that it does not crash trying to lowercase None. Then the initialization continued further, until it failed with "USBRAW device must have both Bulk-In and Bulk-out endpoints". So the error is deeper than just missing serial number.

What I would like to do when we get VISA communication working

A somewhat bulky bundle of different GUIs and interfaces can be downloaded free from Thorlabs (select "Software -> Full Installer")

The ThorlabsOSASW_Full_setup.exe installer can be run under Linux+Wine and before it fails with some error, it unpacks useful files, e.g.:

... Program Files/IVI Foundation/VISA/Win64/include/TLCCS.h
... Program Files/IVI Foundation/VISA/Win64/TLCCS/TLCCS.c

which seem to contain C++implementation of communication.

But without VISA initialization, I cannot start trying to implement any pythonic interface.

[MatthieuDartiailh]

I gave a quick look at the Thorlabs page and did not see anything related to VISA support. The software tab suggests that there is a Communications Protocol manual that you could use to figure out how to communicate with the instrument over USB without going through Thorlabs software. If you can get this manual, you could try to directly use pyusb to communicate, however I do not think that pyvisa will help you since the instrument does not seem to rely on VISA.

[FilipDominec]

@MatthieuDartiailh , thank you for your fast response. Focusing on physics, I am rather a newbie to instrumentation.

1) In the _/Feedback\_ tab, response from December 2017, Thorlabs responds: "The crucial point will be the implementation of the USB connection in Linux. In Windows this is done by using the VISA interface. VISA-like interfaces for Linux (pyvisa-py, LibreVISA) are available."

2) If I am to implement (a subset of) the TLCCS.c interface, I am probably going to use VISA likewise. It is 670 lines of readable code. But on line 29 it refers to "NI LabWindows/CVI 2012SP1".

Maybe pyvisa is the way to go, maybe libusb is it, maybe both, maybe CCS200 won't work on Linux ever. Searching internet, it seems CCS200+Linux is something new. This is motivating. I would greatly appreciate any hint.

[MatthieuDartiailh]

Interesting I missed those comments. I would be curious to see the C example Thorlabs is referring at, could you share them with me by email ? The USB interface is not the most stable part of PyVISA-py. So to avoid wasting time I would try to go through a more battle tested VISA implementation first. If you have a Windows machine it should be easy, otherwise you can install for example Keysight VISA on Linux (it is restricted to some kernels though). If you manage to communicate through those but PyVISA-py fails we will be able to try to fix it.

[FilipDominec]

Dear Matthieu, under the following link you find the zipped folder pertinent to the Thorlabs' spectrometer VISA interface.

http://fzu.cz/~dominecf/IVI.zip

Filip

On Fri, 12 Feb 2021 at 11:19, Matthieu Dartiailh notifications@github.com wrote:

Interesting I missed those comments. I would be curious to see the C example Thorlabs is referring at, could you share them with me by email ? The USB interface is not the most stable part of PyVISA-py. So to avoid wasting time I would try to go through a more battle tested VISA implementation first. If you have a Windows machine it should be easy, otherwise you can install for example Keysight VISA on Linux (it is restricted to some kernels though). If you manage to communicate through those but PyVISA-py fails we will be able to try to fix it.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/pyvisa/pyvisa-py/issues/296#issuecomment-778107819, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAD47TWSRHWOKW6Q5SDMO4TS6T6DVANCNFSM4XQLHREQ .

[FilipDominec]

Further I observe that the RAW interface to VISA is the recommended one in the officially supplied documentation.

In particular, IVI Foundation/VISA/Win64/TLCCS/Manual/TLCCS_files/Functtlccs_init.html gives example VISA resource name as "USB0::0x1313::0x8080::DEVICE-SERIAL-NUMBER::RAW"

[FilipDominec]

And this is how the spectrometer reports in lsusb.

lsusb-v_for_Thorlabs.txt

[MatthieuDartiailh]

It will take me some time to go through that. I will try to to come back to you before next Wednesday. Going quickly over the TLCCS.c file it looks like communication is going mostly through usb_bulk_in and usb_bulk_out so VISA may help but not much. Also note that RAW USB resources are not part of the VISA standard only of the NI implementation, so using Keysight may not work.

From pyvisa-py side it seems the first things to figure out is why we do not read the serial number.

[FilipDominec]

I will be here to give you information regarding the hardware.

Meanwhile I am downloading Keysight VISA right now, but installing it might take a lot of prior learning for me, so let's wait until you get some suggestions.

Thank you a lot for your fast reaction!

[MatthieuDartiailh]

You may want yo have a look at the following resources about configuration of USB RAW resources from NI (https://zone.ni.com/reference/en-XX/help/370131S-01/ni-visa/configuringni-visarecognizerawusbdevice/). Ideally I would like to get the serial number read properly since without that we are unlikely to go anywhere. If you are on an Arch system you could also try https://aur.archlinux.org/packages/ni-visa/.

Note however that this thread suggests that the way NI handles RAW resources is old and may not be well supported on modern distros.

[FilipDominec]

Matthieu, do you think then it would be cleaner to write a pyUSB driver from scratch?

It could be integrated e.g. into https://github.com/ap--/python-seabreeze, which also contain an example of similar code. But I would have to find or wiretap the binary commands over USB, which is not going to be easy even for such a simple instrument.

[MatthieuDartiailh]

Honestly I have no idea at this point. I find the idea that the NI protocol may not be properly supported on modern distros a bit worrying. For wiretrapping you may want to have a look at WireShark.

[CompThing]

Hi Filip, couple of suggestions. I might not be able to help much as it is over a year since I worked with VISA with Linux and Python. Your output from lsusb with no detail on device classes hints that the device implementation is minimal. Have you tried running command as root, as without setting up appropriate group you can hit permissions issues reading from USB? When I was working with a USB VISA instrument with Linux on a Raspberry Pi, I had some better results using usbtmc. That might have changed as there have been enhancements to pyVISA-py since then. The Thor Labs instrument you use seems to take material directly from IVI Foundation. You could look here for some ideas: https://www.ivifoundation.org/default.aspx

[FilipDominec]

@CompThing 1. Yes, to make sure, I ran all tests also as root. And I tested the usual tricks with /etc/udev/rules.d/...

2. I was somewhat confused by the diversity of all protocols, interfaces and communication layers, so I maybe gave up too early and did not give https://github.com/python-ivi/python-usbtmc a try. Maybe this is the way to go, but I will wait: Meanwhile I asked Thorlabs directly if they could send me some Python+Linux related documentation. They promised to discuss the conditions and get back to me in few days.

@MatthieuDartiailh If all of the above fails, and if I found enought free time, Wireshark+USBPcap will be the last resort. But this can be really tedious.

[FilipDominec]

Thorlabs support dpt. is friendly, but their answer was discouraging. In short, what I understood so far: 1) They cannot send me detailed documentation of the protocol, because it would have to be written first (?). 2) The official driver explicitly depends on NI-VISA implementation of e.g. USB Control Transfer. I don't know anything about its being available among other VISA implementations. For a hypothetical new Linux driver, Thorlabs suggested the messages could be constructed from scratch using libusb and correctly warned me this can be challenging. 3) The spectrometer loads its firmware upon each connecting to the Thorlabs application. So any rudimentary spectral measurement would require prior binary firmware upload and repeated enumeration after that. Good news is that the firmware seems publicly available in the big package from the official website (namely, it unpacks into C:\\Program Files\Thorlabs\Thorlabs OSA\CCS\inf\). This approach is somewhat similar to cheap flatbed scanners, BTW.

I will have to think about it. If I was to develop a proof-of-concept Linux driver for CCS200, I would have to learn a lot. Currently there are more urgent tasks at my work to be done.

[heath-smith]

@FilipDominec I realize this topic is over a year old.. did you ever get any more information on this or make anymore attempts at this? I am a Thorlabs employee, currently developing a QC system in which we need the CCS200 on a Raspberry Pi. I think a Linux implementation of the CCS200 would be a great help to many others and I would be happy to work with you towards a solution. I have already reached out to some key folks in tech support to obtain any current info on this topic as well as the Communication Protocol manual.

[FilipDominec]

@heath-smith Unfortunately, I did not find enough time to proceed with CCS200, which is ocassionally used as handheld device by our PhD student and instead I am currently constructing a new high-performance spectrometer with echelle configuration already tested on my early prototype. Also the rp2daq project will hopefully soon get a TCD1304 CCD interface for simple grating spectrometers. In fact, on my workbench I have several spectrometer designs in different stages of development; still it would be certainly very interesting to access the CCS200 from Python.

If Thorlabs changes their mind and release some docs on communication with CCS200, I will be happy to become a beta-tester!