Problem with TCPIP HISLIP connection

[jonnojohnson]

I can connect to my Keysight scope on Ubuntu over LAN using inst0 but I get the following error when trying to use hislip0:

In [5]: scope = rm.open_resource("TCPIP0::141.121.97.63::hislip0::INSTR")
---------------------------------------------------------------------------
Exception                                 Traceback (most recent call last)
<ipython-input-5-7399acaeaf7f> in <module>()
----> 1 scope = rm.open_resource("TCPIP0::141.121.97.63::hislip0::INSTR")

/home/mj-ubuntu/anaconda/lib/python2.7/site-packages/pyvisa/highlevel.pyc in open_resource(self, resource_name, access_mode, open_timeout, resource_pyclass, **kwargs)
   1642                 raise ValueError('%r is not a valid attribute for type %s' % (key, res.__class__.__name__))
   1643 
-> 1644         res.open(access_mode, open_timeout)
   1645 
   1646         for key, value in kwargs.items():

/home/mj-ubuntu/anaconda/lib/python2.7/site-packages/pyvisa/resources/resource.pyc in open(self, access_mode, open_timeout)
    201         logger.debug('%s - opening ...', self._resource_name, extra=self._logging_extra)
    202         with self._resource_manager.ignore_warning(constants.VI_SUCCESS_DEV_NPRESENT):
--> 203             self.session, status = self._resource_manager.open_bare_resource(self._resource_name, access_mode, open_timeout)
    204 
    205             if status == constants.VI_SUCCESS_DEV_NPRESENT:

/home/mj-ubuntu/anaconda/lib/python2.7/site-packages/pyvisa/highlevel.pyc in open_bare_resource(self, resource_name, access_mode, open_timeout)
   1599         :return: Unique logical identifier reference to a session.
   1600         """
-> 1601         return self.visalib.open(self.session, resource_name, access_mode, open_timeout)
   1602 
   1603     def open_resource(self, resource_name,

/home/mj-ubuntu/anaconda/lib/python2.7/site-packages/pyvisa-py/highlevel.pyc in open(self, session, resource_name, access_mode, open_timeout)
    189         cls = sessions.Session.get_session_class(parsed.interface_type_const, parsed.resource_class)
    190 
--> 191         sess = cls(session, resource_name, parsed)
    192 
    193         return self._register(sess), constants.StatusCode.success

/home/mj-ubuntu/anaconda/lib/python2.7/site-packages/pyvisa-py/sessions.pyc in __init__(self, resource_manager_session, resource_name, parsed)
    182                       constants.VI_ATTR_RSRC_CLASS: parsed.resource_class,
    183                       constants.VI_ATTR_INTF_TYPE: parsed.interface_type}
--> 184         self.after_parsing()
    185 
    186     def after_parsing(self):

/home/mj-ubuntu/anaconda/lib/python2.7/site-packages/pyvisa-py/tcpip.pyc in after_parsing(self)
     60 
     61         if error:
---> 62             raise Exception("error creating link: %d" % error)
     63 
     64         self.link = link

Exception: error creating link: 3

This is my setup:

mj-ubuntu@Austin-Compaq-8510p:~$ python -m visa info
Machine Details:
   Platform ID:    Linux-3.13.0-63-generic-x86_64-with-debian-jessie-sid
   Processor:      x86_64

Python:
   Implementation: CPython
   Executable:     /home/mj-ubuntu/anaconda/bin/python
   Version:        2.7.10
   Compiler:       GCC 4.4.7 20120313 (Red Hat 4.4.7-1)
   Bits:           64bit
   Build:          May 28 2015 17:02:03 (#default)
   Unicode:        UCS4

PyVISA Version: 1.8

Backends:
   ni:
      Version: 1.8 (bundled with PyVISA)
      Binary library: Not found
   py:
      Version: 0.3.dev0
      ASRL INSTR: Available via PySerial (2.7)
      TCPIP INSTR: Available 
      USB RAW: Available via PyUSB (1.0.0b2). Backend: libusb1
      USB INSTR: Available via PyUSB (1.0.0b2). Backend: libusb1
      GPIB INSTR:
         Please install linux-gpib to use this resource type.
         No module named gpib
      TCPIP SOCKET: Available 
   sim:
      Version: 0.3
      Spec version: 1.1

[hgrecco]

We have only implemented VXI-11 in pyvisa-py. HISLIP is a different, newer, better protocol. I would love to see this implemented in pyvisa-py. As far as I know there is no Python library (or open source implementation in other language) for this protocol so we would have to do it from scratch. It involves doing something similar to what is done in vxi11.py and rpc.py.

The specification is here: http://www.ivifoundation.org/downloads/Class%20Specifications/IVI-6.1_HiSLIP-1.1-2011-02-24.pdf

I can help and guide, but have no time to do it my self.

[hgrecco]

One available thing is the wireshark routines to analyze hislip traffic:

https://github.com/wireshark/wireshark/blob/dbf314a4ab3e005bc9650d69f47fb44aa446fb58/epan/dissectors/packet-hislip.c

It might help or we might ping the people who have developed them so see if they want to help.

[jonnojohnson]

Ok thanks @hgrecco I wish I was a better programmer or I'd jump right in. Hislip is definitely the future from what I've seen so it would be great to have.

[bjaraujo]

I am trying HiSLIP with pyvisa and sometimes I get: VI_ERROR_CONN_LOST during query_binary_values with large binary data counts ~96007 bytes. The same transaction works in NI-MAX.

[MatthieuDartiailh]

@bjaraujo can you please post the output of pyvisa-info ? Also the original thread was about using pyvisa-py with HiSlip, this was not supported at the time and is sill not supported. So I am guessing you are not using pyvisa-py as a backend but simply the bindings to NI VISA.

I would suggest you compare the settings used in NI-MAX and in pyvisa. Both call the same underlying library so if configured in the same way they should give you the same results.

[bjaraujo]

pyvisa details:

Machine Details: Platform ID: Windows-10-10.0.16299-SP0 Processor: Intel64 Family 6 Model 78 Stepping 3, GenuineIntel

Python: Implementation: CPython Executable: C:\Users\baraujo\AppData\Local\Programs\Python\Python37-32\python.exe Version: 3.7.2 Compiler: MSC v.1916 32 bit (Intel) Bits: 32bit Build: Dec 23 2018 22:20:52 (#tags/v3.7.2:9a3ffc0492) Unicode: UCS4

PyVISA Version: 1.9.1

Backends: ni: Version: 1.9.1 (bundled with PyVISA)

1: C:\WINDOWS\system32\visa32.dll:

     found by: auto
     bitness: 32
     Vendor: National Instruments
     Impl. Version: 18875648
     Spec. Version: 5244928

I agree maybe it is the way it is setup. It shouldn't time out because I have a timeout of 5 seconds. In NI-MAX it does the same thing but ONLY if I do not read the entire buffer.

[MatthieuDartiailh]

Thanks for the details.

Could you elaborate what you mean by NI-MAX it does the same thing but ONLY if I do not read the entire buffer ? Is it related to the buffer size ? If you read everything at once it works but if you read the answer by chunk it fails ?

[bjaraujo]

Sure.

The data response is pretty large. With small chunks like 9,000 bytes it works on both systems. With a bigger response, like 90,000 bytes it fails in pyvisa. It also fails in NI-MAX but only if I do not read the entire buffer. If I read for example only 1024 bytes NI-MAX also reports connection lost but I guess because the message is incomplete so it breaks the protocol.

[MatthieuDartiailh]

You can set the buffer size used by PyVisa using the chunk_size attribute. By default it 20*1024 but increasing it may help in your case. You can also pass a size argument to read_raw but it means you will have to manually parse the answer.

[bjaraujo]

Ok thanks. Then that is certainly the problem. The error wasn't that clear.

[MatthieuDartiailh]

If it fixes the issue let me know I may need to clarify the docs.

[bjaraujo]

So I need to change the chunk_size for my case. Why is there a limit anyway?

[bjaraujo]

Yes. I changed it to: chunk_size = 2000 * 1024 And it fixed the issue.

It could check the header and check if it was bigger than chunk_size. Also would be good to pass chunk_size as parameter to query_binary_values.

Thanks once again.

[MatthieuDartiailh]

Thanks for letting me know. Checking the header is already too late since you have already read a chunk when you can do it. I agree that adding the capability to set the chunck size when reading data make sense. If you feel like it, feel free to make a PR in that direction. Otherwise I will try to add it and update the documentation.

[bjaraujo]

Hi

What I found was pyvisa doesn't read the header before it reads rest of the content and this may cause problems when data is larger than 20*1024 bytes specially in HiSLIP protocol. In HiSLIP it cannot read partial messages.

I don't think increasing chunk_size is a good fix as then there is always a number above which it might fail.

So my fix is to read the header first and then determine how many bytes to read from visa. I have tested for IEEE format and it works. Then I also changed the HP format for consistency and it may have similar problem (I haven't tested for HP).

472c472
<         block = self._read_raw()
---
>         block = b''
476c476,488
<             offset, data_length = util.parse_ieee_block_header(block)
---
>             hash = self.read_bytes(1)                   # Hash '#'
>             digits = self.read_bytes(1)                 # Digits
>             block_size = self.read_bytes(int(digits))   # Block size
> 
>             block = block + hash
>             block = block + digits
>             block = block + block_size
> 
>             block = block + self.read_bytes(int(block_size))
>             
>             offset = 2 + int(digits)
>             data_length = int(block_size)
>                         
477a490
>             
479,480c492,503
<             offset, data_length = util.parse_hp_block_header(block,
<                                                              is_big_endian)
---
>         
>             hash = self.read_bytes(2)                   # Hash '#A'        
>             block_size = self.read_bytes(4)             # Block size
> 
>             block = block + hash
>             block = block + block_size
> 
>             offset = 6
>             data_length = int_from_bytes(block_size,
>                                  byteorder='big' if is_big_endian else 'little'
>                                  )
>             
700d722
< 

Tested for block sizes of 80,000 bytes which it couldn't do before. Before it would say: VI_ERROR_CONN_LOST

[bjaraujo]

With this new patch:

Size (bytes)	Result
0	PASSED
64	PASSED
960	PASSED
72,000	PASSED
784,000	PASSED
7,840,000	PASSED

[f03el]

I just ran across this Python implementation of HiSLIP: https://github.com/llemish/PyHiSLIP. Would that be of any help here?

[MatthieuDartiailh]

This could definitively help. It would need to get wrapped to into pyvisa-py to provide the proper API but that should not be too hard. Do you want to give it a shot ?

[f03el]

I just looked through it briefly, but I imagine that we might want to modify the code somewhat to make it align better with the rest of the PyVISA-py codebase? I'll try to find the time to work on this.

[f03el]

Would it be OK to implement this using the asyncio module? I assume we don't have to worry too much about Python 2 support.

Also, does anybody know if PyVISA/PyVISA-py has a vendor ID (as found in the VXI plug&play specification VPP-9)? This is apparently used when connecting to instruments over HiSLIP. According to the docs, vendor IDs are assigned free of charge by the IVI Foundation and vendors don't need to join the foundation.

[MatthieuDartiailh]

I don't think we have a vendor ID.

We do not support Python 2 anymore so feel free to use asyncio if it helps.

[KosRaut]

Hello! I reached here trying to use pyvisa-py (on Apple M1 Silicon) with a Keysight equipment using HiSLIP. It seems like HiSLIP protocol may not have implemented yet. Are there any plans to do so?

My setup: Machine Details: Platform ID: macOS-10.16-x86_64-i386-64bit Processor: i386

Python: Implementation: CPython Executable:
Version: 3.9.5 Compiler: Clang 6.0 (clang-600.0.57) Bits: 64bit Build: May 3 2021 13:17:02 (#v3.9.5:0a7dcbdb13) Unicode: UCS4

PyVISA Version: 1.11.3

Backends: ivi: Version: 1.11.3 (bundled with PyVISA) Binary library: Not found py: Version: 0.5.2 USB INSTR: PyUSB does not seem to be properly installed. Please refer to PyUSB documentation and install a suitable backend like libusb 0.1, libusb 1.0, libusbx, libusb-win32 or OpenUSB. No backend available USB RAW: PyUSB does not seem to be properly installed. Please refer to PyUSB documentation and install a suitable backend like libusb 0.1, libusb 1.0, libusbx, libusb-win32 or OpenUSB. No backend available TCPIP INSTR: Available TCPIP SOCKET: Available GPIB INSTR: Please install linux-gpib (Linux) or gpib-ctypes (Windows, Linux) to use this resource type. Note that installing gpib-ctypes will give you access to a broader range of funcionality. No module named 'gpib'

Hardware Overview: Model Name: Mac mini Model Identifier: Macmini9,1 Chip: Apple M1 Total Number of Cores: 8 (4 performance and 4 efficiency) Memory: 8 GB System Firmware Version: 6723.101.4 OS Loader Version: 6723.101.4

[MatthieuDartiailh]

I have been contacted offline by somebody that worked on HiSLIP and got approved to open source its work. So it will likely happen but I cannot give you any timeline.