ni / nimi-python

Python bindings for NI Modular Instrument drivers.
Other
112 stars 84 forks source link

Using Python FGEN API to control PXIe-5413 signal generator works first time but fails after power cycling devices with error "-88717: No device by the given name was found" #2042

Closed drjk94 closed 6 months ago

drjk94 commented 6 months ago

Hi, I hope I am correct posting this here, as the issue is seemingly only encountered when using the Python API to control the device. We have already tried many things but are somewhat at a loss. We are somewhat new to using Python when it comes to controlling hardware, being more used to LabVIEW.

Our setup consists of PXIe-1071 chassis with the following modules: A PXIe-8301 remote control module, a PXIe-4480 acquisition card and a PXIe-5413 signal generator. A Dell laptop, running Windows 10 is connected to the chassis using a thunderbolt cable between the laptop and the remote control module.

When connecting to the equipment, the following procedure is followed, starting with both the chassis and laptop’s powered down. -Connect cables -Power on PXIe-1071 chassis with button on front of chassis -Power on PC -Log into PC

When following this procedure, all modules will correctly show up as present in NI-MAX. We also have no troubles consistently controlling the generator using Instrument Studio or the FGEN LabVIEW examples (such as FGEN Sweep Generator.vi).

However, when using the Python API to control the PXIe-5413 card we encounter a strange error.

Description of issue

After resetting the Configuration Data in NI-MAX (tools-Reset Configuration Data) and connecting to the hardware following the procedure above, we can use the Python API to control our generator without issue. An example of a simple piece of code to generate a chirp signal is added in the attachment which can be run from the command prompt. However, after the next power cycle (during which chassis and PC have both been powered down and restarted), when running the same python script, we instead get the following error:

nifgen.errors.DriverError: -88717: No device by the given name was found.

The problem seems to be associated with the Session.commit() function, though if we leave out the Session.commit(), the same error will be thrown at Session.initiate().

The full output of the terminal when running the script is also added as an attachment. You will notice that querying the temperature of the card for instance with Session.read_current_temperature() does work successfully, which is why this error seems so strange.

After resetting the NI MAX Configuration Data and restarting, the code will again run as expected, until the next power cycle.

I have also used the NI I/O trace tool to check the communication with the chassis. This further suggests that the error occurs during the Session.commit() step, though it remains unclear why. If needed, I can also upload the results from the trace tool.

We also use the Python API to control the acquisition card in parallel to the generator. None of these issues appear to affect the acquisition side, though of course this uses the NI DAQmx Python API.

System report

OS: Name: Windows Version: 10.0.19045 Bits: 64 Driver: Name: NI-FGEN Version: 21.8.0.49444 Module: Name: nifgen Version: 1.4.6 Python: Version: 3.11.4 | packaged by Anaconda, Inc. | (main, Jul 5 2023, 13:38:37) [MSC v.1916 64 bit (AMD64)]

Steps to reproduce issue:

Running the following python script will result in the desired waveform generation when the script is run the first time (and subsequent times) after resetting the NI MAX Configuration Database. However, after power cycling the hardware and laptop, running the same script will return the error message:

import nifgen
from nifgen import enums
from numpy.fft import fft
from scipy.signal import chirp
import numpy as np
import time

#define generation and aqcuisition parameters:
sample_rate=20e6 #gen
sample_number=2e6 #gen
amplitude = 2 #Volt
offset_ = 0 #Volt
lower_freq = 20e3
upper_freq = 500e3
n_avg = 20
#create waveform array
end_time = sample_number/sample_rate
time_points = np.linspace(0,end_time,int(sample_number))
waveform = chirp(time_points,lower_freq, end_time, upper_freq)

def call_ni_function(sample_rate, sample_number, waveform, n_avg, amplitude, offset_):
    options = {'simulate': False}
    fgen = nifgen.Session(resource_name='PXI1Slot2',channel_name='0', reset_device=True, options=options)
    fgen.output_mode = nifgen.OutputMode.ARB
    fgen.arb_sample_rate = sample_rate
    waveform_handle = fgen.create_waveform(waveform_data_array=waveform)
    fgen.trigger_mode = enums.TriggerMode.SINGLE
    fgen.arb_marker_position=0
    fgen.marker_event_output_terminal="PXI_Trig1"
    fgen.output_impedance=50;        
    fgen.arb_waveform_handle = waveform_handle
    fgen.configure_arb_waveform(waveform_handle=waveform_handle, gain=amplitude, offset=offset_)
    print(fgen.read_current_temperature())
    fgen.commit()

    i=1
    while i<=n_avg:
        fgen.initiate()
        time.sleep(1)
        fgen.abort()        
        i+=1

    fgen.close()
    return 1

call_ni_function(sample_rate, sample_number,  waveform, n_avg, amplitude, offset_)
drjk94 commented 6 months ago

The output when the script fails is: 38.875 Traceback (most recent call last): File "C:\Users\jkeukeli\OneDrive - UGent\ATP\Tools\Small Chassis US\Clone\USB card\datacomparison\exc_pxi_meas_pxi\call_ni_generation.py", line 62, in call_ni_function(sample_rate, sample_number, waveform, navg, amplitude, offset) File "C:\Users\jkeukeli\OneDrive - UGent\ATP\Tools\Small Chassis US\Clone\USB card\datacomparison\exc_pxi_meas_pxi\call_ni_generation.py", line 46, in call_ni_function fgen.commit() File "C:\Users\jkeukeli\AppData\Roaming\Python\Python311\site-packages\nifgen\session.py", line 39, in aux return f(*xs, **kws) ^^^^^^^^^^^^^ File "C:\Users\jkeukeli\AppData\Roaming\Python\Python311\site-packages\nifgen\session.py", line 3337, in commit self._interpreter.commit() File "C:\Users\jkeukeli\AppData\Roaming\Python\Python311\site-packages\nifgen_library_interpreter.py", line 175, in commit errors.handle_error(self, error_code, ignore_warnings=False, is_error_handling=False) File "C:\Users\jkeukeli\AppData\Roaming\Python\Python311\site-packages\nifgen\errors.py", line 123, in handle_error raise DriverError(code, description) nifgen.errors.DriverError: -88717: No device by the given name was found.

drjk94 commented 6 months ago

I should stress that when executed the first time (and subsequent times untill the device is restarted) after performing a reset of the NI MAX database, the code will execute just fine, producing the desired waveform on the output.

drjk94 commented 6 months ago

I no longer believe this to be a purely Python API related issue. I was able to discover the offending line of code is actually the line: fgen.marker_event_output_terminal="PXI_Trig1

However, it seems when using Instrument Studio to sent a marker event through the PXI_Trig1 line, Instrument Studio will also give an error after the second power cycle, even if the Python script was not run during the first power cycle.

tobiasgordon commented 6 months ago

Hi, @drjk94, I appreciate you posting this question.

I'm glad to see you can reproduce the issue outside of Python as this does not look like something that should be API-specific.

Can you provide more information related to your laptop? I am specifically interested in whether it is using Thunderbolt 4. There are known issues using integrated Thunderbolt 4 controllers (Maple Ridge, device ID 0x1137) with NI products. These issues tend to present as intermittent failures when performing certain operations on a variety of devices. Please check the hardware ID for the Thunderbolt Controller in Device Manager.

Here's what that looks like for an Intel NUC's Thunderbolt controller: 102ed523-8cc2-440f-aa15-a98670408b7e

NI is engaged with Intel on root causing the problem with these Maple Ridge controllers, but for now, unfortunately the only proven solution is to use either a discrete Thunderbolt 4 controller or to use a different connectivity solution (e.g. MXI or Thunderbolt 3).

If you can confirm that you are not using a Maple Ridge controller, we can investigate other avenues to get your system running.

marcoskirsch commented 6 months ago

This sounds like an issue with trigger routing. It can be due to:

  1. User error (is "PXI_Trig1" a valid string? looks like it) or
  2. NI Routing Coordinator service being in a bad state, our drivers use for creating trigger routes.

Confirm taking out fgen.marker_event_output_terminal="PXI_Trig1" makes the issue go away. Make sure your chassis is identified in MAX. Try different output terminals, maybe "/PXI1Slot2/PXI_Trig1"

Let us know! Thanks

drjk94 commented 6 months ago

@tobiasgordon Hi, thanks for your response. I've checked and in the case of my PC I find these details: image

In the Thunderbolt Control Center app I can also find these details, suggesting it's indeed a Gen 4 port: image

@marcoskirsch

Hi, thanks for your response. removing the line fgen.marker_event_output_terminal="PXI_Trig1" makes the issue go away. The chassis and generator cards are correctly identified in NI MAX. During the first power cycle after a reset (when everything normally works), in case an incorrect resource name is given you get a different error code. Don't quite recall the code but it clearly suggests an incorrect resource name. During the second power cycle after a reset, you get the error nifgen.errors.DriverError: -8871 regardless of which name you enter. I've also tried using other output terminals, like the PFI0 which is located on the front side of the generator card. The same error code is returned.

I have now also tested using LabVIEW and the result is the same: after the second power cycle, error -88717 is returned whenever trying to acces any terminal besides the 2 analog output terminals.

As suggested, I am now certain the issue is related to the thunderbolt connection/remote control unit. I will likely open a seperate ticket with NI. It's clear though that it doesn't affect all cards, because there is also an acquisition card in the unit that has no such issues.

Thank you for your comments!

drjk94 commented 6 months ago

I figured I would post this here in case others stumble upon this issue.

After raising a ticket with NI I was told to try disabling Kernal DMA protection as is explained here .

This did indeed solved the issue, though feels as very much a work around. Given our PC's are also company managed I had to jump through some hoops to achieve this. I hope a proper fix is implemented soon.

marcoskirsch commented 6 months ago

I want to confirm some of the information.

Before you followed the steps in "Disabling Kernel DMA Protection To Avoid BSOD When Using NI-MXI Card", the program would work...

  1. when executed the first time (and subsequent times untill the device is restarted) after performing a reset of the NI MAX database
  2. when _removing the line fgen.marker_event_output_terminal="PXITrig1"

And this is pretty reliable? I will try to find out a bit more about that Knowledge Base article and if the NI-FGEN driver is doing something it should not.

marcoskirsch commented 6 months ago

Closing issue since this is not a nimi-python bug.

drjk94 commented 6 months ago

I want to confirm some of the information.

Before you followed the steps in "Disabling Kernel DMA Protection To Avoid BSOD When Using NI-MXI Card", the program would work...

  1. when executed the first time (and subsequent times untill the device is restarted) after performing a reset of the NI MAX database
  2. when _removing the line fgen.marker_event_output_terminal="PXITrig1"

And this is pretty reliable? I will try to find out a bit more about that Knowledge Base article and if the NI-FGEN driver is doing something it should not.

Dear Marco, Indeed in those cases the code would work. I would also add it's not specific to "PXI_Trig1" but rather any output or input line other than the two analog output channels (Like PFI0, PXI_Trig2, etc) I enquired with NI if what I experienced was a known issue and if there are any plans to fix the issue in the form of a driver/firmware update. I was told this is more likely a Windows issue (my version of Windows is Windows 10 enterprise 22H2 build 19045.4046) and similar problems were experienced before as in this article

Thank you for your assistance. I apologize for posting my question in the wrong forum, but it wasn't clear initially that the issue was not nimi-python related.