search

Azure / iotedge

The IoT Edge OSS project
MIT License
1.46k stars 458 forks source link

Azure Stream analytics module on edge not connected #6301

Open thecyrusdh opened 2 years ago

thecyrusdh commented 2 years ago

Hi,

I am having a problem using Azure IoT Stream analytics on edge. In particular, I have set a pipeline in this way (over IoT Edge device):

Azure IoT Publisher Edge Module -> Azure Stream Analytics Module -> Azure IoT Edge module -> $Upstream

It sometimes happen that the local connection for Azure Stream Analytics module on edge goes down.

The only module that reports some information seems to be the edgeHub module which reports:

[WRN] - Module <device>/<stream analytics module> is not connected

After this log entry [INF] - Entering periodic task to reauthenticate connected clients

there is always the same message [WRN] - Module <device>/<stream analytics module> is not connected

and the module is not connected. There is a data loss until the module is restarted manually then the connection is established again.

How can I fix this?

I have the same issue with: mcr.microsoft.com/azure-stream-analytics/azureiotedge:1.0.8 and mcr.microsoft.com/azure-stream-analytics/azureiotedge:1.0.10

Thanks Ciro

jlian commented 2 years ago

Hi @thecyrusdh would you be able to show full logs and edgeHub and the stream analytics module itself?

thecyrusdh commented 2 years ago

Hi @jlian ,

these are the full logs for the two modules.

iiot-saedge-multiplexed-values_log.txt edgeHub_log1.txt

Stream analytics module was restarted the 21st of April at 10.20 (as reported in the logs).

I would like to report another strange issue, after the module was restarted, the messages collected and that should have stored were lost and not sent.

Any ideas? Thanks Ciro

thecyrusdh commented 2 years ago

Hi @jlian any news?

and-rewsmith commented 2 years ago

@jlian Can you please pull in Xin Tian for IoT Edge related troubleshooting for ASA module? Having trouble finding their github account.

jlian commented 2 years ago

Looks like Xin Tian is out of office until May 3.

@thecyrusdh is this urgent?

thecyrusdh commented 2 years ago

Hi @jlian,

yes, we had this problem in production and this has blocked the data flow with data loss (the store and forward functionality didn't work as expected because of this issue). I have afraid that it could happen again. I can't monitor the edge 24/7.

and-rewsmith commented 2 years ago

@thecyrusdh Hi Ciro. I believe you are in communications with customer services for Azure Stream Analytics, but if not please let me know. After discussing with Xin, we found ASA module is on an old version of C# Devices.Client SDK which is likely the issue. That is the first step to try to mitigate this and Xin said it will be included in the next release.

xitia commented 2 years ago

Per suggestion from Iotedge team, we will update client sdk version in the ASA module, which should be available in the next release (Late May to mid June).

Maci3jPy commented 2 years ago

Hi, are there any updates regarding these? I encountered the same problem in a production environment, which led to lost in the data until I noticed that's something is not right. I needed to manually reset the module.

github-actions[bot] commented 2 years ago

This issue is being marked as stale because it has been open for 30 days with no activity.

and-rewsmith commented 2 years ago

This slipped through the cracks, sorry for the late reply. @xitia Did ASA module get updated?

xitia commented 2 years ago

Hi, unfortunately we delayed the image release to July and plan to release a beta image with SDK update in coming few weeks to let customer try with the change first. Will update this thread once the new image is released.

Maci3jPy commented 1 year ago

Hi, is the issue fixed in 1.0.11?

Arjun-Sivasree commented 11 months ago

I have been facing this issue with 1.0.9 version. We have 1.0.13veraion now as latest available. Is this issue already fixed in latest version? Could someone please confirm?