Imperva Cloud WAF connector failing

[philfy20]

I have deployed this data connector for imperva cloud WAF - https://github.com/Azure/Azure-Sentinel/tree/master/Solutions/ImpervaCloudWAF/Data%20Connectors and the connector is not working. Below is an extract of the error I am getting

2024-01-10T06:00:09Z [Error] Something wrong. Exception error text: local variable 'events_data' referenced before assignment 2024-01-10T06:00:09Z [Information] Downloading file 1744_1750947.log 2024-01-10T06:00:09Z [Information] Successfully downloaded file: 1744_1750946.log 2024-01-10T06:00:10Z [Information] Successfully downloaded file: 1744_1750947.log 2024-01-10T06:00:10Z [Error] Something wrong. Exception error text: local variable 'events_data' referenced before assignment 2024-01-10T06:00:10Z [Information] Downloading file 1744_1750948.log 2024-01-10T06:00:10Z [Information] Unpacking and decrypting file 1744_1750947.log 2024-01-10T06:00:10Z [Information] Successfully downloaded file: 1744_1750948.log 2024-01-10T06:00:10Z [Information] Unpacking and decrypting file 1744_1750948.log 2024-01-10T06:00:10Z [Error] Something wrong. Exception error text: local variable 'events_data' referenced before assignment 2024-01-10T06:00:10Z [Information] Downloading file 1744_1750949.log 2024-01-10T06:00:10Z [Information] Unpacking and decrypting file 1744_1750949.log 2024-01-10T06:00:10Z [Error] Something wrong. Exception error text: local variable 'events_data' referenced before assignment 2024-01-10T06:00:10Z [Information] Downloading file 1744_1750950.log 2024-01-10T06:00:10Z [Information] Successfully downloaded file: 1744_1750949.log 2024-01-10T06:00:10Z [Information] Successfully downloaded file: 1744_1750950.log 2024-01-10T06:00:10Z [Information] Unpacking and decrypting file 1744_1750950.log 2024-01-10T06:00:10Z [Error] Something wrong. Exception error text: local variable 'events_data' referenced before assignment

Looks like it downloads the files and then some sort of error happens which stops it from sending the logs the LA workspace

[github-actions[bot]]

Thank you for submitting an Issue to the Azure Sentinel GitHub repo! You should expect an initial response to your Issue from the team within 5 business days. Note that this response may be delayed during holiday periods. For urgent, production-affecting issues please raise a support ticket via the Azure Portal.

[v-sudkharat]

Hi @philfy20, Thanks for flagging this issue, we will investigate this issue and get back to you with some updates by 16-01-2024. Thanks!

[v-sudkharat]

Hi @philfy20, could you please check the function app by updating the WEBSITE_RUN_FROM_PACKAGE with below shared URL in the function app. and let us know if function app is not breaking.

https://github.com/Azure/Azure-Sentinel/raw/v-mchatla/ImpervaCloudWAF-8602IssueFix/Solutions/ImpervaCloudWAF/Data%20Connectors/ImpervaWAFCloudSentinelConn.zip

Once it gets updated, please restart the function app.

Thanks!

[philfy20]

Hi @v-sudkharat, same error when updating the WEBSITE_RUN_FROM_PACKAGE to the requested URL.

Error below

2024-01-10T22:30:11Z [Error] Something wrong. Exception error text: local variable 'events_data' referenced before assignment 2024-01-10T22:30:11Z [Information] Downloading file 1744_1752001.log 2024-01-10T22:30:11Z [Information] Successfully downloaded file: 1744_1752001.log 2024-01-10T22:30:11Z [Information] Unpacking and decrypting file 1744_1752001.log 2024-01-10T22:30:11Z [Error] Something wrong. Exception error text: local variable 'events_data' referenced before assignment 2024-01-10T22:30:11Z [Information] Downloading file 1744_1752002.log 2024-01-10T22:30:11Z [Information] Successfully downloaded file: 1744_1752002.log 2024-01-10T22:30:11Z [Error] Something wrong. Exception error text: local variable 'events_data' referenced before assignment 2024-01-10T22:30:11Z [Information] Downloading file 1744_1752003.log 2024-01-10T22:30:11Z [Information] Unpacking and decrypting file 1744_1752002.log 2024-01-10T22:30:11Z [Information] Successfully downloaded file: 1744_1752003.log 2024-01-10T22:30:11Z [Information] Unpacking and decrypting file 1744_1752003.log 2024-01-10T22:30:11Z [Error] Something wrong. Exception error text: local variable 'events_data' referenced before assignment

[v-sudkharat]

@philfy20, thanks for your response. we will check on this and get back to you by - 16-01-2024. Thanks!

[v-sudkharat]

Hi @philfy20, could you please share one of the failing file or content of that file? so we can check it from our end. Thanks!

[philfy20]

hi @v-sudkharat What file are you referring to and where can I find it? looking at the storage account that is deployed with the solution there are no logs files showing.

The only file there is the funcstatemarkerfile. Thanks

[v-sudkharat]

Hi @philfy20, we required one of the .log fie Example - 1744_1752002.log. Thanks!

[v-sudkharat]

Hi @philfy20, We are trying to modify the code according to the error. Could you please check with below link by updating WEBSITE_RUN_FROM_PACKAGE - https://github.com/Azure/Azure-Sentinel/raw/868ba7dff6779a7d163f93dafbf2ed1d1b1d9228/Solutions/ImpervaCloudWAF/Data%20Connectors/ImpervaWAFCloudSentinelConn.zip

And share the result with us. Thanks!

[philfy20]

hi @v-sudkharat, function app now is running through however showing files have been processed

also the function app does not save these log files that are been downloading from impreva so I can't pull them to see the contents

[v-sudkharat]

Hi @philfy20, thanks for update. Could you please try this one -https://github.com/Azure/Azure-Sentinel/raw/c73f71cbbc345f02933127827bb877055b995de4/Solutions/ImpervaCloudWAF/Data%20Connectors/ImpervaWAFCloudSentinelConn.zip We can't replicate this issue due to unavailability of credentials. so we are making modifications in code locally according to your shared error logs. Is it possible to you share the temporary credentials with us? so we can replicate this issue and check on it. You can share the details with us on this mail - v-sudkharat@microsoft.com

Thanks!

[philfy20]

sorry no chance I can't share customer creds with you goes against all their policies and its stored in an Azure key vault. Seeing MS built the connector I am hoping they have access to some creds?

Also note that I have this connector working fine with other customers however for some reason this customer does not work. have checked API keys permissions and all looks correct.

Thanks,

[v-sudkharat]

Hi @philfy20, okay we understand on this. please check with the WEBSITE_RUN_FROM_PACKAGE - https://github.com/Azure/Azure-Sentinel/raw/c73f71cbbc345f02933127827bb877055b995de4/Solutions/ImpervaCloudWAF/Data%20Connectors/ImpervaWAFCloudSentinelConn.zip

And also is it possible to share this file with us? -file: 1744_1760068.log, it will help us to look into it.

Thanks!

[philfy20]

hi @v-sudkharat

same outcome with the updated WEBSITE_RUN_FROM_PACKAGE

2024-01-17T05:30:03Z [Information] Downloading file 1744_1762355.log 2024-01-17T05:30:04Z [Information] Successfully downloaded file: 1744_1762355.log 2024-01-17T05:30:04Z [Information] Unpacking and decrypting file 1744_1762355.log 2024-01-17T05:30:04Z [Information] Chunk was processed with 0 events from the file: 1744_1762355.log 2024-01-17T05:30:04Z [Information] Downloading file 1744_1762356.log 2024-01-17T05:30:04Z [Information] Successfully downloaded file: 1744_1762356.log 2024-01-17T05:30:04Z [Information] Unpacking and decrypting file 1744_1762356.log 2024-01-17T05:30:04Z [Information] Chunk was processed with 0 events from the file: 1744_1762356.log 2024-01-17T05:30:04Z [Information] Downloading file 1744_1762357.log 2024-01-17T05:30:05Z [Information] Successfully downloaded file: 1744_1762357.log 2024-01-17T05:30:05Z [Information] Unpacking and decrypting file 1744_1762357.log 2024-01-17T05:30:05Z [Information] Chunk was processed with 0 events from the file: 1744_1762357.log 2024-01-17T05:30:05Z [Information] Downloading file 1744_1762358.log 2024-01-17T05:30:05Z [Information] Successfully downloaded file: 1744_1762358.log 2024-01-17T05:30:05Z [Information] Unpacking and decrypting file 1744_1762358.log 2024-01-17T05:30:05Z [Information] Chunk was processed with 0 events from the file: 1744_1762358.log 2024-01-17T05:30:05Z [Information] Downloading file 1744_1762359.log 2024-01-17T05:30:06Z [Information] Successfully downloaded file: 1744_1762359.log 2024-01-17T05:30:06Z [Information] Unpacking and decrypting file 1744_1762359.log 2024-01-17T05:30:06Z [Information] Chunk was processed with 0 events from the file: 1744_1762359.log 2024-01-17T05:30:06Z [Information] Downloading file 1744_1762360.log 2024-01-17T05:30:06Z [Information] Successfully downloaded file: 1744_1762360.log 2024-01-17T05:30:06Z [Information] Unpacking and decrypting file 1744_1762360.log 2024-01-17T05:30:06Z [Information] Chunk was processed with 0 events from the file: 1744_1762360.log 2024-01-17T05:30:06Z [Information] Downloading file 1744_1762361.log 2024-01-17T05:30:07Z [Information] Successfully downloaded file: 1744_1762361.log 2024-01-17T05:30:07Z [Information] Unpacking and decrypting file 1744_1762361.log 2024-01-17T05:30:07Z [Information] Chunk was processed with 0 events from the file: 1744_1762361.log 2024-01-17T05:30:07Z [Information] Downloading file 1744_1762362.log 2024-01-17T05:30:07Z [Information] Successfully downloaded file: 1744_1762362.log 2024-01-17T05:30:07Z [Information] Unpacking and decrypting file 1744_1762362.log 2024-01-17T05:30:07Z [Information] Chunk was processed with 0 events from the file: 1744_1762362.log 2024-01-17T05:30:07Z [Information] Downloading file 1744_1762363.log 2024-01-17T05:30:07Z [Information] Successfully downloaded file: 1744_1762363.log 2024-01-17T05:30:07Z [Information] Unpacking and decrypting file 1744_1762363.log 2024-01-17T05:30:07Z [Information] Chunk was processed with 0 events from the file: 1744_1762363.log 2024-01-17T05:30:07Z [Information] Downloading file 1744_1762364.log 2024-01-17T05:30:08Z [Information] Successfully downloaded file: 1744_1762364.log 2024-01-17T05:30:08Z [Information] Unpacking and decrypting file 1744_1762364.log 2024-01-17T05:30:08Z [Information] Chunk was processed with 0 events from the file: 1744_1762364.log 2024-01-17T05:30:08Z [Information] Downloading file 1744_1762365.log 2024-01-17T05:30:08Z [Information] Successfully downloaded file: 1744_1762365.log 2024-01-17T05:30:08Z [Information] Unpacking and decrypting file 1744_1762365.log 2024-01-17T05:30:08Z [Information] Chunk was processed with 0 events from the file: 1744_1762365.log

[v-sudkharat]

Hi @philfy20, We have updated the zip package, could you please try with below once -

https://github.com/Azure/Azure-Sentinel/raw/d299ac3f71c4c790ced128a376dc8a8aa7566295/Solutions/ImpervaCloudWAF/Data%20Connectors/ImpervaWAFCloudSentinelConn.zip

If issue still persist, Can we have a call on this? Could you please share your mail id with us so we can connect with you. Please can you mail us on this mail id - v-muuppugund@microsoft.com, v-sudkharat@microsoft.com

Thanks!

[v-sudkharat]

Hi @philfy20, We are waiting for your response on comment. Thanks!

[philfy20]

@v-sudkharat. I have done a screen share with the customer and found that they have encrypt logs enabled attached to that API account (even though I said turn it off). Removing the encrypt logs has resolved the issue. Recommending adding in the data connector instructions for Imperva cloud waf to not enable encryption on the Imperva API account. Thanks

[v-sudkharat]

Hi @philfy20, thanks you for sharing the information with us. We will share your valuable feedback with our respective team on this. We just want to know in customer function app which URL you kept for WEBSITE_RUN_FROM_PACKAGE? the Old one or those which we have share over recent comments. And can we close this issue from GitHub?

[philfy20]

using - https://aka.ms/sentinel-impervawafcloud-functionapp and you can close ticket. Thanks

[v-sudkharat]

@philfy20, thanks for confirmation, closing this issue. If you still need support for this issue, feel free to re-open it any time. Thank you for your co-operation.