ImpervaCloudWAF "last_file_point" function bug

[A773ycat]

The "last_file_point" function fails if the file name of the last log file processed is not found in the current "logs.index" file retrieved from the location referred to by the application setting "ImpervaLogServerURI".

To Reproduce Use of prior versions of the connector that were unable to parse log files or stop the function app until the file name of the last log file processed is no longer included in the "logs.index" file retrieved from the URI defined by the "ImpervaLogServerURI" application setting then restart the function app. Log files will no longer be processed and ongoing use of the Data Connector only results in wasting processing cycles, small amounts of bandwidth, and electricity.

Expected behavior If the file name assigned to the variable, "past_file" is not found in the current "logs.index", all available log files should be processed with the "past_file" variable being set to the name of the subsequent files available as each is processed.

Screenshots N/A

Desktop (please complete the following information): N/A

Smartphone (please complete the following information): N/A

Additional context Recommended fix: Within the file, "init.py", in the "last_file_point" function, change the line "if past_file is not None:" to "if past_file is not None and self.files_array.index(past_file):" If the file name of the last log file processed is no longer included in the "logs.index" file, this will allow the code to continue to the next "else:" section instead of ending in error.

[github-actions[bot]]

Thank you for submitting an Issue to the Azure Sentinel GitHub repo! You should expect an initial response to your Issue from the team within 5 business days. Note that this response may be delayed during holiday periods. For urgent, production-affecting issues please raise a support ticket via the Azure Portal.

[github-actions[bot]]

Thank you for submitting an Issue to the Azure Sentinel GitHub repo! You should expect an initial response to your Issue from the team within 5 business days. Note that this response may be delayed during holiday periods. For urgent, production-affecting issues please raise a support ticket via the Azure Portal.

[github-actions[bot]]

Thank you for submitting an Issue to the Azure Sentinel GitHub repo! You should expect an initial response to your Issue from the team within 5 business days. Note that this response may be delayed during holiday periods. For urgent, production-affecting issues please raise a support ticket via the Azure Portal.

[github-actions[bot]]

Thank you for submitting an Issue to the Azure Sentinel GitHub repo! You should expect an initial response to your Issue from the team within 5 business days. Note that this response may be delayed during holiday periods. For urgent, production-affecting issues please raise a support ticket via the Azure Portal.

[v-rbajaj]

Hi @A773ycat, thanks for flagging this issue, we will soon get back to you on this.

[github-actions[bot]]

Thank you for submitting an Issue to the Azure Sentinel GitHub repo! You should expect an initial response to your Issue from the team within 5 business days. Note that this response may be delayed during holiday periods. For urgent, production-affecting issues please raise a support ticket via the Azure Portal.

[github-actions[bot]]

Thank you for submitting an Issue to the Azure Sentinel GitHub repo! You should expect an initial response to your Issue from the team within 5 business days. Note that this response may be delayed during holiday periods. For urgent, production-affecting issues please raise a support ticket via the Azure Portal.

[v-rbajaj]

Hi @A773ycat, can you please share the screenshot of the error message you are receiving?

[A773ycat]

Before changing the line "if past_file is not None:" to "if past_file is not None and self.files_array.index(past_file):" in the "last_file_point" function, the error observed appeared as shown in the following screen excerpt:

The error is thrown, no processing of any log files occurs, and the process completes "successfully" (albeit, presumably not as intended) repeating this same error each time the function "ImpervaWAFCloudSentinelConnector" triggers. Adding a check that the value assigned to "past_file" exists in the "logs.index" file seems to remedy the problem.

[v-rbajaj]

We are looking this and we will get back to you soon.

[v-rbajaj]

We are getting in touch with concerned team, we will get back to you.

[v-rbajaj]

We are in touch with concerned team, we will get back to you once we receive an update.

[v-rbajaj]

Hi @A773ycat, can you please update WEBSITE_RUN_FROM_PACKAGE with this URL in the function app

https://github.com/Azure/Azure-Sentinel/raw/v-mchatla/ImpervaCloudWAF-8602IssueFix/Solutions/ImpervaCloudWAF/Data%20Connectors/ImpervaWAFCloudSentinelConn.zip

Then do restart the Function App and check if you are still facing the issue .

If this fixes the issue then we will merge the changes to master. Please let us know

[A773ycat]

Hello,

I appreciate the attention to the matter. It will take me a couple of days to fit the work into the current task load. I will report back once the updated package has been evaluated.

[v-rbajaj]

Thanks @A773ycat, let us know once you update the package.

[v-rbajaj]

Hi @A773ycat, please let us know if the provided update was helpful.

[A773ycat]

Hello v-rbajaj,

Testing with the provided update was successful. Within the Function App, assigning the configuration setting "WEBSITE_RUN_FROM_PACKAGE" with"https://github.com/Azure/Azure-Sentinel/raw/v-mchatla/ImpervaCloudWAF-8602IssueFix/Solutions/ImpervaCloudWAF/Data%20Connectors/ImpervaWAFCloudSentinelConn.zip", saving the change, then restarting the Function App (may not have been needed as the "save" action may have done this but just to be sure...) showed the log files being processed properly and the content available in the table (~10-15 minute lag for the data to appear presumably due to the 10 minute processing cycle).

Thank You

[v-rbajaj]

Hello v-rbajaj, Testing with the provided update was successful. Within the Function App, assigning the configuration setting "WEBSITE_RUN_FROM_PACKAGE" with"https://github.com/Azure/Azure-Sentinel/raw/v-mchatla/ImpervaCloudWAF-8602IssueFix/Solutions/ImpervaCloudWAF/Data%20Connectors/ImpervaWAFCloudSentinelConn.zip", saving the change, then restarting the Function App (may not have been needed as the "save" action may have done this but just to be sure...) showed the log files being processed properly and the content available in the table (~10-15 minute lag for the data to appear presumably due to the 10 minute processing cycle). Thank You

Thanks for letting us know. We will push these changes on master and closing this issue.

[github-actions[bot]]

Thank you for submitting an Issue to the Azure Sentinel GitHub repo! You should expect an initial response to your Issue from the team within 5 business days. Note that this response may be delayed during holiday periods. For urgent, production-affecting issues please raise a support ticket via the Azure Portal.