Sheet consumption stored failed Parent APP

[ceddataspark]

🐛 Bug report

Steps to Reproduce

1. reload multi tenant parent app analyzer

Expected behavior

I expect the reload to complete without giving an error

Actual behavior

The multi tenant enabled parent app analyzer gives an error message while storing sheet consumption to azure blob

I have also made a community post about this https://community.qlik.com/t5/Deployment-Management/OEM-Monitoring-problem-with-parent-tenant-app-analyzer/m-p/2477250#M27843

[danielpilla]

Hi @ceddataspark , Can you ensure that all of the apps are on the same versions and confirm that the vu_qlik_meta_collection_parent_dir variable points to a directory without prior version QVDs?

Cheers,

[ceddataspark]

Hi @danielpilla

Thank you for your answer,

I have already reinstalled all the childs and the parent monitoring multiple times. They are all the exact same version. Furtermore, I have also already cleaned out the whole azure blob and manually deleted all QVD's in the qlik tenants.

Furthermore, the OEM monitoring was working for a day, and then it suddenly didn't work anymore. Below is a screenshot of the reload history.

The problems seem to be caused by the Parent tenant storing the compiled sheet consumption to the storing location. The error is only happening with the "sheet consumption" the other storing procedures are working correctly. Lastly, the parent app analyzer is the only monitoring app that gives this error, the reload analyzer and the entitlement analyzer are working fine.

Thank you kindly in advance,

[ceddataspark]

Hi,

I'm brainstorming here because we are still encountering errors with Sheet consumption. Weirdly enough I was able to do 1 manually reload succesfully, but the automatic and manually reloads afterwards were failing again, while I didn't change / do anything.

Furthermore, I have noticed that the child app_analyzer "sheet consumption.qvd" are not always being generated in the Qlik environment but are still being stored in the Blob. It seems that some "sheet consumption.qvd" are getting stored as an empty file to the blob. Could this be the culprit?

Thank you kindly,

[danielpilla]

Hi @ceddataspark, Could you try altering the end of the get_audit_sheet_consumption subroutine to resemble the following?

If $(vIncrementalQVDsAvailable)=1 And NoOfRows('SheetConsumption')>0 Then 

    IncrementalSheetConsumptionMap:
    Mapping Load Distinct
        _IncrementalSheetConsumptionID,
        1
    Resident SheetConsumption;

    Trace Calling load_qvd on Consumption;
    Call load_qvd('SheetConsumption')
    Trace Calling store_qvd on Consumption;
    Call store_qvd('SheetConsumption');

ElseIf NoOfRows('SheetConsumption')>0 Then

    Trace Calling store_qvd on Consumption;
    Call store_qvd('SheetConsumption');

Else

    Trace Calling load_qvd on Consumption;
    Set vSheetConsumptionNoNewRecords = 1;
    Call load_qvd('SheetConsumption')

End if;

Note that I've removed the Drop Field _IncrementalSheetConsumptionID; line. I have a feeling that is causing the QVDs to not concatenate properly. While in "Parent" mode, on the first reload they will contain the _IncrementalSheetConsumptionID, and then on subsequent loads, they would drop that field and then not autoconcatenate. Can you give that a try?

Cheers,

[ceddataspark]

@danielpilla Thank you for your reply and your thoughts.

After trying the suggested changes, the error doesn't seem to be resolved. What I do think, is that the error is created by the Child apps as these are storing completely empty Sheet-consumption.qvd files into the blob.

For example, there is an child app which isn't being used by the user thus the sheet-consumption.qvd should be empty. After loading the "child" app app-analyzer there won't be any sheet-consumption.qvd stored in the qlik local library, However, for some reason a completely empty sheet-consumption.qvd gets stored onto the azure blob environment as can be seen in the screenshot I provided in my earlier post.

While the error occurs in the parent app, I do think that it's caused by the Child apps storing empty sheet-consumption.qvd into the azure blob, and the parent app later using these to try and compile these empty files, which is probably causing the error, right?

What if we can change the code so if an sheet-consumption is empty, it will not be stored into the blob environment in the same way that an empty sheet-consumption.qvd will not be stored in the local qlik library.

Looking forward for your next magic thoughts :D

[danielpilla]

Hi just to confirm; did you reload all of the child apps with this change and then the parent app with this change? The change is in the child app, however the issue will present itself in the parent app.

[ceddataspark]

@danielpilla I haven't applied the changes to all Childs and the parent app yet, this will take some time haha.

What I did is that I added the changes to 1 child app, and then downloaded the sheet-consumption file that got uploaded to the Azure Blob, which confirmed me that the sheet-consumption.qvd got uploaded there completely empty.

I'm gonna deleted the whole azure blob now, and implement the suggest changes to all apps, and will get back to you with the result.

Thank you for your help!

[ceddataspark]

@danielpilla After deleting all the files in Azure blob, and reinstalling every app analyzer with the suggested changes the error seems to have been fixed for sheet-consumption.qvd. after the initial first manual refresh. However, subsequent automatic refreshes failed due to a new error.

Should we remove the "Drop Field _IncrementalSheetConsumptionID;" from "app consumption" as well?

[danielpilla]

Hi @ceddataspark -- yes, I would test with removing that one, too. What you are seeing with the -20 is the autoconcat not working due to varying schemas in the QVDs due to the conditional drop statement. If removing that solves the problem for you, I will update the app and push out a new version with those changes. But I can confirm that it is a bug from an OEM Dashboard perspective. Thank you for testing, as I do not have a readily available testbed to simulate the issue you have, but it makes sense to me why it is happening now.

[ceddataspark]

@danielpilla Thank you for helping and sharing your perspective.

I will do another full fresh install with the "Drop Field _IncrementalSheetConsumptionID;" removed from sheet-consumption and consumption store procedures.

I will let you know the result once its completed.

[ceddataspark]

@danielpilla Hi there, Update!

After removing the "Drop Field _IncrementalSheetConsumptionID;" line from "App consumption", "Audit Sheet Consumption" and "Audit Export" in the child and parent app analyzer the reloads seem to working fine without any errors.

[danielpilla]

@ceddataspark This is great to hear, thanks for testing! I will make the changes to the app and get a new version pushed out today.