search

Adobe-Consulting-Services / acs-aem-commons

http://adobe-consulting-services.github.io/acs-aem-commons/
Apache License 2.0
455 stars 603 forks source link

acs commons report unable to download complete data #2861

Open ranjith129143 opened 2 years ago

ranjith129143 commented 2 years ago

Required Information

Expected Behavior

In report all data will be displayed from start date to end date .

Actual Behavior

In report download csv file not all data is shown some recent date uploaded assets are not shown.

Steps to Reproduce

Starting from a vanilla instance of AEM at the above version, describe in explicit steps how to reproduce. Link to any supporting assets, e.g. content packages, below

Links

_

ranjith129143 commented 2 years ago

please give inputs to resolve for this issue

ranjith129143 commented 2 years ago

Hi team , will you take as critical issue please give me your inputs

davidjgonzalez commented 2 years ago

@ranjith129143 can you provide the parameters/heuristics to re-produce this?

Are there any errors in the error.log?

ranjith129143 commented 2 years ago

Basically It is occurring in production not in lower environments, In production we have 140000 assets but our downloaded report is showing around 85000 assets data only. we all packages ui, content, apps In lower environment we don't have issue so there is no error showing because we have around 12k assets in lower environments.

ranjith129143 commented 2 years ago

we are using start and end parameters in report

davidjgonzalez commented 2 years ago

@ranjith129143 are there any errors in the log when it fails to collect all results? Also, what version of AEM are you on? I know AEM CS has a request timeout limit .. thought that doesnt sound right since you are getting something back..

140k assets is gonna be a hard one to validate locally :P

@klcodanr not sure if you have any thoughts off the top of your head.

ranjith129143 commented 2 years ago

Aem version 6.5.9 and acs commons 5.0.6 . In production we don't have access for system console to check logs. As I am it because more number nodes query should search and fetch the data there cause some time out or stopping the process in middle .

On Wed, Jun 15, 2022, 23:27 david g. @.***> wrote:

@ranjith129143 https://github.com/ranjith129143 are there any errors in the log when it fails to collect all results? Also, what version of AEM are you on? I know AEM CS has a request timeout limit .. thought that doesnt sound right since you are getting something back..

140k assets is gonna be a hard one to validate locally :P

@klcodanr https://github.com/klcodanr not sure if you have any thoughts off the top of your head.

— Reply to this email directly, view it on GitHub https://github.com/Adobe-Consulting-Services/acs-aem-commons/issues/2861#issuecomment-1156768943, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGRDIN5DQRBZM7NH2YC5RPLVPIKPPANCNFSM5YXHBXLQ . You are receiving this because you were mentioned.Message ID: @.*** com>

ranjith129143 commented 2 years ago

What if we increase limit configuration query engine service ?

On Wed, Jun 15, 2022, 23:34 Ranjith Kumar @.***> wrote:

Aem version 6.5.9 and acs commons 5.0.6 . In production we don't have access for system console to check logs. As I am it because more number nodes query should search and fetch the data there cause some time out or stopping the process in middle .

On Wed, Jun 15, 2022, 23:27 david g. @.***> wrote:

@ranjith129143 https://github.com/ranjith129143 are there any errors in the log when it fails to collect all results? Also, what version of AEM are you on? I know AEM CS has a request timeout limit .. thought that doesnt sound right since you are getting something back..

140k assets is gonna be a hard one to validate locally :P

@klcodanr https://github.com/klcodanr not sure if you have any thoughts off the top of your head.

— Reply to this email directly, view it on GitHub https://github.com/Adobe-Consulting-Services/acs-aem-commons/issues/2861#issuecomment-1156768943, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGRDIN5DQRBZM7NH2YC5RPLVPIKPPANCNFSM5YXHBXLQ . You are receiving this because you were mentioned.Message ID: @.*** com>

davidjgonzalez commented 2 years ago

i would imagine that you would get 0 results if the query was failing - since the query will fail in oak query engine, even before any results are made available to the ACS report tool.

There's now way you can have your Admin team get you teh logs that contain the report run to see if there's anything of note?

ranjith129143 commented 2 years ago

Sure, will do that.

On Wed, Jun 15, 2022, 23:40 david g. @.***> wrote:

i would imagine that you would get 0 results if the query was failing - since the query will fail in oak query engine, even before any results are made available to the ACS report tool.

There's now way you can have your Admin team get you teh logs that contain the report run to see if there's anything of note?

— Reply to this email directly, view it on GitHub https://github.com/Adobe-Consulting-Services/acs-aem-commons/issues/2861#issuecomment-1156780311, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGRDIN66QFU7FRLWW7SXIE3VPIMCPANCNFSM5YXHBXLQ . You are receiving this because you were mentioned.Message ID: @.*** com>

ranjith129143 commented 2 years ago

MicrosoftTeams-image

ranjith129143 commented 2 years ago

MicrosoftTeams-image (1)

ranjith129143 commented 2 years ago

If we increase this limit 100000 to some higher number there is an issue ?

ranjith129143 commented 2 years ago

Hi David, please give me your inputs

davidjgonzalez commented 2 years ago

Would it be possible to provide the report or the report's query? As a temporary fix you could increase the query read limit, but ultimately you need to either:

1) Create/update an Oak index so the query uses an index 2) Alter the query such that it can use an existing index.

I recommend you plugging the query youre using to run the report into AEM Explain Query [1] and then figure out which you want to optimize, the index or the query [2]

Unfortunately, since you are putting the query into Report Builder, there isnt much the ACS Commons project can do to optimize for your custom query - if you can provide your query, we might be able to provide some pointers on places for you too look for optimizations though.

[1] https://experienceleague.adobe.com/docs/experience-manager-64/administering/operations/operations-dashboard.html?lang=en#explain-query

[2] https://experienceleague.adobe.com/docs/experience-manager-64/developing/bestpractices/troubleshooting-slow-queries.html?lang=en