LUX Fronts Next Steps: Backend Usage Insights (from 1078)

[gigamorph]

Problem Description: The log analysis scripts produce TXT, JSON, and TSV files. Manually, TSV files are then saved as a spreadsheet and formatted. All charts are manually created. #1002 recommends seeking out a more polished solution that would enable the team to gain insights more efficiently. For instance, some such products would enable one to change the period and have all visuals immediately updated. This project’s scripts would have to be rerun and visuals recreated, manually. Start by finding out how far AWS can get us.

Expected Behavior/Solution: Describe the solution to the problem or the expected behavior.

Requirements: List of details required for the completion of the issue or requirements for the feature/bug. This can also include requirements that lie outside of the teams such as new design docs or clarification from an outside source.

Needed for promotion: If an item on the list is not needed, it should be crossed off but not removed.

• [ ] ~Wireframe/Mockup - Mike~
• [ ] ticket clarification
• [ ] next steps identified
• [ ] decision
• [ ] Committee discussions - Sarah
• [ ] Feasibility/Team discussion - Sarah
• [ ] Backend requirements - TBD
• [ ] Frontend requirements- TBD
• [ ] Are new regression tests required for QA - Amy
• [ ] Questions
• what is scope/problem and solution?
• how is it tested?
• examples?

UAT/LUX Examples:

• Example endpoints from LUX to use for development and testing, if applicable.

Dependencies/Blocks:

• Blocked By: Issues that are blocking the completion of the current issue.
• Blocking: Issues being blocked by the completion of the current issue.

Related Github Issues:

• Old Github Tix: https://git.yale.edu/lux-its/marklogic/issues/1078

Related links:

• These links can consist of resources, bugherds, etc.

~Wireframe/Mockup:~ ~Place wireframe/mockup for the proposed solution at end of ticket.~

[roamye]

@gigamorph & @xinjianguo -

From the old ticket ( https://git.yale.edu/lux-its/marklogic/issues/1078 ) it looks like this ticket should be in the 'In Progress' status once it answers the questions of:

• what is scope/problem and solution
• how is it tested?
• examples?

I can format the ticket, but which one of you would be able to provide the information to move this ticket forward?

[roamye]

@xinjianguo - from the 7/3 ticket forming meeting it was noted that you will be the one to provide more information as you worked on this ticket.

[xinjianguo]

@roamye OK let me try to answer these questions:

• what is scope/problem and solution

The scope is limited to MarkLogic cluster usage. Solution is to use AWS CludWatch, Logs Insights.

• how is it tested?

It is tested via AWS CludWatch Dashboards.

• examples?

Some example screen shots as follow:

CPU and network utilization

MarkLogic app server errors

LUX top 10 long queries

Hourly requests count