Backend Performance

[fredericalpers]

Initiative

Optimize backend performance for the onOffice for WP-Websites Plugin.

Epic

Analyse and improve the performance of the plugin backend.

User Story

As a real estate agent relying on the onOffice for WP-Websites Plugin, I've experienced performance issues in the backend, particularly when utilizing the Gutenberg Editor and displaying property lists or forms. The presence of redundant database queries and excessive API calls can lead to timeouts and errors. To ensure a fast, reliable and efficient backend experience, I request the optimiziation of the plugin's performance by reviewing and reducing/bundling uncessary API calls and scripts and perhaps implementing a caching mechanism for the backend.

The current backend performance of the onOffice for WP-Websites Plugin reduces productivity, especially when dealing with the Gutenberg Editor and property lists / form displays. The optimization efforts should focus on identifying and rectifying/bundling redundant API calls and scripts to enhance the overall efficiency of the backend.

Tasks

• [x] Conduct a thorough audit of existing API calls made by the onOffice for WP-Websites Plugin in the backend
• [x] Identify the bottlenecks in the system and assess the necessity of each API call, eliminating redundant queries and optimizing frequency of essential calls.
• [x] Reduce the number of round trips. -- Avoid making multiple calls to the same service or database if the data can be aggregated into one call. -- Use a query that returns multiple results sets from the database if possible. -- When working with a loop, make sure to call the database outside of the loop. -- Asynchronous processing for long-running requests -- Compress responses to reduce transfer time
• [x] Review and consolidate JavaScript and CSS scripts utilized in the backend to minimize the number of requests
• [x] Address specific performance challenges related to the Gutenberg Editor -- Introduce a caching mechanism for the backend, allowing frequently accessed data to be stored and retrieved locally to minimize redundant queries. -- Implement a cache expiry strategy to ensure that cached data remains up-to-date, reflecting any changes made through onOffice enterprise.

By addressing the performance issues in the backend and implement optimizations, the onOffice for WP-Websites Plugin can offer real estates agents a smoother and more responsive experience, particulalry when working the Gutenberg Editor and managing property lists and forms.

[yeneastgate]

@fredericalpers Here are the preliminary results we temporarily found in this issue:

• Conduct a thorough audit of existing API calls made by the onOffice for WP-Websites Plugin in the backend

=> Here is a list of APIs Which need optimization:

• GET fields https://apidoc.onoffice.de/actions/informationen-abfragen/feldkonfiguration/
  I see this API is calling twice in one function Ex: getDataOfSortByInput (getOnlyDefaultSortByFields, readFieldnames), ...

• Reduce the number of round trips.

=> This is a list of queries which can be optimized and All functions which can limit the number of calls I see the positions that have complex queries to DB same: Ex: getRecordsSortedAlphabetically, createCustomsLabelsByFormIdQuery, createMultiBaseQuery,...

However, to list "accurately and completely and give more specific details" about "the parts need to refactor". => I need more time to investigate more deeply and respond to you about the estimated time.

• Review and consolidate JavaScript and CSS scripts utilized in the backend to minimize the number of requests

=> The "consolidate JavaScript and CSS scripts" is resolved at https://github.com/onOffice-Web-Org/oo-wp-plugin/issues/573

• Address specific performance challenges related to the Gutenberg Editor

In my understanding, the problem you mentioned:

• Step 1: We embed the shortcode "onOffice WordPress Plugin" in page WordPress
• Step 2: Then perform the action "save or update or reload", The page will load slowly. Because in line https://github.com/onOffice-Web-Org/oo-wp-plugin/blob/f7a4e88c8886096348e92e31c8ebccdf0b128f44/plugin/Controller/DetailViewPostSaveController.php#L158, we hand "save_post" action and has many queries into the database => The page loading speed was reduced. => Your desire: Do you want me to introduce a caching mechanism for the backend and a cache expiry strategy?

https://github.com/onOffice-Web-Org/oo-wp-plugin/assets/104892108/227b46f3-f0bb-419f-b58f-8adcfbb7cc2e

Please confirm to me whether my understanding of your desire is correct? => So that I can continue to investigate and give you our plans and solutions to introduce the cache correctly.

[fredericalpers]

@yeneastgate please go ahead and try to optimize all APIs and investigate further on which queries can be optimized. As mentioned in our meeting we specifically face problems when using the Gutenberg Editor and it's preview. Therefore we want to implement caching. Any other suggestions for performance improvements are also welcome. Thank you :)

[yeneastgate]

@fredericalpers Now, this result for my investigate: 1.List of APIs that need to be optimized

2.Here is the list of queries that need to be optimized Note: I used the "query monitor" plugin to identify Slow Queries instead of manual

3.For Introduce a caching mechanism for the backend => I have found a caching mechanism 'Object Caching' that may be useful for improving the backend performance. However, I need implement emulator this mechanism thoroughly before determining its feasibility and effectiveness. We will continue to investigate when this issue continues."

[fredericalpers]

@yeneastgate please keep investigating, thank you

[yeneastgate]

=> I have found a caching mechanism 'Object Caching' that may be useful for improving the backend performance. However, I need implement emulator this mechanism thoroughly before determining its feasibility and effectiveness. We will continue to investigate when this issue continues."

@fredericalpers After investigating:

1. The mechanism to load the "Gutenberg editor" of the backend WP is called "Editor Blocks" Rest API (https://developer.wordpress.org/rest-api/reference/blocks/) =>Solutions for optimize performance: I will Introduce a caching mechanism for "Editor Blocks" Rest API to the database.

2. I have implemented this solution at the "43305-backend-performance-v2" branch, and the performance results are presented below.

[fredericalpers]

@yeneastgate Are those results only including the caching mechanism or also other API optimizations?

[yeneastgate]

@fredericalpers this results only for the caching mechanism.

[fredericalpers]

@yeneastgate do you think we can improve the performance more through API call optimization or should we move this part to a different issue?

[yeneastgate]

@fredericalpers

do you think we can improve the performance more through API call optimization

• According to my prediction, Backend Performance can only be improved slightly through "API call optimization".
• I will let you know the exact change number after I implement a test branch.

should we move this part to a different issue?

• I think this is not necessary because all solutions have the same purpose to improve Backend Performance

[yeneastgate]

@fredericalpers

• According to my prediction, Backend Performance can only be improved slightly through "API call optimization".

1. I have implemented this solution at the "https://github.com/onOffice-Web-Org/oo-wp-plugin/tree/43305-backend-performance" branch, and the performance results are presented below (the loading time decreased from 20,38s to 16,18s)

2. While implementing "improve the Backend Performance more through API call optimization". I found "another case" where we can optimize to improve Backend Performance.

Currently, the logic gets the page ID for "Pages using the shortcode". We need to handle two actions in core WordPress ('save_post' and 'wp_trash_post').

However, in the current code, the logic of this part is always called even when the End User does not use the behavior "update page" (save_post) and "trash page" ( wp_trash_post)

=> Solution: We will only load "the logic to get the page ID for 'Pages using the shortcode'" when the End User uses the behavior "update page" and "trash page"

=> I tried implementing in this branch "https://github.com/onOffice-Web-Org/oo-wp-plugin/tree/43305-backend-performance", and here are the results (the loading time decreased from 24,05s to 2,62s)

Please let me know your opinion about this. Thanks!

[yeneastgate]

@fredericalpers To give you an overview of the changes in Backend Performance, I'd like to list the following improvements:

1. The caching mechanism for "Gutenberg editor", the loading time decreased from 14.16s to 13.52s.
2. Through API call optimization and query optimization, the loading time decreased from 20.38s to 16.18s.
3. Will only load "the logic to get the page ID for 'Pages using the shortcode'" when the End User uses the behavior "update page" and "trash page", the loading time decreased from 24.05s to 2.62s.

Note: We have clearly stated details about each solution in the comments above.

[fredericalpers]

@yeneastgate This seems to be promising, thank you :) I will get back to you asap.

[fredericalpers]

@yeneastgate please go ahead and create the pull requests :) thank you

[yeneastgate]

@fredericalpers I created a PR that includes the logic of the following solutions:

1. We will only load "the logic to get the page ID for 'Pages using the shortcode'" when the End User uses the behavior "update page" and "trash page" => Loading time of "Gutenberg editor" in the edit screen decreased from 25.6s to 2.5s

2. Through API call optimization and query optimization => In this PR, we refactored the logic code of the screens: Estate List, Unit List, and 4 forms (Contact, Interest, Owner, Applicant Search). => The loading time of the Estate List has been decreased from 15.06s to 13.46s The loading time of the Owner Form has been decreased from 15.73s to 14.03s

   

Note: During retesting:

1. I realize that the logic of the solution caching mechanism for "Gutenberg editor" has many potential errors because we are caching WP's Rest API and this does not increase performance much (as reported in the comment above https://github.com/onOffice-Web-Org/oo-wp-plugin/issues/701#issuecomment-1888792166) => So I think we should not merge this logic into the PR
2. Optimizing performance for the load preview of "Gutenberg editor", we have reduced it significantly with solution 1 (only load "the logic to get the page ID ) above

[BenjaminZekavica]

@dai-eastgate @yeneastgate, here's some information about our WP blocks:

• We utilize ACF Blocks for development, configuring all blocks for Preview Mode. Clicking inside a block reveals all ACF fields, while clicking outside switches the editor to Preview Mode.

• In this PR, we encountered performance issues on our staging environment. Clicking away from a block takes approximately 3-4 seconds to display the final preview of the shortcode.

[andernath]

@yeneastgate and @dai-eastgate we will discuss, if Preview Mode of ACF Fields is necessary for the first steps. I will contact you later.