Host lookups do not cache when there is no match for site key

swicken-dotcms commented 8 months ago

Parent Issue

No response

Problem Statement

When looking up host names the function bySiteName we only cache the result if we find a site. We need to also cache the result if we do not find a site, as subsequent requests will always do full table scans looking for that site.

Steps to Reproduce

I don't have reproduction steps, but you can see the code here: https://github.com/dotCMS/core/blob/7c4a68324138f8c818da867512f4778411655ee4/dotCMS/src/main/java/com/dotmarketing/portlets/contentlet/business/HostFactoryImpl.java#L176

Specifically if the result set is empty, we return null without caching anything:

                if (dbResults.isEmpty()) {
                    return null;
                }

This resulted in significant performance issues in a production instance with many rows in their contentlet table. They were making requests to the dotcms.cloud URL, which did not match a site key, and the prerender plugin now does site lookups on every request to determine if there is a configuration.

Acceptance Criteria

Cache empty results to avoid duplicate lookups

dotCMS Version

23.10, master

Proposed Objective

Application Performance

Proposed Priority

Priority 2 - Important

External Links... Slack Conversations, Support Tickets, Figma Designs, etc.

No response

Assumptions & Initiation Needs

No response

Quality Assurance Notes & Workarounds

No response

Sub-Tasks & Estimates

No response

wezell commented 7 months ago

Need to add a 404_cache pattern here.

erickgonzalez commented 1 month ago

Tested locally with debug mode; it now uses cache.

wezell commented 1 month ago

Please do an ab performance test on this where the hostname does not exist in dotCMS (which would fall back to default site) vs. a url that is on a known hostname.

erickgonzalez commented 1 month ago

AB Testing Results:

Where the hostname does not exist in dotCMS


erickgonzalez@EricksMBPWork ~ % ab -k -n 10000 -c 20 http://non-exist-testing-erick.dotcms.com:8082/testpage
This is ApacheBench, Version 2.3 <$Revision: 1913912 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking non-exist-testing-erick.dotcms.com (be patient) Completed 1000 requests Completed 2000 requests Completed 3000 requests Completed 4000 requests Completed 5000 requests Completed 6000 requests Completed 7000 requests Completed 8000 requests Completed 9000 requests Completed 10000 requests Finished 10000 requests

Server Software: da22de0439bf|04b759a07c Server Hostname: non-exist-testing-erick.dotcms.com Server Port: 8082

Document Path: /testpage Document Length: 27092 bytes

Concurrency Level: 20 Time taken for tests: 4.445 seconds Complete requests: 10000 Failed requests: 0 Keep-Alive requests: 0 Total transferred: 278080000 bytes HTML transferred: 270920000 bytes Requests per second: 2249.88 [#/sec] (mean) Time per request: 8.889 [ms] (mean) Time per request: 0.444 [ms] (mean, across all concurrent requests) Transfer rate: 61098.20 [Kbytes/sec] received

Connection Times (ms) min mean[+/-sd] median max Connect: 0 0 0.3 0 20 Processing: 2 9 9.9 6 92 Waiting: 1 8 9.4 6 88 Total: 2 9 9.9 6 92

Percentage of the requests served within a certain time (ms) 50% 6 66% 7 75% 8 80% 9 90% 14 95% 22 98% 47 99% 64 100% 92 (longest request)


* URL on a known hostname

erickgonzalez@EricksMBPWork ~ % ab -k -n 10000 -c 20 http://testing-erick.dotcms.com:8082/testpage This is ApacheBench, Version 2.3 <$Revision: 1913912 $> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking testing-erick.dotcms.com (be patient) Completed 1000 requests Completed 2000 requests Completed 3000 requests Completed 4000 requests Completed 5000 requests Completed 6000 requests Completed 7000 requests Completed 8000 requests Completed 9000 requests Completed 10000 requests Finished 10000 requests

Server Software: da22de0439bf|04b759a07c Server Hostname: testing-erick.dotcms.com Server Port: 8082

Document Path: /testpage Document Length: 27100 bytes

Concurrency Level: 20 Time taken for tests: 3.987 seconds Complete requests: 10000 Failed requests: 0 Keep-Alive requests: 0 Total transferred: 278160000 bytes HTML transferred: 271000000 bytes Requests per second: 2508.01 [#/sec] (mean) Time per request: 7.974 [ms] (mean) Time per request: 0.399 [ms] (mean, across all concurrent requests) Transfer rate: 68127.65 [Kbytes/sec] received

Connection Times (ms) min mean[+/-sd] median max Connect: 0 0 0.2 0 4 Processing: 1 8 11.0 5 103 Waiting: 0 7 10.1 5 102 Total: 1 8 11.0 5 103

Percentage of the requests served within a certain time (ms) 50% 5 66% 6 75% 7 80% 7 90% 11 95% 25 98% 51 99% 65 100% 103 (longest request)

dotCMS / core