JavaScript and VTL scripting API Performance Test

[wezell]

Parent Issue

26227

Task

We need to performance test our APIs - this includes:

• raw content query API
• graphqlAPI
• VTL API
• JS API

We should make each api respond as closely as possible to the raw content query api, e.g. to this api

http://127.0.0.1:8082/api/content/render/false/query/+contentType:Blog%20+(conhost:48190c8c-42c4-46af-8d1a-0cd5db894797%20conhost:SYSTEM_HOST)%20+languageId:1%20+deleted:false%20+working:true%20+variant:default/orderby/modDate%20desc

We can run the tests using ab or something similar.

ab -n 10000 -c 25 -k "http://127.0.0.1:8082/api/content/render/false/query/+contentType:Blog%20+(conhost:48190c8c-42c4-46af-8d1a-0cd5db894797%20conhost:SYSTEM_HOST)%20+languageId:1%20+deleted:false%20+working:true%20+variant:default/orderby/modDate%20desc"

Here is the get.vtl code for the /vtl/api

#set($contentlets = [])
#foreach($con in $dotcontent.pull("+contentType:blog", 5, "moddate"))
    #set($content = {})
    $content.put("pageTitle", $con.pageTitle)
    $content.put("ogTitle", $con.ogTitle)
    $content.put("publishDate", $con.publishDate)
    $content.put("inode", $con.inode)
    $content.put("identifier", $con.identifier)
    $content.put("tags", $con.tags)
    $content.put("folder", $con.folder)
    $content.put("hostName", $con.host.hostname)
    $content.put("urlMap", $con.urlMap)
    $content.put("blogContent", "$con.blogContent.getJson()");
    $content.put("image", $con.image)
    $contentlets.add($content)
#end
$dotJSON.put("contentlets", $contentlets);

[jdotcms]

This is what you can use on js

(async function get(context) {

    const request = context.request;
    const response = context.response;
    const dotlogger = context.dotlogger;

    try {
        const result =  dotcontent.pull('+contentType:blog',5,'modDate desc');

        return response.status(200).json(result);
    } catch (err) {

        if (err instanceof NotFoundError) {
            return response.status(404).text(`Contentlet ${id} not found`);
        }
        return response.status(500).text(`Error on getting contentlet` + err);
    }
})

[jdotcms]

Results

Endpoint       cache?  10,000 requests   Req/Sec
REST API       no    16.843 seconds        593.73 [#/sec]
GraphQL       no    18.014 seconds         555.11 [#/sec]
GraphQL      yes    16.573 seconds        603.39 [#/sec]
api/vtl           no     13.679 seconds        731.07 [#/sec]
api/vtl           yes   13.256 seconds        754.38 [#/sec]
api/js            no     16.562 seconds        603.79 [#/sec]
api/js            yes   16.092 seconds        621.41 [#/sec]

[jdotcms]

Those are the links I have used:

export TOK=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJhcGk1MjU3NmJkZS05MjcyLTQyMzktYWU3YS03ZjRhNjBjMjFmZjAiLCJ4bW9kIjoxNzAyNTc0MjA4MDAwLCJuYmYiOjE3MDI1NzQyMDgsImlzcyI6IjQ1ZGNiNThhMzIiLCJsYWJlbCI6ImFwaSIsImV4cCI6MTc5NzIyODAwMCwiaWF0IjoxNzAyNTc0MjA4LCJqdGkiOiJmYWRkOWM2ZC1jMDc4LTQ0ZTUtOGI3MS01MTk1NmQ4ZDQ2OGYifQ.MI79mY65-MDtzwXBXmdAxuEoj5RjbI_qaCKwjYTBfAU

API:
ab -n 10000 -c 25 -H "Authorization: Bearer $TOK" -k "http://127.0.0.1:8080/api/content/render/false/query/+contentType:Blog%20+(conhost:48190c8c-42c4-46af-8d1a-0cd5db894797%20conhost:SYSTEM_HOST)%20+languageId:1%20+deleted:false%20+working:true%20+variant:default/orderby/modDate%20desc"

GRAPHQL:

ab -n 10000 -c 25  -H "Authorization: Bearer $TOK"  -T' application/json' -p  graphql-data  -k http://localhost:8080/api/v1/graphql

ab -n 10000 -c 25  -H "Authorization: Bearer $TOK"  -T' application/json' -p  graphql-data  -k http://localhost:8080/api/v1/graphql?dotcachettl=0

VTLAPI
ab -n 10000 -c 25 -H "Authorization: Bearer $TOK"  -k "http://localhost:8080/api/vtl/vtlabtest"

JSAPI
ab -n 10000 -c 25 -H "Authorization: Bearer $TOK"  -k "http://localhost:8080/api/js/abtest"

[jdotcms]

Transactinal Summary (Graphql is using so much db in comparison others approaches)

[jdotcms]

When add cache to the graphresults

Document Path:          /api/v1/graphql?dotcachettl=600
Document Length:        67915 bytes

Concurrency Level:      25
Time taken for tests:   1.134 seconds
Complete requests:      10000
Failed requests:        0
Keep-Alive requests:    9910
Total transferred:      684337533 bytes
Total body sent:        9110000
HTML transferred:       679150000 bytes
Requests per second:    8820.77 [#/sec] (mean)
Time per request:       2.834 [ms] (mean)
Time per request:       0.113 [ms] (mean, across all concurrent requests)
Transfer rate:          589490.56 [Kbytes/sec] received
                        7847.38 kb/s sent
                        597337.94 kb/s total

[jdotcms]

if you add this to the vtl, the response will be also cached

$dotJSON.put("dotcache", 60)

The result

Concurrency Level:      25
Time taken for tests:   2.164 seconds
Complete requests:      10000
Failed requests:        0
Keep-Alive requests:    0
Total transferred:      518530000 bytes
HTML transferred:       512770000 bytes
Requests per second:    4621.37 [#/sec] (mean)
Time per request:       5.410 [ms] (mean)
Time per request:       0.216 [ms] (mean, across all concurrent requests)
Transfer rate:          234015.47 [Kbytes/sec] received