Egeria use only one CPU core for queries

MaximVetlugin commented 2 years ago

Hi, Please, help me to understand where is a problem with performance. Thank you!

I send this request to Egeria

{{baseURL}}/servers/{{server}}/open-metadata/repository-services/users/admin/enterprise/instances/entities/from-entity/{guid}/by-neighborhood?level=2

body { "class" : "EntityNeighborhoodFindRequest", "limitResultsByStatus" : [ "ACTIVE" ], "relationshipTypeGUIDs" : [ "f00df00d-0000-0000-0000-000000000000", "babebabe-0000-0000-0000-000000000000", "e222e222-0000-0000-0000-000000000000" ] }

(we are using our Entity types)

Entity example in responce

        "class": "EntityDetail",
        "headerVersion": 1,
        "type": {
            "class": "InstanceType",
            "headerVersion": 0,
            "typeDefCategory": "ENTITY_DEF",
            "typeDefGUID": "00ddba11-0000-0000-0000-000000000000",
            "typeDefName": "BusinessTerm",
            "typeDefVersion": 1,
            "typeDefDescription": "Бизнес-термин (WIP)",
            "typeDefSuperTypes": [
                {
                    "headerVersion": 1,
                    "guid": "896d14c2-7522-4f6c-8519-757711943fe6",
                    "name": "Asset",
                    "status": "ACTIVE_TYPEDEF"
                },
                {
                    "headerVersion": 1,
                    "guid": "a32316b8-dc8c-48c5-b12b-71c1b2a080bf",
                    "name": "Referenceable",
                    "status": "ACTIVE_TYPEDEF"
                },
                {
                    "headerVersion": 1,
                    "guid": "4e7761e8-3969-4627-8f40-bfe3cde85a1d",
                    "name": "OpenMetadataRoot",
                    "status": "ACTIVE_TYPEDEF"
                }
            ],
            "validStatusList": [
                "ACTIVE",
                "STANDBY",
                "DELETED"
            ],
            "validInstanceProperties": [
                "qualifiedName",
                "additionalProperties",
                "owner",
                "ownerType",
                "latestChange",
                "zoneMembership",
                "name",
                "description",
                "startDateUse",
                "regulation",
                "regulationDocument",
                "normativeDocument",
                "businessProcessCode",
                "note",
                "algorithmCalculationTemplate"
            ]
        },
        "instanceProvenanceType": "LOCAL_COHORT",
        "metadataCollectionId": "69a6e501-d2d0-49cc-8d9e-8aaafc7fc3a8",
        "metadataCollectionName": "paukd-platform-dev",
        "createdBy": "admin",
        "createTime": "2022-06-27T05:45:10.904+00:00",
        "version": 1,
        "guid": "68f5b2d1-9295-4bcf-8d9c-684720e3fe14",
        "status": "ACTIVE"
    },

It takes from 30 sec to 6 min.

And every time only one cpu core uses!

Local repository config

{ "class": "Connection", "connectorType": { "class": "ConnectorType", "connectorProviderClassName": "org.odpi.egeria.connectors.juxt.xtdb.repositoryconnector.XtdbOMRSRepositoryConnectorProvider" }, "configurationProperties": { "xtdbConfigEDN": "{:xtdb.rocksdb/block-cache {:xtdb/module xtdb.rocksdb/->lru-block-cache :cache-size 1073741824}:xtdb/index-store {:kv-store {:xtdb/module xtdb.rocksdb/->kv-store :block-cache :xtdb.rocksdb/block-cache :db-dir \"data/servers/{{server}}/rdb-index\"}} :kafka-config {:xtdb/module xtdb.kafka/->kafka-config :bootstrap-servers \"{{kafka-cluster}}\"} :xtdb/tx-log {:xtdb/module xtdb.kafka/->tx-log :kafka-config :kafka-config :tx-topic-opts {:topic-name \"{{tx-topic-name}}\" :replication-factor {{tx-topic-replication-factor}}} :poll-wait-duration \"PT0.050S\"} :xtdb.lucene/lucene-store {:db-dir \"data/servers/{{server}}/lucene\" :indexer {:xtdb/module xtdb.lucene.egeria/->egeria-indexer} :analyzer {:xtdb/module xtdb.lucene.egeria/->ci-analyzer}} :xtdb/document-store {:xtdb/module xtdb.jdbc/->document-store :connection-pool {:dialect {:xtdb/module xtdb.jdbc.psql/->dialect} :pool-opts {:maximumPoolSize 10} :db-spec {:jdbcUrl \"jdbc:postgresql://{{PostgreSQL}}/{{PostgreSQLDBName}}?user={{PostgreSQLLogin}}&password={{PostgreSQLPassword}}&ApplicationName={{PostgreSQLApplicationName}}\"}}:poll-sleep-duration \"PT1S\"}}", "luceneRegexes": true, "syncIndex": true } }

JVM parameters /bin/java -javaagent:/opt/jmx_exporter/jmx_prometheus_javaagent-0.17.0.jar=9101:/opt/jmx_exporter/config.yaml -XX:MaxDirectMemorySize=1024m -Xms32g -Xmx32g -Dloader.path=/opt/platform/lib -jar /opt/platform/server-chassis-spring.jar

planetf1 commented 2 years ago

Parallelism :

I believe currently these requests are single threaded in Egeria itself. If the query is across multiple repos an enterprise connector could implement this in parallel - but our current one does not. In terms of xtdb itself, behaviour will depend on how that itself is configured .

Overall Performance:

How much data are you retrieving? Can you use a more tightly scoped call? Would indexing help (there is work underway to add an index - but for searches vs explicit gets)? Beyond that I think capturing profile information would help to understand any hotspots

refset commented 2 years ago

Just chiming in to mention that XTDB happily supports independent queries running with multiple threads & cores, but within the scope of a single query execution is blocking and single-threaded.

Multiple threads can coordinate and execute related queries in parallel using a stable (consistent) tx-id basis, so complex parallel designs can be implemented on top, but note that a db should be created per thread.

odpi / egeria-connector-xtdb

Egeria use only one CPU core for queries #380