hypertrace / hypertrace-service

Multiple hypertrace services combined together to form a single service.
Other
4 stars 15 forks source link

feat: Hypertrace Data Query Service & Hypertrace Data Config Service #63

Closed findingrish closed 3 years ago

findingrish commented 3 years ago

Description

Idea is to combine some of the similar nature query services together, which can then be deployed as a single piece. Currently we have following query layer services:

The fact that these services are stateless makes it easy to combine them.

Broadly we can have 2 services hypertrace-data-config-service: [config, attribute, entity] hypertrace-data-query-service: [graphql, gateway, query, entity]

In this pr, to start with I have created the top level service hypertrace-data-query-service which runs gateway-service & query-service together In a subsequent pr, I will create another top level service called hypertrace-data-config-service Unlike https://github.com/hypertrace/hypertrace-service/pull/61, I have chosen to create a top level service to keep the configs separate and simple. Adding multiple charts and configs in the same module was making things difficult to understand.

Open questions:

  1. Can graphql service be included in hypertrace-data-query-service ?
  2. Since entity-service is used by ingestion path, keeping it in hypertrace-data-query-service doesn't seem like a good option. It can be kept alone or combined with hypertrace-data-config-service ?

Testing

Add another service which combines entity, attribute and config service

Tested the ht setup locally with hotrod app (using https://github.com/hypertrace/hypertrace/compare/hypertrace-data-config-service)

all-config-bootstrapper-fnwzt                     0/1     Completed   0          77s
hypertrace-data-config-service-6cdf849c8b-b9trl   1/1     Running     0          78s
hypertrace-data-query-service-78b78f7775-x5ls8    1/1     Running     0          77s
all-views-creation-job-8mvbl                      0/1     Completed   0          94s
all-views-generator-f6c98ddfc-c25zt               1/1     Running     0          78s
hypertrace-graphql-service-8cfbdf85-7wdc9         1/1     Running     0          78s
hypertrace-kafka-topics-creator-7dh5d             0/1     Completed   0          2m41s
hypertrace-oc-collector-685c846b99-7hnfn          1/1     Running     0          77s
hypertrace-trace-enricher-7bd7786cf-xxhdl         1/1     Running     0          78s
hypertrace-ui-cdbc96b65-mn76v                     1/1     Running     0          78s
kafka-0                                           1/1     Running     0          34m
mongo-0                                           1/1     Running     0          34m
pinot-broker-0                                    1/1     Running     2          34m
pinot-controller-0                                1/1     Running     0          34m
pinot-minion-0                                    1/1     Running     2          34m
pinot-server-0                                    1/1     Running     2          34m
raw-spans-grouper-0                               1/1     Running     0          77s
schema-registry-567c8467fb-gxpl9                  1/1     Running     0          34m
span-normalizer-58c867d84c-pf5wt                  1/1     Running     0          77s
view-generation-kafka-topics-creator-ck9hv        0/5     Completed   0          105s

Issue: https://github.com/hypertrace/hypertrace/issues/166