search

ConduitIO / conduit

Conduit streams data between data stores. Kafka Connect replacement. No JVM required.
https://conduit.io
Apache License 2.0
396 stars 46 forks source link

Performance benchmarks: Better configuration #556

Open hariso opened 2 years ago

hariso commented 2 years ago

Feature description

Currently, we're using bash scripts to configure a Docker container (in which Conduit will be running) and also to create test pipelines. There are two implications of this:

  1. The configuration for a single test is split into two files (run-workload.sh which runs a Docker container, and workload scripts which create pipelines)
  2. Workload scripts are pretty verbose (cURL calls)

Once we're done with pipeline config files, we can easily configure a whole performance test with a single YML file (one part of it would configure the test instance, another would the pipeline YML config).

Depends on:

uchennakevinm1 commented 2 years ago

This issue might be a requirement before we do automated benchmarks. At this point, it is not important.