Create Kafka Topic input/output format

QMSTR / qmstr

QMSTR compliance tool

https://qmstr.org

31 stars 1 forks source link

Create Kafka Topic input/output format #534

Closed fullsushidev closed 3 years ago

fullsushidev commented 3 years ago

In order to save the information collected by QMSTR into FASTEN's metadata database, we need to create our Kafka topic input/output and document it into FASTEN's wiki following their guideline.

https://github.com/fasten-project/fasten/wiki/Kafka-Topics

marcomicera commented 3 years ago

One Kafka topic for each “metadata” field in the KB schema. In bold, the ones that we’re probably gonna generate:

Package
Dependencies
Package versions
Modules
Binary modules
Edges
Callables
Files

marcomicera commented 3 years ago

We need to query both InfoNodes and DataNodes from DGraph.

fullsushidev commented 3 years ago

Done with https://github.com/endocode/qmstr/commit/fc07e81fc95b34c7f6c7f8e2a25046a3509ce614 and https://github.com/endocode/qmstr/commit/2c1682f4abec9d9dce8d237cf3d2019e8dedd783 at https://github.com/endocode/qmstr/tree/fasten_reporter