Closed kordless closed 1 year ago
Commit 1e2c7cfd31c6c50296c133caefbd0eda2f0f9195 implements the container running FB.
Most of this is done with cf9cfabef42d5dd0879668ae448a7b22e851acf7. Moving the schema POST functionality requirements to a LLM schemer example which will sample Kafka data for modeling the scheme.
A Docker container is required for running the FeatureBase binary. This container should also run Kafka, Zookeeper, and the Kafka consumer, as seen here: https://github.com/FeatureBaseDB/featurebase-examples/tree/main/kafka-starter
To enable a configurable schema file, create a simple Flask endpoint that takes a JSON file POST, saves it to the container and then restarts the consumer. The consumer should start with a sample schema JSON (as seen in the kafka-starter example) and display the current schema. Here are the endpoints and sample outputs:
/schema [GET] [ { "name": "user_id", "path": ["user_id"], "type": "string" }, { "name": "name", "path": ["name"], "type": "string" }, { "name": "age", "path": ["age"], "type": "id" } ]
/schema [POST] (validates JSON) {"response": "OK"}
Invalid JSON: {"response": "FAILED"}
The port to expose for the Flask endpoint is to be 20202.
A sample Python file that sends its schema to the port and then submits data via the Kafka libs is also needed. A README.md is to be written and posted to the /docker-example directory in this repo.