Possible implementation for #57
Can do:
Get Context from task message from RabbitMQ: gets data, merges models based on properties fields, add context to logger (elastic logs)
Get Data from Object Storage: gets data from local storage (elastic, minio) fallback can be set to opdm (options to run the validation or model retriever pipeline)
Publish Data to Object Storage: publishes data to local storage, file tree (prefix) is composed from the filename. Metadata can be published to elastic
Cannot do:
Publish Data to OPDM: publishes the data to opdm but from the opdm side, validation still fails
Possible implementation for #57 Can do: Get Context from task message from RabbitMQ: gets data, merges models based on properties fields, add context to logger (elastic logs) Get Data from Object Storage: gets data from local storage (elastic, minio) fallback can be set to opdm (options to run the validation or model retriever pipeline) Publish Data to Object Storage: publishes data to local storage, file tree (prefix) is composed from the filename. Metadata can be published to elastic Cannot do: Publish Data to OPDM: publishes the data to opdm but from the opdm side, validation still fails