search

vitrivr / vitrivr-engine

vitrivr's next-generation retrieval engine. It is capable of extracting and retrieving a wider range of multimedia objects such as audio, video, images or 3d models.
https://vitrivr.org
MIT License
6 stars 3 forks source link

parameters in ExporterConfig ignored #16

Closed lucaro closed 1 year ago

lucaro commented 1 year ago

When passing a parameters map in an ExporterConfig, they appear to have no effect. Might be similar for other config types. Also, what is the difference between org.vitrivr.engine.index.config.pipeline.ExporterConfig and org.vitrivr.engine.core.config.ExporterConfig ?!

net-cscience-raphael commented 1 year ago

The incostitence of the two exporterConfig arose from the fact that the thumbnail exporter and the pipeline were developed in parallel. This resulted in a org.vitrivr.engine.core.config.ExporterConfig config that is currently needed in the schema and a org.vitrivr.engine.index.config.pipeline.ExporterConfig config that is needed in the pipeline. The same is the case with the org.vitrivr.engine.core.config.FieldConfig and the org.vitrivr.engine.index.config.pipeline.ExtractorConfig.

The problem with the parameters for addExtractor and addExporter is due to the reason that they are loaded from the schema. The question now is: where do we want to parameterize the exporters and extractors? (A) When loading into the schema (parameter-config in config.json) or (B) when creating the pipeline (parameter-config in config-pipeline.json). In my opinion, this should happen with A. When we have clarified this question, I will implement this.

lucaro commented 1 year ago

I see, thank you for the clarification. In general, I think we should use the same configuration classes consistently. The question of priority of parameters is indeed a good one, and one we will also need to address on the retrieval side, since there, such parameters can even come from the outside. Ideally, we would define a merging mechanism that is applicable for both cases.

net-cscience-raphael commented 1 year ago

see 9d155bf971b4130c144f26e574fbf05c917414c7