This PR proposes to change the ingestion table to only contain a generic key that can be checked for existence. This allows for a more generic handling of the duplicate detection and can be used as the respective plugin sees fit. For example, Java's PomAnalyzer will use keys like gid:aid:packaging:version-lane, but C/Python might want to use different keys, e.g., if they do not use the lane concept. We could also think about using this key for additional duplication checks later in the pipeline, e.g., by making the pipeline stage part of it.
This PR proposes to change the ingestion table to only contain a generic
keythat can be checked for existence. This allows for a more generic handling of the duplicate detection and can be used as the respective plugin sees fit. For example, Java's PomAnalyzer will use keys likegid:aid:packaging:version-lane, but C/Python might want to use different keys, e.g., if they do not use the lane concept. We could also think about using this key for additional duplication checks later in the pipeline, e.g., by making the pipeline stage part of it.The adopted schema file can be found in the docker-compose repository.