The lazy ingestion of packages is not only quite slow. Looking at the implementation, I found the current exception handling to be unnecessarily complex and the business logic for the ingestion to be very convoluted. We currently also download POM files on every request, which is quite an unnecessary load that we put on Maven Central.
This PR addresses these two points and consists of two main changes:
I have revised the exception handling and by doing so simplified the API signatures for all related endpoints
I have improved the ingestion logic. We first check now for already ingested packages, which is the fastest check we can do. Only for un-ingested packages, we will now contact Maven Central to verify their existence. However, instead of downloading the POM file via GET, we now only send a HEAD request and check for the existence. Validated, non-ingested artifacts will then be added to the Kafka input topic.
As a result, the answer to previously ingested packages should be much faster now and most requests should consume fewer resources, both for us and Maven Central.
The lazy ingestion of packages is not only quite slow. Looking at the implementation, I found the current exception handling to be unnecessarily complex and the business logic for the ingestion to be very convoluted. We currently also download POM files on every request, which is quite an unnecessary load that we put on Maven Central.
This PR addresses these two points and consists of two main changes:
As a result, the answer to previously ingested packages should be much faster now and most requests should consume fewer resources, both for us and Maven Central.