spdx / LicenseListPublisher

Tool that generates license data found in the license-list-data repository from the license-list-XML source
Apache License 2.0
11 stars 18 forks source link

Performance improvement - move concurrency in cross-ref URL checking to XmlLicenseProvideerWithCrossRefDetails next method #73

Closed goneall closed 3 years ago

goneall commented 4 years ago

If we move the asynchronous behavior to the XmlLicenseProviderWithCrossRefDetails https://github.com/spdx/LicenseListPublisher/blob/c56fef6c44fe4b8f69c5fc157cc38045cdd6be6b/src/org/spdx/licensexml/XmlLicenseProviderWithCrossRefDetails.java#L58 by making the buildUrlDetails return Future<String[]> and change the implementation of buildUrlDetails to synchronously call the underlying methods, we could launch a number of license URL builders in parallel.

We would essentially launch a maximum number of future computations and on the next() call check to see which one has completed and return the ones that are ready only blocking and waiting if we run out of readily completed checks.

goneall commented 3 years ago

Resolved in PR #82