Speed up listing available extensions

[kaklakariada]

When listing available extensions, EM queries all files in BucketFS to check if the required files for an extension are present. This was meant as a performance optimization to avoid listing files multiple times.

When there are 20k files (especially large archives like SLC, e.g. Sagemaker Extension) this process takes between 6 to 14 seconds which is not acceptable for end users.

Acceptance Criteria

• Speed up listing of extensions to 1 to 2 seconds even with 20k files and archives

Workaround

Configure EM to use a BucketFSBasePath that contains no large .tar.gz or .zip files but only relevant adapter .jar files.

[kaklakariada]

Total working time: 27h = 3.375d