Archive comprehension for input normalization

[lptr]

Passing inputs to a task as an archive of sorts (such as a ZIP), or even a list of archives, is a frequently used pattern. Besides the content of files, archives typically include metadata about the files as well, such as timestamps and access permissions. Typically, a task will not be interested in this metadata, nor will the order of files inside the archive influence the output. It is indeed desired that Gradle ignores such metadata and file order when checking for up-to-date-ness and calculating cache keys.

Gradle currently supports doing this for @Classpath and @CompileClasspath inputs. These annotations are intended to be used with actual JVM classpaths, as they carry additional meaning. There also trigger additional behavior: unlike @InputFiles, these annotations take the order of the files on the classpath itself into account.

Using these annotations to ignore archive metadata and file order inside files on non-JVM-classpath inputs can thus be problematic.

We could instead add a feature for non-JVM-classpath archive inputs. We could use an annotation like @Archives similar to how @Classpath works today.

Note that normalization for @Classpath inputs can be further filtered via Project.getNormalization().runtimeClasspath { /* ... */ }. We could also implement similar filtering for @Archives (though if we do so, it would probably make sense to extend the same filtering to @InputFiles and @InputDirectory inputs as well).

With @Classpath we only support ZIP files (such as JARs). For @Archives it might be useful to add support for more archive formats, like TAR, TAR + Gzip, TAR + Bzip2 etc.

It might be necessary to configure what should happen if an un-recognized archive is detected in an @Archives input, though simply failing the build is probably a good first implementation.

---

cc: @gradle/execution

[ducrohet]

While it's a step in the right direction, I think having the ability to do per-task filtering would be useful to something that's meant to handle any random archives. I think something like Project.getNormalization().archives {} is too generic.

But I like the general direction, thanks!

[lptr]

I agree that Project.normalization.archives {} is too generic, and we need something more fine-grained. If we do that, I think we'll also want to do it for runtime classpaths.