Revisit support for Spack target architectures, for best fit with Wave requirements

[marcodelapierre]

We want to make sure we are able to build Wave Containers with target micro-architectures that are relevant/useful to the service.

Current status:

• Nextflow is capable of feeding fine grained CPU archs, through spackArch when using the target subdirective, see L65 in https://github.com/nextflow-io/nextflow/blob/master/modules/nextflow/src/test/groovy/nextflow/processor/ArchitectureTest.groovy#L63-L65
• The Wave Client in the Nextflow Wave Plugin indeed uses spackArch, see https://github.com/nextflow-io/nextflow/blob/ec156bc68232e48ba2462ebbfcf2b15af975c8e2/plugins/nf-wave/src/main/io/seqera/wave/plugin/WaveClient.groovy#L385
• The current default in the Wave Client is currently x86_64, see https://github.com/nextflow-io/nextflow/blob/ec156bc68232e48ba2462ebbfcf2b15af975c8e2/plugins/nf-wave/src/main/io/seqera/wave/plugin/WaveClient.groovy#L85
• [UPDATED] The client hard codes submitting the dockerArch as the platform to Wave, therefore losing any information of the Spack target, see https://github.com/nextflow-io/nextflow/blob/ec156bc68232e48ba2462ebbfcf2b15af975c8e2/plugins/nf-wave/src/main/io/seqera/wave/plugin/WaveClient.groovy#L490-L506

TO-DO:

• Nextflow side: change default to x86_64_v3
• [UPDATED] Nextflow side: diversify between container platform required for container builder and target for Spack commands in containerfile (coordinated with Wave backend)
• Wave backend side: see https://github.com/seqeralabs/wave/issues/336

For reference, list of Spack supported targets as of today: https://spack.readthedocs.io/en/latest/basic_usage.html#support-for-specific-microarchitectures

[pditommaso]

What's the difference of x86_64 vs x86_64_v3?

[marcodelapierre]

x86_64_v3 is a CPU family that corresponds to newer CPUs; it features a number of additional features, such as AVX2 (256 bit floating point registers for vector arithmetics). Compiling for it results in more performant binaries than for x86_64. Pretty much all CPUs marketed after 2015 fall in this family, so there is very, very little risk of incompatibility.

[pditommaso]

OK, but still there's the need to support the target microarchitecture then

[marcodelapierre]

Yes, in Spack jargon: name: 'linux/x86_64', target: 'x86_64_v3'

I have updated the original post above - indeed at first I had missed the point you have just made

[pditommaso]

I see. Main problem there's no support for target arch in the current request object

https://github.com/nextflow-io/nextflow/blob/d2dd216fd0c1ff0ed25863822dff0abc09501fed/plugins/nf-wave/src/main/io/seqera/wave/plugin/SubmitContainerTokenRequest.groovy#L34-L122

[marcodelapierre]

exactly, we will need to add it