Unsupported vector mode is passed incorrectly

[Quuxplusone]

Bugzilla Link	PR39501
Status	NEW
Importance	P normal
Reported by	H.J. Lu (hjl.tools@gmail.com)
Reported on	2018-10-30 08:58:38 -0700
Last modified on	2020-07-01 16:18:51 -0700
Version	trunk
Hardware	PC Linux
CC	craig.topper@gmail.com, echristo@gmail.com, erich.keane@intel.com, iains-llvm@btconnect.com, llvm-bugs@lists.llvm.org, rnk@google.com
Fixed by commit(s)
Attachments
Blocks
Blocked by
See also

[hjl@gnu-cfl-1 tmp]$ cat x.c
typedef int __attribute__((mode(QI))) qi;
typedef qi __attribute__((vector_size (32))) v32qi;

v32qi foo (int x)
{
 v32qi y = {'0','1','2','3','4','5','6','7','8','9','a','b','c','d','e','f',
         '0','1','2','3','4','5','6','7','8','9','a','b','c','d','e','f'};
 return y;
}
[hjl@gnu-cfl-1 tmp]$ clang -mno-avx -S -O2 x.c

    movaps  .LCPI0_0(%rip), %xmm0   # xmm0 =
[48,49,50,51,52,53,54,55,56,57,97,98,99,100,101,102]
    movaps  %xmm0, %xmm1
    retq

-mno-avx should pass v32qi as before AVX was enabled.  For example,

[hjl@gnu-cfl-1 tmp]$ cat z.c
typedef int __attribute__((mode(SI))) si;
typedef si __attribute__((vector_size (128))) v;

v
foo (int x)
{
 v y = {'0','1','2','3','4','5','6','7','8','9','a','b','c','d','e','f',
         '0','1','2','3','4','5','6','7','8','9','a','b','c','d','e','f'};
 return y;
}
[hjl@gnu-cfl-1 tmp]$ clang -S -O2 z.c
...
    movaps  .LCPI0_0(%rip), %xmm0   # xmm0 = [99,100,101,102]
    movaps  %xmm0, 112(%rdi)
    movaps  .LCPI0_1(%rip), %xmm1   # xmm1 = [56,57,97,98]
    movaps  %xmm1, 96(%rdi)
    movaps  .LCPI0_2(%rip), %xmm2   # xmm2 = [52,53,54,55]
    movaps  %xmm2, 80(%rdi)
    movaps  .LCPI0_3(%rip), %xmm3   # xmm3 = [48,49,50,51]
    movaps  %xmm3, 64(%rdi)
    movaps  %xmm0, 48(%rdi)
    movaps  %xmm1, 32(%rdi)
    movaps  %xmm2, 16(%rdi)
    movaps  %xmm3, (%rdi)
    movq    %rdi, %rax
    retq

If there were AVX1024, clang would have passed y in vector register. When
AVX1024 were disabled, clang would have passed y in memory and issued an
ABI change warning.

[Quuxplusone]

If I'm understanding what's happening right. The frontend is always sending the type directly into IR as the full vector type. The backend will try to fit that type into 4 of whatever vector register width is available. If it doesn't fit it falls back to memory.

[Quuxplusone]

Candidate patch https://reviews.llvm.org/D53919

[Quuxplusone]

As of 2831a317b689c7f005a29f008a8e4c24485c0711 we should at least warn on ABI mismatches. But we still pass large vectors differently than gcc.