SIGILL, Illegal instruction on Jetson

[ilya-lavrenov]

Thread 1 "application" received signal SIGILL, Illegal instruction.
Xbyak_aarch64::util::CpuInfoLinux::setSysRegVal (this=0x5559246900) at /home/user/application/aarch64/xbyak_aarch64/src/util_impl_linux.h:424
424       void setSysRegVal() { XBYAK_AARCH64_READ_SYSREG(cacheInfo_.midr_el1, MIDR_EL1); }
(gdb) bt
#0  Xbyak_aarch64::util::CpuInfoLinux::setSysRegVal (this=0x5559246900) at /home/user/application/aarch64/xbyak_aarch64/src/util_impl_linux.h:424
#1  0x0000005556e99084 in Xbyak_aarch64::util::CpuInfoLinux::CpuInfoLinux (this=0x5559246900) at /home/user/application/aarch64/xbyak_aarch64/src/util_impl_linux.h:59
#2  0x0000005556e99fcc in Xbyak_aarch64::util::Cpu::Cpu (this=0x55590c6f40 <dnnl::impl::cpu::aarch64::cpu()::cpu_>)

cat /proc/cpuinfo:

$ cat /proc/cpuinfo
processor       : 0
model name      : ARMv8 Processor rev 1 (v8l)
BogoMIPS        : 38.40
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x41
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0xd07
CPU revision    : 1

processor       : 1
model name      : ARMv8 Processor rev 1 (v8l)
BogoMIPS        : 38.40
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x41
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0xd07
CPU revision    : 1

processor       : 2
model name      : ARMv8 Processor rev 1 (v8l)
BogoMIPS        : 38.40
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x41
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0xd07
CPU revision    : 1

processor       : 3
model name      : ARMv8 Processor rev 1 (v8l)
BogoMIPS        : 38.40
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x41
CPU architecture: 8
CPU variant     : 0x1
CPU part        : 0xd07
CPU revision    : 1

I see that some other projects use guard before accessing this registers (code https://github.com/flame/blis/blob/915daaa43cd189c86d93d72cd249714f126e9425/frame/base/bli_cpuid.c#L1031C7-L1052):

#ifdef __linux__
    if ( getauxval( AT_HWCAP ) & HWCAP_CPUID )
    {
        // Also available from
        // /sys/devices/system/cpu/cpu0/regs/identification/midr_el1
        // and split out in /proc/cpuinfo (with a tab before the colon):
        // CPU part : 0x0a1

        uint64_t midr_el1;
        __asm("mrs %0, MIDR_EL1" : "=r" (midr_el1));
        /*
         * MIDR_EL1
         *
         * 31          24 23     20 19          16 15          4 3         0
         * -----------------------------------------------------------------
         * | Implementer | Variant | Architecture | Part Number | Revision |
         * -----------------------------------------------------------------
         */
        implementer = (midr_el1 >> 24) & 0xFF;
        part        = (midr_el1 >> 4)  & 0xFFF;
    }

    has_sve = getauxval( AT_HWCAP ) & HWCAP_SVE;
    if (has_sve)
        *features |= FEATURE_SVE;
#endif //__linux__

Should the same be in xbyak arch64?

ARM Compute Library also has such a guard https://github.com/ARM-software/ComputeLibrary/blob/2b2ffe758dfff7255cf459a7eab26cb8aeff3061/src/common/cpuinfo/CpuInfo.cpp#L310-L323 and first tries to use registers, then tries to parse info from /proc/cpu files

[kawakami-k]

@ilya-lavrenov Thank you for the bug report. Could you tell me what Linux distribution, OS version and H/W (NVIDIA Jetson?) you use?

[ilya-lavrenov]

$ uname -a
Linux jetson-01 4.9.253-tegra #1 SMP PREEMPT Sat Feb 19 08:59:22 PST 2022 aarch64 aarch64 aarch64 GNU/Linux

$ cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=18.04
DISTRIB_CODENAME=bionic
DISTRIB_DESCRIPTION="Ubuntu 18.04.6 LTS"

Output of what commands can help as well?

[kawakami-k]

$ uname -a
Linux jetson-01 4.9.253-tegra #1 SMP PREEMPT Sat Feb 19 08:59:22 PST 2022 aarch64 aarch64 aarch64 GNU/Linux

$ cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=18.04
DISTRIB_CODENAME=bionic
DISTRIB_DESCRIPTION="Ubuntu 18.04.6 LTS"

Output of what commands can help as well?

For now, the above information is enough. Thank you.

[ilya-lavrenov]

One more link with comments https://go.dev/src/internal/cpu/cpu_arm64_hwcap.go what we cannot use that register on older Linux versions, because access from user space is denied.

[kawakami-k]

I reproduced the same error of sample/cpuinfo.exe on Jetson.

[kawakami-k]

I found the same topic was discussed at https://github.com/xianyi/OpenBLAS/issues/2715 I think it's good to introduce ARM Compute Library's method, /sys/devices/system/cpu/cpu[n]/regs/identification/midr_el1 first, /proc/cpuinfo second. It didn't access system register which may be unaccecible from user programs. It can handle heterogeneous cores, where each core may have different MIDR_EL1 value, cache size and H/W capabilities.

[ilya-lavrenov]

Hi @kawakami-k Kindly asking, do you plan to work on the fix?

[kawakami-k]

@ilya-lavrenov I'll fix the bug, but it may after my summer vacation :-)

[ilya-lavrenov]

@kawakami-k Hope you had a great summer holidays :) Have you been able to start working on this issue?

[kawakami-k]

@ilya-lavrenov

Could you please try this branch. If it's suitable for you, I'll merge it to main branch. https://github.com/fujitsu/xbyak_aarch64/tree/capability

[ilya-lavrenov]

@kawakami-k

Thanks for the fix, SIGILL is eliminated.