gpt2tc wrong cpu flags - Githubissues

Isomorph70 commented 1 year ago

Describe the bug

gpt2tc fails and report wrong/missing cpu flags.

Steps To Reproduce

Steps to reproduce the behavior:

$ gpt2tc
GPT2TC version 2021-04-24, Copyright (c) 2019-2021 Fabrice Bellard
Text Completion with GPT-2
usage: gpt2tc [options] g text            text completion
                        c infile outfile  text compression
                        d infile outfile  text decompression
                        cs text           short text compression
                        ds text           short text decompression

General options:
-h                                 show this help
-v                                 show statistics
-T n                               number of threads (default=1)
-m [117M|345M|774M|1558M]          select the model (default=117M)
-f filename                        force the model filename
-k k                               top k value (default=40)
-p p                               top p value (default=0.90)
-t temp                            temperature (default=1.0)
-l n                               set the maximum output length
-s n                               set the seed (0=random, default=0)

$  gpt2tc g 'Sem'
Your CPU does not support AVX2+FMA - exiting

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ this error

Expected behavior

Something else than the error 'Your CPU does not support AVX2+FMA - exiting'

Additional context

My cpu info might be useful.

$ lscpu 
Architecture:            x86_64
  CPU op-mode(s):        32-bit, 64-bit
  Address sizes:         36 bits physical, 48 bits virtual
  Byte Order:            Little Endian
CPU(s):                  8
  On-line CPU(s) list:   0-7
Vendor ID:               GenuineIntel
  Model name:            Intel(R) Core(TM) i7-3770 CPU @ 3.40GHz
    CPU family:          6
    Model:               58
    Thread(s) per core:  2
    Core(s) per socket:  4
    Socket(s):           1
    Stepping:            9
    CPU(s) scaling MHz:  87%
    CPU max MHz:         3400,0000
    CPU min MHz:         1600,0000
    BogoMIPS:            6784,67
    Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant
                         _tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr p
                         dcm pcid sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer xsave avx f16c rdrand lahf_lm cpuid_fault epb pti ssbd ibrs ibpb stibp tpr_shadow vnmi flexpr
                         iority ept vpid fsgsbase smep erms xsaveopt dtherm arat pln pts md_clear flush_l1d
Virtualization features: 
  Virtualization:        VT-x
Caches (sum of all):     
  L1d:                   128 KiB (4 instances)
  L1i:                   128 KiB (4 instances)
  L2:                    1 MiB (4 instances)
  L3:                    8 MiB (1 instance)
NUMA:                    
  NUMA node(s):          1
  NUMA node0 CPU(s):     0-7
Vulnerabilities:         
  Itlb multihit:         KVM: Mitigation: VMX disabled
  L1tf:                  Mitigation; PTE Inversion; VMX conditional cache flushes, SMT vulnerable
  Mds:                   Mitigation; Clear CPU buffers; SMT vulnerable
  Meltdown:              Mitigation; PTI
  Mmio stale data:       Unknown: No mitigations
  Retbleed:              Not affected
  Spec store bypass:     Mitigation; Speculative Store Bypass disabled via prctl
  Spectre v1:            Mitigation; usercopy/swapgs barriers and __user pointer sanitization
  Spectre v2:            Mitigation; Retpolines, IBPB conditional, IBRS_FW, STIBP conditional, RSB filling, PBRSB-eIBRS Not affected
  Srbds:                 Vulnerable: No microcode
  Tsx async abort:       Not affected

Notify maintainers

@anna328p

Metadata

Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.

[user@system:~]$ nix-shell -p nix-info --run "nix-info -m"
 - system: `"x86_64-linux"`
 - host os: `Linux 6.1.32, NixOS, 23.11 (Tapir), 23.11pre493410.21951114383`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.15.1`
 - channels(root): `"nixos"`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos`

anna328p commented 1 year ago

gpt2tc package currently uses compiled binaries from Fabrice Bellard; I could rewrite the package to compile from the open-source release of its code, but it still relies on a closed-source library called libnc that uses vector CPU extensions for tensor operations, so I don't know how much that would help.

I'm pretty sure you'd still need a compatible CPU.

Isomorph70 commented 1 year ago

I fell like there should be some kind of block or a warning, so you don't end up with useless software your computer.

It also might able to set it up, so different versions is used, if the flags is not supported. Like GPT4ALL did https://discourse.nixos.org/t/gpt4all-nix-derivation/27744/8