Compilation error on linux ubuntu

[ParisNeo]

Hi there. I have installed cuda toolkit 11.7 using conda. Also gcc (tested multiple versions). Here with 9.4.0 I get this error:

  -- Build files have been written to: /nobackup/sa226037/627327.0/pip-install-ouvtejw5/ctransformers_b35eb18ba93148019a7fa6999f9eed8d/_skbuild/linux-x86_64-3.10/cmake-build
  [1/6] Building C object CMakeFiles/ctransformers.dir/models/ggml/k_quants.c.o
  FAILED: CMakeFiles/ctransformers.dir/models/ggml/k_quants.c.o
  /home/200.3-PYTHON/commun/envs/lollms/bin/cc -DGGML_CUDA_DMMV_X=32 -DGGML_CUDA_DMMV_Y=1 -DGGML_USE_CUBLAS -DGGML_USE_K_QUANTS -DK_QUANTS_PER_ITERATION=2 -Dctransformers_EXPORTS -I/nobackup/sa226037/627327.0/pip-install-ouvtejw5/ctransformers_b35eb18ba93148019a7fa6999f9eed8d/models -isystem /home/200.3-PYTHON/commun/envs/lollms/include -O3 -DNDEBUG -std=gnu11 -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wdouble-promotion -Wshadow -Wstrict-prototypes -Wpointer-arith -mfma -mavx2 -mf16c -mavx -pthread -MD -MT CMakeFiles/ctransformers.dir/models/ggml/k_quants.c.o -MF CMakeFiles/ctransformers.dir/models/ggml/k_quants.c.o.d -o CMakeFiles/ctransformers.dir/models/ggml/k_quants.c.o -c /nobackup/sa226037/627327.0/pip-install-ouvtejw5/ctransformers_b35eb18ba93148019a7fa6999f9eed8d/models/ggml/k_quants.c
  In file included from /nobackup/sa226037/627327.0/pip-install-ouvtejw5/ctransformers_b35eb18ba93148019a7fa6999f9eed8d/models/ggml/k_quants.c:1:
  /nobackup/sa226037/627327.0/pip-install-ouvtejw5/ctransformers_b35eb18ba93148019a7fa6999f9eed8d/models/ggml/k_quants.h:26:15: error: expected declaration specifiers or '...' before 'sizeof'
     26 | static_assert(sizeof(block_q2_K) == 2*sizeof(ggml_fp16_t) + QK_K/16 + QK_K/4, "wrong q2_K block size/padding");
        |               ^~~~~~
  /nobackup/sa226037/627327.0/pip-install-ouvtejw5/ctransformers_b35eb18ba93148019a7fa6999f9eed8d/models/ggml/k_quants.h:26:79: error: expected declaration specifiers or '...' before string constant
     26 | static_assert(sizeof(block_q2_K) == 2*sizeof(ggml_fp16_t) + QK_K/16 + QK_K/4, "wrong q2_K block size/padding");
        |                                                                               ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  /nobackup/sa226037/627327.0/pip-install-ouvtejw5/ctransformers_b35eb18ba93148019a7fa6999f9eed8d/models/ggml/k_quants.h:38:15: error: expected declaration specifiers or '...' before 'sizeof'
     38 | static_assert(sizeof(block_q3_K) == sizeof(ggml_fp16_t) + QK_K / 4 + 11 * QK_K / 64, "wrong q3_K block size/padding");
        |               ^~~~~~
  /nobackup/sa226037/627327.0/pip-install-ouvtejw5/ctransformers_b35eb18ba93148019a7fa6999f9eed8d/models/ggml/k_quants.h:38:86: error: expected declaration specifiers or '...' before string constant
     38 | static_assert(sizeof(block_q3_K) == sizeof(ggml_fp16_t) + QK_K / 4 + 11 * QK_K / 64, "wrong q3_K block size/padding");
        |                                                                                      ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  /nobackup/sa226037/627327.0/pip-install-ouvtejw5/ctransformers_b35eb18ba93148019a7fa6999f9eed8d/models/ggml/k_quants.h:50:15: error: expected declaration specifiers or '...' before 'sizeof'
     50 | static_assert(sizeof(block_q4_K) == 2*sizeof(ggml_fp16_t) + 3*QK_K/64 + QK_K/2, "wrong q4_K block size/padding");
        |               ^~~~~~
  /nobackup/sa226037/627327.0/pip-install-ouvtejw5/ctransformers_b35eb18ba93148019a7fa6999f9eed8d/models/ggml/k_quants.h:50:81: error: expected declaration specifiers or '...' before string constant
     50 | static_assert(sizeof(block_q4_K) == 2*sizeof(ggml_fp16_t) + 3*QK_K/64 + QK_K/2, "wrong q4_K block size/padding");
        |                                                                                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  /nobackup/sa226037/627327.0/pip-install-ouvtejw5/ctransformers_b35eb18ba93148019a7fa6999f9eed8d/models/ggml/k_quants.h:63:15: error: expected declaration specifiers or '...' before 'sizeof'
     63 | static_assert(sizeof(block_q5_K) == 2*sizeof(ggml_fp16_t) + 3*QK_K/64 + QK_K/2 + QK_K/8, "wrong q5_K block size/padding");
        |               ^~~~~~
  /nobackup/sa226037/627327.0/pip-install-ouvtejw5/ctransformers_b35eb18ba93148019a7fa6999f9eed8d/models/ggml/k_quants.h:63:90: error: expected declaration specifiers or '...' before string constant
     63 | static_assert(sizeof(block_q5_K) == 2*sizeof(ggml_fp16_t) + 3*QK_K/64 + QK_K/2 + QK_K/8, "wrong q5_K block size/padding");
        |                                                                                          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  /nobackup/sa226037/627327.0/pip-install-ouvtejw5/ctransformers_b35eb18ba93148019a7fa6999f9eed8d/models/ggml/k_quants.h:75:15: error: expected declaration specifiers or '...' before 'sizeof'
     75 | static_assert(sizeof(block_q6_K) == sizeof(ggml_fp16_t) + QK_K / 16 + 3*QK_K/4, "wrong q6_K block size/padding");
        |               ^~~~~~
  /nobackup/sa226037/627327.0/pip-install-ouvtejw5/ctransformers_b35eb18ba93148019a7fa6999f9eed8d/models/ggml/k_quants.h:75:81: error: expected declaration specifiers or '...' before string constant
     75 | static_assert(sizeof(block_q6_K) == sizeof(ggml_fp16_t) + QK_K / 16 + 3*QK_K/4, "wrong q6_K block size/padding");
        |                                                                                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  /nobackup/sa226037/627327.0/pip-install-ouvtejw5/ctransformers_b35eb18ba93148019a7fa6999f9eed8d/models/ggml/k_quants.h:83:15: error: expected declaration specifiers or '...' before 'sizeof'
     83 | static_assert(sizeof(block_q8_K) == sizeof(float) + QK_K + QK_K/16*sizeof(int16_t), "wrong q8_K block size/padding");
        |               ^~~~~~
  /nobackup/sa226037/627327.0/pip-install-ouvtejw5/ctransformers_b35eb18ba93148019a7fa6999f9eed8d/models/ggml/k_quants.h:83:85: error: expected declaration specifiers or '...' before string constant
     83 | static_assert(sizeof(block_q8_K) == sizeof(float) + QK_K + QK_K/16*sizeof(int16_t), "wrong q8_K block size/padding");
        |                                                                                     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  [2/6] Building C object CMakeFiles/ctransformers.dir/models/ggml/ggml.c.o
  FAILED: CMakeFiles/ctransformers.dir/models/ggml/ggml.c.o

Is there any prevompiled version I can try to install directly? Or do you have an idea why this fails?

[marella]

Hi, not sure why it fails with conda. Are you getting this error on other versions of CUDA as well? I'm currently traveling and have a slow internet, so will check this later.

[ParisNeo]

Take your time. The thing is this works fine on all my windows PCs but when I need it on my server, it fails and I'm obliged to use another binding or use it in CPU mode which is a big loss of performance.

[ParisNeo]

By the way, Is it possible to have a function that shows the model information (model size in memory, vram occupation etc)? And it also would be cool to have a function that returns the seed that was used for the current message (for example if I set seed=-1 I want to be able to get the actual seed that was used so that I can make the generation repeatable).