Build failure on Ubuntu 24.10

[NeQuissimus]

Building this encounters the following:

[...]

#########################################
Compiling LLM runtime for Linux...
#########################################

-- The C compiler identification is GNU 14.2.0
-- The CXX compiler identification is GNU 14.2.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/gcc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/g++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Configuring done (1.4s)
-- Generating done (0.0s)
-- Build files have been written to: /home/ubuntu/ezrknpu/ezrknpu/ezrknn-llm/rkllm-runtime/examples/rkllm_api_demo/build/build_linux_aarch64_Release
[ 25%] Building CXX object CMakeFiles/llm_demo.dir/src/llm_demo.cpp.o
[ 50%] Building CXX object CMakeFiles/multimodel_demo.dir/src/multimodel_demo.cpp.o
In file included from /home/ubuntu/ezrknpu/ezrknpu/ezrknn-llm/rkllm-runtime/examples/rkllm_api_demo/src/llm_demo.cpp:18:
/home/ubuntu/ezrknpu/ezrknpu/ezrknn-llm/rkllm-runtime/examples/rkllm_api_demo/../../runtime/Linux/librkllm_api/include/rkllm.h:52:5: error: ‘uint8_t’ does not name a type
   52 |     uint8_t      reserved[112];    /**< reserved */
      |     ^~~~~~~
/home/ubuntu/ezrknpu/ezrknpu/ezrknn-llm/rkllm-runtime/examples/rkllm_api_demo/../../runtime/Linux/librkllm_api/include/rkllm.h:1:1: note: ‘uint8_t’ is defined in header ‘<cstdint>’; this is probably fixable by adding ‘#include <cstdint>’
  +++ |+#include <cstdint>
    1 | #ifndef _RKLLM_H_
In file included from /home/ubuntu/ezrknpu/ezrknpu/ezrknn-llm/rkllm-runtime/examples/rkllm_api_demo/src/multimodel_demo.cpp:18:
/home/ubuntu/ezrknpu/ezrknpu/ezrknn-llm/rkllm-runtime/examples/rkllm_api_demo/../../runtime/Linux/librkllm_api/include/rkllm.h:52:5: error: ‘uint8_t’ does not name a type
   52 |     uint8_t      reserved[112];    /**< reserved */
      |     ^~~~~~~
/home/ubuntu/ezrknpu/ezrknpu/ezrknn-llm/rkllm-runtime/examples/rkllm_api_demo/../../runtime/Linux/librkllm_api/include/rkllm.h:1:1: note: ‘uint8_t’ is defined in header ‘<cstdint>’; this is probably fixable by adding ‘#include <cstdint>’
  +++ |+#include <cstdint>
    1 | #ifndef _RKLLM_H_
make[2]: *** [CMakeFiles/llm_demo.dir/build.make:76: CMakeFiles/llm_demo.dir/src/llm_demo.cpp.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:85: CMakeFiles/llm_demo.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
make[2]: *** [CMakeFiles/multimodel_demo.dir/build.make:76: CMakeFiles/multimodel_demo.dir/src/multimodel_demo.cpp.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:111: CMakeFiles/multimodel_demo.dir/all] Error 2
make: *** [Makefile:91: all] Error 2

[...]

The following fixes the issue but I am not sure if this needs to be conditional for Armbian or Ubuntu < 24.10

diff --git a/rkllm-runtime/runtime/Linux/librkllm_api/include/rkllm.h b/rkllm-runtime/runtime/Linux/librkllm_api/include/rkllm.h
index e565e6c..90a250b 100644
--- a/rkllm-runtime/runtime/Linux/librkllm_api/include/rkllm.h
+++ b/rkllm-runtime/runtime/Linux/librkllm_api/include/rkllm.h
@@ -5,6 +5,8 @@
 extern "C" {
 #endif

+#include <cstdint>
+
 /**
  * @typedef LLMHandle
  * @brief A handle used to manage and interact with the large language model.

[MartynaKowalska]

I confirm both the build failure and the fix proposed by @NeQuissimus, on Ubuntu Rockchip 24.04 LTS for rk3588. With the fix it now installs, however when giving a prompt to a model it's still not working as it should, possibly because I'm using Python 3.12 instead of 3.10 or 3.8, which is on me.

Edit: haven't checked entirely how everything works, but using a qwen model converted with the latest toolkit works, while using the qwens from this repo's page fails altogether. In case anyone needs it, it's the qwen.rkllm from here, it's now talking to itself so I guess it's not an instruct model and I'm abusing it, but it's the most recent one, supposedly the only one publicly available converted with the latest toolkit. I'm still using Python 3.12. The command now requires to specify max new tokens and max context length:

rkllm model_path max_new_tokens max_context_len

I'm a complete noob here so there might be mistakes, but hopefully it helps someone else. A big shoutout to the repo's maintainer Pelochus, and to NeQuissimus for the fix!