Open Sawwave opened 1 month ago
rob-p
commented
1 month ago Can you say what version of GCC or clang you are using? Sometimes compilers update and break in an incompatible way (the newer compilers are stricter). We generally test on GCC 11, so there may be compilation problems on GCC 13 we aren't catching.
Sawwave
commented
1 month ago gcc --version
gcc (GCC) 14.2.1 20240910
Copyright (C) 2024 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.hi @rob-p , any idea if this issue will be addressed?
rob-p
commented
4 weeks ago Can you please see if it's addressed on the develop branch, @Sawwave? If so, we can merge into master, but we don't have a copy of a new enough GCC on our development machine to test locally.
Sawwave
commented
3 weeks ago Unfortunately, it seems to still have build errors. I've included the first few pages of the errors when make is run
patching file kmc_core/queues.h
patching file Makefile
[ 8%] No configure step for 'prj_kmc'
[ 10%] Performing build step for 'prj_kmc'
In file included from kmc_core/kff_writer.cpp:1:
kmc_core/kff_writer.h:7:6: error: variable or field ‘StoreBigEndian’ declared void
7 | void StoreBigEndian(uint8_t* buff, const T& data)
| ^~~~~~~~~~~~~~
kmc_core/kff_writer.h:7:21: error: ‘uint8_t’ was not declared in this scope
7 | void StoreBigEndian(uint8_t* buff, const T& data)
| ^~~~~~~
kmc_core/kff_writer.h:5:1: note: ‘uint8_t’ is defined in header ‘<cstdint>’; this is probably fixable by adding ‘#include <cstdint>’
4 | #include <vector>
+++ |+#include <cstdint>
5 |
kmc_core/kff_writer.h:7:30: error: ‘buff’ was not declared in this scope
7 | void StoreBigEndian(uint8_t* buff, const T& data)
| ^~~~
kmc_core/kff_writer.h:7:36: error: expected primary-expression before ‘const’
7 | void StoreBigEndian(uint8_t* buff, const T& data)
| ^~~~~
kmc_core/kff_writer.h:18:26: error: ‘uint8_t’ does not name a type
18 | static constexpr uint8_t VER_MAJOR = 1;
| ^~~~~~~
kmc_core/kff_writer.h:18:26: note: ‘uint8_t’ is defined in header ‘<cstdint>’; this is probably fixable by adding ‘#include <cstdint>’
kmc_core/kff_writer.h:19:26: error: ‘uint8_t’ does not name a type
19 | static constexpr uint8_t VER_MINOR = 0;
| ^~~~~~~
kmc_core/kff_writer.h:19:26: note: ‘uint8_t’ is defined in header ‘<cstdint>’; this is probably fixable by adding ‘#include <cstdint>’
kmc_core/kff_writer.h:22:9: error: ‘uint64_t’ does not name a type
22 | uint64_t k;
| ^~~~~~~~
kmc_core/kff_writer.h:22:9: note: ‘uint64_t’ is defined in header ‘<cstdint>’; this is probably fixable by adding ‘#include <cstdint>’
kmc_core/kff_writer.h:24:9: error: ‘uint64_t’ does not name a type
24 | uint64_t counter_size;
| ^~~~~~~~
kmc_core/kff_writer.h:24:9: note: ‘uint64_t’ is defined in header ‘<cstdint>’; this is probably fixable by adding ‘#include <cstdint>’
kmc_core/kff_writer.h:25:9: error: ‘uint64_t’ does not name a type
25 | uint64_t min_count;
| ^~~~~~~~
kmc_core/kff_writer.h:25:9: note: ‘uint64_t’ is defined in header ‘<cstdint>’; this is probably fixable by adding ‘#include <cstdint>’
kmc_core/kff_writer.h:26:9: error: ‘uint64_t’ does not name a type
26 | uint64_t max_count;
| ^~~~~~~~
kmc_core/kff_writer.h:26:9: note: ‘uint64_t’ is defined in header ‘<cstdint>’; this is probably fixable by adding ‘#include <cstdint>’
kmc_core/kff_writer.h:29:9: error: ‘uint64_t’ does not name a type
29 | uint64_t cur_file_size = 0;
| ^~~~~~~~
kmc_core/kff_writer.h:29:9: note: ‘uint64_t’ is defined in header ‘<cstdint>’; this is probably fixable by adding ‘#include <cstdint>’
kmc_core/kff_writer.h:31:72: error: ‘uint8_t’ was not declared in this scope
31 | void storeIndexPair(const char* type, int64_t val, std::vector<uint8_t>& tmp);
| ^~~~~~~
kmc_core/kff_writer.h:31:72: note: ‘uint8_t’ is defined in header ‘<cstdint>’; this is probably fixable by adding ‘#include <cstdint>’
kmc_core/kff_writer.h:31:79: error: template argument 1 is invalid
31 | void storeIndexPair(const char* type, int64_t val, std::vector<uint8_t>& tmp);
| ^
kmc_core/kff_writer.h:31:79: error: template argument 2 is invalid
kmc_core/kff_writer.h:37:17: error: ‘uint64_t’ does not name a type
37 | uint64_t where_to_store_nb_recs;
| ^~~~~~~~
kmc_core/kff_writer.h:37:17: note: ‘uint64_t’ is defined in header ‘<cstdint>’; this is probably fixable by adding ‘#include <cstdint>’
kmc_core/kff_writer.h:38:17: error: ‘uint64_t’ does not name a type
38 | uint64_t nb_recs;
| ^~~~~~~~
kmc_core/kff_writer.h:38:17: note: ‘uint64_t’ is defined in header ‘<cstdint>’; this is probably fixable by adding ‘#include <cstdint>’
kmc_core/kff_writer.h:43:54: error: ‘uint8_t’ has not been declared
43 | explicit CKFFWriter(const std::string& path, uint8_t canonical, uint64_t k, uint64_t counter_size, uint64_t min_count, uint64_t max_count, uint8_t encoding = 0b00011011);
| ^~~~~~~
kmc_core/kff_writer.h:43:73: error: ‘uint64_t’ has not been declared
43 | explicit CKFFWriter(const std::string& path, uint8_t canonical, uint64_t k, uint64_t counter_size, uint64_t min_count, uint64_t max_count, uint8_t encoding = 0b00011011);
| ^~~~~~~~
kmc_core/kff_writer.h:43:85: error: ‘uint64_t’ has not been declared
43 | explicit CKFFWriter(const std::string& path, uint8_t canonical, uint64_t k, uint64_t counter_size, uint64_t min_count, uint64_t max_count, uint8_t encoding = 0b00011011);
| ^~~~~~~~
kmc_core/kff_writer.h:43:108: error: ‘uint64_t’ has not been declared
43 | CKFFWriter(const std::string& path, uint8_t canonical, uint64_t k, uint64_t counter_size, uint64_t min_count, uint64_t max_count, uint8_t encoding = 0b00011011);
| ^~~~~~~~
kmc_core/kff_writer.h:43:128: error: ‘uint64_t’ has not been declared
43 | ::string& path, uint8_t canonical, uint64_t k, uint64_t counter_size, uint64_t min_count, uint64_t max_count, uint8_t encoding = 0b00011011);
| ^~~~~~~~
kmc_core/kff_writer.h:43:148: error: ‘uint8_t’ has not been declared
43 | 8_t canonical, uint64_t k, uint64_t counter_size, uint64_t min_count, uint64_t max_count, uint8_t encoding = 0b00011011);
| ^~~~~~~
kmc_core/kff_writer.h:45:32: error: ‘uint8_t’ has not been declared
45 | void StoreWholeSection(uint8_t* data, uint64_t n_kmers);
| ^~~~~~~
kmc_core/kff_writer.h:45:47: error: ‘uint64_t’ has not been declared
45 | void StoreWholeSection(uint8_t* data, uint64_t n_kmers);
| ^~~~~~~~
kmc_core/kff_writer.h:48:31: error: ‘uint8_t’ has not been declared
48 | void StoreSectionPart(uint8_t* data, uint64_t n_recs);
| ^~~~~~~
kmc_core/kff_writer.h:48:46: error: ‘uint64_t’ has not been declared
48 | void StoreSectionPart(uint8_t* data, uint64_t n_recs);
| ^~~~~~~~
kmc_core/kff_writer.cpp:5:11: error: ‘uint8_t’ does not name a type
5 | constexpr uint8_t CKFFWriter::VER_MAJOR;
| ^~~~~~~
kmc_core/kff_writer.cpp:4:1: note: ‘uint8_t’ is defined in header ‘<cstdint>’; this is probably fixable by adding ‘#include <cstdint>’
3 | #include <cstring>
+++ |+#include <cstdint>
4 |
kmc_core/kff_writer.cpp:6:11: error: ‘uint8_t’ does not name a type
6 | constexpr uint8_t CKFFWriter::VER_MINOR;
| ^~~~~~~
kmc_core/kff_writer.cpp:6:11: note: ‘uint8_t’ is defined in header ‘<cstdint>’; this is probably fixable by adding ‘#include <cstdint>’
kmc_core/kff_writer.cpp:7:49: error: ‘uint8_t’ has not been declared
7 | CKFFWriter::CKFFWriter(const std::string& path, uint8_t canonical, uint64_t k, uint64_t counter_size, uint64_t min_count, uint64_t max_count, uint8_t encoding) :
| ^~~~~~~
kmc_core/kff_writer.cpp:7:68: error: ‘uint64_t’ has not been declared
7 | CKFFWriter::CKFFWriter(const std::string& path, uint8_t canonical, uint64_t k, uint64_t counter_size, uint64_t min_count, uint64_t max_count, uint8_t encoding) :
| ^~~~~~~~
kmc_core/kff_writer.cpp:7:80: error: ‘uint64_t’ has not been declared
7 | CKFFWriter::CKFFWriter(const std::string& path, uint8_t canonical, uint64_t k, uint64_t counter_size, uint64_t min_count, uint64_t max_count, uint8_t encoding) :
| ^~~~~~~~
kmc_core/kff_writer.cpp:7:103: error: ‘uint64_t’ has not been declared
7 | CKFFWriter(const std::string& path, uint8_t canonical, uint64_t k, uint64_t counter_size, uint64_t min_count, uint64_t max_count, uint8_t encoding) :
| ^~~~~~~~
kmc_core/kff_writer.cpp:7:123: error: ‘uint64_t’ has not been declared
7 | ::string& path, uint8_t canonical, uint64_t k, uint64_t counter_size, uint64_t min_count, uint64_t max_count, uint8_t encoding) :
| ^~~~~~~~
kmc_core/kff_writer.cpp:7:143: error: ‘uint8_t’ has not been declared
7 | 8_t canonical, uint64_t k, uint64_t counter_size, uint64_t min_count, uint64_t max_count, uint8_t encoding) :
| ^~~~~~~
kmc_core/kff_writer.cpp: In constructor ‘CKFFWriter::CKFFWriter(const std::string&, int, int, int, int, int, int)’:
kmc_core/kff_writer.cpp:8:9: error: class ‘CKFFWriter’ does not have any field named ‘k’
8 | k(k), counter_size(counter_size),
| ^
kmc_core/kff_writer.cpp:8:15: error: class ‘CKFFWriter’ does not have any field named ‘counter_size’
8 | k(k), counter_size(counter_size),
| ^~~~~~~~~~~~
kmc_core/kff_writer.cpp:9:9: error: class ‘CKFFWriter’ does not have any field named ‘min_count’
9 | min_count(min_count),
| ^~~~~~~~~
kmc_core/kff_writer.cpp:10:9: error: class ‘CKFFWriter’ does not have any field named ‘max_count’
10 | max_count(max_count)
| ^~~~~~~~~
kmc_core/kff_writer.cpp:16:9: error: ‘cur_file_size’ was not declared in this scope
16 | cur_file_size += 3;
| ^~~~~~~~~~~~~
kmc_core/kff_writer.cpp:18:17: error: ‘VER_MAJOR’ was not declared in this scope
18 | fwrite(&VER_MAJOR, 1, 1, file);
| ^~~~~~~~~
kmc_core/kff_writer.cpp:21:17: error: ‘VER_MINOR’ was not declared in this scope
21 | fwrite(&VER_MINOR, 1, 1, file);
| ^~~~~~~~~
kmc_core/kff_writer.cpp:27:9: error: ‘uint8_t’ was not declared in this scope
27 | uint8_t unique_kmers = 1;
| ^~~~~~~
kmc_core/kff_writer.cpp:27:9: note: ‘uint8_t’ is defined in header ‘<cstdint>’; this is probably fixable by adding ‘#include <cstdint>’
kmc_core/kff_writer.cpp:28:17: error: ‘unique_kmers’ was not declared in this scope
28 | fwrite(&unique_kmers, 1, 1, file);
| ^~~~~~~~~~~~
kmc_core/kff_writer.cpp:34:9: error: ‘uint32_t’ was not declared in this scope
34 | uint32_t free_size = 0;
| ^~~~~~~~
kmc_core/kff_writer.cpp:34:9: note: ‘uint32_t’ is defined in header ‘<cstdint>’; this is probably fixable by adding ‘#include <cstdint>’
kmc_core/kff_writer.cpp:35:28: error: template argument 2 is invalid
35 | std::vector<uint8_t> tmp(sizeof(uint64_t));
| ^
kmc_core/kff_writer.cpp:35:41: error: ‘uint64_t’ was not declared in this scope
35 | std::vector<uint8_t> tmp(sizeof(uint64_t));
| ^~~~~~~~
kmc_core/kff_writer.cpp:35:41: note: ‘uint64_t’ is defined in header ‘<cstdint>’; this is probably fixable by adding ‘#include <cstdint>’
kmc_core/kff_writer.cpp:36:28: error: request for member ‘data’ in ‘tmp’, which is of non-class type ‘int’
36 | StoreBigEndian(tmp.data(), free_size);
| ^~~~We can use the conda package, so this isn't a blocker on our end, but it's likely something you will want to address at some point. Thank you for looking into this.
rob-p
commented
3 weeks ago Thanks for the update. The KMC reliance on undeclared types under the newer compiler seems to be killing us. I’ll try and set up a newer GCC somewhere to test this and see what can be easily patched.
rob-p
commented
2 weeks ago Hi @Sawwave,
Can you try on develop once more? I made a docker image with ubuntu 24.04 and gcc-14 and fixed the patch so that KMC (and cuttlefish) now seem to properly compile.
Thanks! Rob
Sawwave
commented
2 weeks ago Still get a few errors, but it's a lot less this time! Here's all the error I received.
simde-amalgamated-0.7.2/x86/xop.h
simde-amalgamated-0.7.2/arm/
simde-amalgamated-0.7.2/arm/neon.h
In file included from kmc_core/mem_disk_file.cpp:12:
kmc_core/critical_error_handler.h: In member function ‘void CCriticalErrorHandler::HandleCriticalError(const std::string&)’:
kmc_core/critical_error_handler.h:77:28: error: ‘runtime_error’ is not a member of ‘std’
77 | throw std::runtime_error(msg);
| ^~~~~~~~~~~~~
kmc_core/critical_error_handler.h:7:1: note: ‘std::runtime_error’ is defined in header ‘<stdexcept>’; this is probably fixable by adding ‘#include <stdexcept>’
6 | #include <condition_variable>
+++ |+#include <stdexcept>
7 |
In file included from kmc_core/queues.h:25,
from kmc_core/params.h:16,
from kmc_core/bkb_writer.h:14,
from kmc_core/bkb_writer.cpp:11:
kmc_core/critical_error_handler.h: In member function ‘void CCriticalErrorHandler::HandleCriticalError(const std::string&)’:
kmc_core/critical_error_handler.h:77:28: error: ‘runtime_error’ is not a member of ‘std’
77 | throw std::runtime_error(msg);
| ^~~~~~~~~~~~~
kmc_core/critical_error_handler.h:7:1: note: ‘std::runtime_error’ is defined in header ‘<stdexcept>’; this is probably fixable by adding ‘#include <stdexcept>’
6 | #include <condition_variable>
+++ |+#include <stdexcept>
7 |
make[3]: *** [Makefile:118: kmc_core/mem_disk_file.o] Error 1
make[3]: *** Waiting for unfinished jobs....
In file included from kmc_core/queues.h:25,
from kmc_core/params.h:16,
from kmc_core/bkb_reader.h:15,
from kmc_core/bkb_reader.cpp:11:
kmc_core/critical_error_handler.h: In member function ‘void CCriticalErrorHandler::HandleCriticalError(const std::string&)’:
kmc_core/critical_error_handler.h:77:28: error: ‘runtime_error’ is not a member of ‘std’
77 | throw std::runtime_error(msg);
| ^~~~~~~~~~~~~
kmc_core/critical_error_handler.h:7:1: note: ‘std::runtime_error’ is defined in header ‘<stdexcept>’; this is probably fixable by adding ‘#include <stdexcept>’
6 | #include <condition_variable>
+++ |+#include <stdexcept>
7 |
make[3]: *** [Makefile:118: kmc_core/bkb_writer.o] Error 1
make[3]: *** [Makefile:118: kmc_core/bkb_reader.o] Error 1
make[2]: *** [CMakeFiles/prj_kmc.dir/build.make:86: prj_kmc-prefix/src/prj_kmc-stamp/prj_kmc-build] Error 2
make[1]: *** [CMakeFiles/Makefile2:104: CMakeFiles/prj_kmc.dir/all] Error 2
make: *** [Makefile:136: all] Error 2
When compiling cuttlefish from source, the included kmc_core seems to have compilation errors due to an incorrect #include.
Specifically, the following errors are generated.
Is the wrong KMC version being downloaded with curl in the cmake build? Currently, the cmake file specifies the following for KMC: