search

ad-freiburg / qlever

Very fast SPARQL Engine, which can handle very large knowledge graphs like the complete Wikidata, offers context-sensitive autocompletion for SPARQL queries, and allows combination with text search. It's faster than engines like Blazegraph or Virtuoso, especially for queries involving large result sets.
Apache License 2.0
370 stars 43 forks source link

Build failing by docker #750

Closed shakti-rajput closed 2 years ago

shakti-rajput commented 2 years ago

I tried setting up qlever using simple steps to build using docker as given in the documentation of Docker and guide on quickstart.

export QLEVER_HOME=/local/data/qlever mkdir $QLEVER_HOME cd $QLEVER_HOME git clone --recursive -j8 https://github.com/ad-freiburg/qlever qlever-code cd qlever-code docker build -t qlever . [Getting error in this last step]

I recently saw the changes in this file. with this commit Parse triplesBlock and solutionModifier using ANTLR (https://github.com/ad-freiburg/qlever/pull/732)

Please help in building and setting it up.

18 233.1 [412/616] Building CXX object src/parser/sparqlParser/CMakeFiles/sparqlParser.dir/SparqlQleverVisitor.cpp.o

18 233.1 FAILED: src/parser/sparqlParser/CMakeFiles/sparqlParser.dir/SparqlQleverVisitor.cpp.o

18 233.1 /usr/bin/c++ -DLOGLEVEL=3 -D_PARALLEL_SORT -I/app/third_party/antlr4/runtime/Cpp/runtime/src -I/app/third_party/googletest/googletest/include -I/app/third_party/googletest/googlemock/include -I/app/third_party/json -I/app/third_party/ctre/include -I/app/third_party/abseil-cpp -I/app/src -isystem /app/third_party/stxxl/include -isystem /app/build/third_party/stxxl/include -isystem /app/third_party/stxxl/extlib/foxxll -isystem /app/build/third_party/stxxl/extlib/foxxll -isystem /app/third_party/stxxl/extlib/foxxll/extlib/tlx -isystem /app/third_party/re2 -Wall -Wextra -fopenmp -O3 -DNDEBUG -O3 -fcoroutines -fdiagnostics-color=always -std=gnu++20 -MD -MT src/parser/sparqlParser/CMakeFiles/sparqlParser.dir/SparqlQleverVisitor.cpp.o -MF src/parser/sparqlParser/CMakeFiles/sparqlParser.dir/SparqlQleverVisitor.cpp.o.d -o src/parser/sparqlParser/CMakeFiles/sparqlParser.dir/SparqlQleverVisitor.cpp.o -c /app/src/parser/sparqlParser/SparqlQleverVisitor.cpp

18 233.1 c++: fatal error: Killed signal terminated program cc1plus

18 233.1 compilation terminated.

I found a solution to it on StackOverflow. https://stackoverflow.com/questions/72632973/make-failing-with-not-used-function

shakti-rajput commented 2 years ago

@Qup42

joka921 commented 2 years ago

Hi, I think that your machine ran out of memory and killed one of the compiler processes (That's how I read the kill signal terminated stuff. How much RAM and what kind of CPU (how many cores/hardware threads) are you using? Maybe you can reduce the degree of parallelism used to compile in the Dockerfile (replace the ninja command at the end of line 19 to ninja -j <numThreads> where numThreads is smaller than your CPU count. Let us know if this helps.

shakti-rajput commented 2 years ago

Hello, Thanks for replying so fast. I have an I7 10th generation. 8 cores. 16 Logical Cores. @2.20 GHz. 16 GB RAM. In your comment " is smaller than your CPU count," are u referring to the number of cores?

joka921 commented 2 years ago

I would suggest you try out. I think on your machine, the default (ninja without arguments) runts 16 threads, which might be too much for 16GB of RAM, especially if other stuff is still running on the machine. Maybe try ninja -j8 or ninja -j4 (less threads will increase the compilation time, but decrease the amount of memory used and therefore the probability of crashes like this. Let me know what worked for you, or if you run into other troubles.

hannahbast commented 2 years ago

You should really use https://github.com/ad-freiburg/qlever-control , as is also recommended in the QLever README.md

If you use QLever via docker, you don't have to compile anything.

shakti-rajput commented 2 years ago

docker build -t qlever .

Thank you, @joka921. This slight modification worked. ninja -j8 for me.

shakti-rajput commented 2 years ago

You should really use https://github.com/ad-freiburg/qlever-control , as is also recommended in the QLever README.md

If you use QLever via docker, you don't have to compile anything.

I was trying to use qlever commands provided on the page, but it was not recognizing the qlever as the command in terminal.

Now also, even after successfully running docker build -t qlever . I am not able to run any qlever command. It is displaying the sudo: qlever: command not found. Not sure how to start it in local.

I arranged the folder structure in the way as mentioned in the qlever-control Cloned https://github.com/ad-freiburg/qlever in cd qlever/ and https://github.com/ad-freiburg/qlever-control in cd qlever/qlever-control Data I unzipped in cd qlever/ using 

wget https://github.com/wallscope/olympics-rdf/raw/master/data/olympics-nt-nodup.zip
unzip olympics-nt-nodup.zip

These were the instructions - Download the qlever script anywhere you like and call it once with the full path. The script will then tell you what else to do. We recommend that you have a directory "qlever" for all things QLever on your machine, with subdirectories for the different components, for example: "qlever-control" (this repository), "qlever-code" (if you want to download the code), "qlever-indices" (with a subfolder for each of your datasets and indices), etc. But it's not a must, you can organize your files any way you like.

Not related to my query but I could not find the repo qlever-indices

hannahbast commented 2 years ago

The instructions say: "Download the qlever script anywhere you like and call it once with the full path."

PS: I revised the README.md now. It was missing some new features of the qlever script.

shakti-rajput commented 2 years ago

Thank you It worked. I was able to understand from ReadMe.

After completion, I got this message.

QLEVER CONFIG

Checking your PATH ...
Added the directory "/mnt/c/Users/hp/Desktop/Freiburg/Semester3/HannaBastProject/qlever-code/qlever-control" to your PATH

Setting up bash autocompletion ...
Done, number of completions: 35

Creating new Qleverfile ...
Copied pre-configured Qleverfile for "olympics" into current directory.

Setup is complete
Type "qlever" and use autocompletion to see which actions are available. Add a
"show" in the end to see what an action does without executing it (for example,
"qlever index show"). Typing "qlever" without arguments gives some basic help
and pointers for further help. Edit your local "Qleverfile" to change settings.
shakti-rajput commented 2 years ago

Can we include this in the documentation of https://github.com/ad-freiburg/qlever/blob/master/docs/quickstart.md?

(replace the ninja command at the end of line 19 to ninja -j where numThreads is smaller than your core count. Maybe try ninja -j8 To avoid developer's error?

hannahbast commented 2 years ago

I have used the occasion to overhaul the way the qlever script provides help: https://github.com/ad-freiburg/qlever-control/commit/7fd542a449304591f436c193045a700e078b72e6

The quickstart.md script is outdated