opencog / benchmark

Benchmarking the AtomSpace, the pattern matcher and other OpenCog systems
GNU Affero General Public License v3.0
6 stars 9 forks source link

Split profile_bindlink on data, configuration and runner #4

Closed vsbogd closed 6 years ago

vsbogd commented 6 years ago

One benchmark consists of atomspace and query. Both parts are expected to be described in scheme. Configuration file is used to describe benchmark data and number of iterations to run. Configuration file contains properties to tune output results and guile behavior. benchmarks_to_run property contains list of benchmarks to run, it also can be overriden using command line.

ngeiswei commented 6 years ago

@vsbogd, in the future, try to limit the first line of the commit comment to 60 chars (to the very max 70), that way a git history still looks nice on 80 col term. Typically, the first line would be the sum-up of the commit, then an empty line, followed by as much explanation as you'd like (keeping the 80 char limit per line). Most git front-ends automatically format commit messages according to these conventions.

Other than that the PR seems good, I'll have a detailed look at it tomorrow.

vsbogd commented 6 years ago

@ngeiswei, ok thanks, will stick to this rule.

ngeiswei commented 6 years ago

@vsbogd , I'm encountering the following problems with following the instructions in query/README.md. Some problems seem to come from my lack of experience in Linux profiling tools, some not, in any case you need to fix the README accordingly. Here there are

  1. The following apt-get install linux-tools doesn't work, my linux distro (Ubuntu 17.10) instead offers me the following alternative

    linux-tools-virtual 4.13.0.38.41
    linux-tools-lowlatency 4.13.0.38.41
    linux-tools-generic 4.13.0.38.41
  2. After that it just says oprofile, I suppose it's a command to run but I'm not sure, a bit more explanation would be welcome.

  3. After building mkdir build; cd build; cmake ..; make -j4, from that build directory I run

    valgrind --tool=callgrind ./opencog/benchmark/profile_bindlink

    it doesn't work. Running find . -name "profile_bindlink" finds nothing.

  4. In Section Using gprof, after building there is no profile_bindlink and no opencog/benchmark.

vsbogd commented 6 years ago

@ngeiswei, I didn't touch README.md yet, will fix it. To run all of query benchmarks one can use:

make benchmark

command. Profiling requires running build/atomspace/query/query_benchmark under atomspace/query dir. I will describe the process in README.md

vsbogd commented 6 years ago

@ngeiswei, added correct description into README.md

ngeiswei commented 6 years ago

@vsbogd The Overview Section of atomspace/query/README.md seems fine. However the Profiling Section isn't (either the variable QUERY_BENCHMARK is wrong, or the working folder). If you consider this isn't important for now, no problem, but at least add a "TODO: ignore the following Section for now", or something of that nature. Good documentation is key ;-)

vsbogd commented 6 years ago

thanks @ngeiswei, fixed it. I have tested with correct path but forgot changing it in readme.

linas commented 6 years ago

linux-tools-virtual 4.13.0.38.41 linux-tools-lowlatency 4.13.0.38.41 linux-tools-generic 4.13.0.38.41

You want generic. The virtual one is for docker, lxc environments, and lowlatency is for real-time kernels. Don't use the wrong one, perf might report wrong/confusing values.