MOZI-AI / annotation-scheme

Human Gene annotation service backend
GNU General Public License v3.0
3 stars 4 forks source link

Add locks as a fix for issue #144 #161

Closed linas closed 4 years ago

linas commented 4 years ago

There are two possible fixes for issue #144 - one is ton not use par-map in main.scm, The problem with par-map is that while it does reduce latency, it does so at considerable cost, I measure the following for multiple-threads:

Elapsed: 526.03 secs. Rate: 28.86 gc/min %cpu-GC: 113.2%  %cpu-use: 266.9%

vs single-theaded performance:

Elapsed: 814.31 secs. Rate: 19.82 gc/min %cpu-GC: 52.31%  %cpu-use: 139.1%

The single-threaded elapsed time is longer, bu the CPU usage is a lot lower.

Never the less, if you really want to use par-map, this patch fixes the crashes that would otherwise result. It slows down overall performance by 2%.

Habush commented 4 years ago

@linas there are compilation errors with this PR. Please check.

tanksha commented 4 years ago

@linas I am getting unbound variable memoize-function-call which module it is defined in?

linas commented 4 years ago

OK, I'll look at this shortly. It has merge conflicts that need to be fixed, anyway.

linas commented 4 years ago

I merged in master and resolved all conflicts ... unit tests pass for me, but fail here; I don't know why. What is the actual error message?

Habush commented 4 years ago

Here is the output from the CI:

./env /usr/bin/guild compile -Wunbound-variable -Warity-mismatch -Wformat -o "annotation/util.go" "annotation/util.scm"
;;; note: auto-compilation is enabled, set GUILE_AUTO_COMPILE=0
wrote `annotation/util.go'
;;;       or pass the --no-auto-compile argument to disable.
;;; compiling /usr/bin/guild
;;; compiled /github/home/.cache/guile/ccache/2.2-LE-8-3.A/usr/bin/guild.go
./env /usr/bin/guild compile -Wunbound-variable -Warity-mismatch -Wformat -o "annotation/functions.go" "annotation/functions.scm"
Backtrace:
In system/base/compile.scm:
   235:18 19 (read-and-compile #<input: annotation/functions.scm 13> …)
   183:32 18 (compile-fold (#<procedure compile-tree-il (x e opts)>) …)
In ice-9/boot-9.scm:
   2312:4 17 (save-module-excursion _)
In language/scheme/compile-tree-il.scm:
    31:15 16 (_)
In ice-9/psyntax.scm:
  1235:36 15 (expand-top-sequence _ _ _ #f _ _ _)
  1182:24 14 (parse _ (("placeholder" placeholder)) ((top) #(# # …)) …)
    285:10 13 (parse _ (("placeholder" placeholder)) (()) _ c&e (# #) #)
In ice-9/eval.scm:
   293:34 12 (_ #<module (#{ g75}#) 55af7c70f640>)
In ice-9/boot-9.scm:
   2862:4 11 (define-module* _ #:filename _ #:pure _ #:version _ # _ …)
  2071:24 10 (call-with-deferred-observers _)
  2875:24  9 (_)
   222:17  8 (map1 (((annotation util)) ((opencog)) ((opencog #)) # …))
  2788:17  7 (resolve-interface (annotation util) #:select _ #:hide _ …)
  2714:10  6 (_ (annotation util) _ _ #:ensure _)
  2982:16  5 (try-module-autoload _ _)
   2312:4  4 (save-module-excursion _)
  3002:22  3 (_)
In unknown file:
           2 (primitive-load-path "annotation/util" #<procedure 55af…>)
In annotation/util.scm:
     62:0  1 (_)
In ice-9/boot-9.scm:
   752:25  0 (dispatch-exception _ _ _)

ice-9/boot-9.scm:752:25: In procedure dispatch-exception:
In procedure module-lookup: Unbound variable: memoize-function-call
make: *** [annotation/functions.go] Error 1
Makefile:1092: recipe for target 'annotation/functions.go' failed
Habush commented 4 years ago

So the issue was it was the memoize-function-call function was getting called before it was defined. Odd, didn't know guile required that. Anyways, it passes all the checks now.

linas commented 4 years ago

OK, last commit fixed this.

Also my ad-blocker was hiding the circle-ci fail messages; I can see them now...

linas commented 4 years ago

Noramlly, in guile, use-before define is fine. However, in this case, you've made each different file into a module, and while compiling the (annotation functions) module, it has to reanch inside of teh (annotation util) module, and got messed up there. But util.scm was compiling just fine...

And also .. on my system, everything was working fine, so this might also be a bug in your guile version...

(I'm running guile-3.01 now)