Closed PEZ closed 2 years ago
So, same here as with #824. We will see the bitset variant flicker in its performance, until we have figured out a way to allow for pinning the heap allocation to the same core as the JVM gets scheduled to.
I decided to remove all NUMA stuff, since I couldn't get the predictable behaviour I wanted through them. Again, I will try use the Discussion forum to raise the awareness for this problem with the benchmark setup. Maybe we can find a way to start the container in a JVM friendly way for the solutions that need it...
However, in perusing the NUMA options path I changed the way the solution is run. Compiling it in the Docker step and then just running it with the java
command. (As opposed to using the clojure
command in the run step, which compiles and runs. (And four times for the four different versions.)
I couldn't see what was going wrong in the build step in CI. I've now moved the commands to a script and separated the commands in the chain. Might give me some more level of diagnostics to fix the issue...
CI logging shows that the uberdeps
packaging step runs for almost 20 minutes and then runs out of disk space:
#9 8.203 Downloading: uberdeps/uberdeps/1.1.2/uberdeps-1.1.2.jar from clojars
#9 10.65 [uberdeps] Packaging target/primes.jar...
#9 10.65 + ./**
#9 1176.8 {:clojure.main/message
#9 1176.8 "Execution error (IOException) at java.io.FileOutputStream/writeBytes (FileOutputStream.java:-2).\nNo space left on device\n",
#9 1176.8 :clojure.main/triage
#9 1176.8 {:clojure.error/class java.io.IOException,
#9 1176.8 :clojure.error/line -2,
#9 1176.8 :clojure.error/cause "No space left on device",
#9 1176.8 :clojure.error/symbol java.io.FileOutputStream/writeBytes,
#9 1176.8 :clojure.error/source "FileOutputStream.java",
#9 1176.8 :clojure.error/phase :execution},
#9 1176.8 :clojure.main/trace
#9 1176.8 {:via
#9 1176.8 [{:type java.io.IOException,
#9 1176.8 :message "No space left on device",
#9 1176.8 :at
#9 1176.8 [java.io.FileOutputStream writeBytes "FileOutputStream.java" -2]}],
#9 1176.8 :trace
#9 1176.8 [[java.io.FileOutputStream writeBytes "FileOutputStream.java" -2]
#9 1176.8 [java.io.FileOutputStream write "FileOutputStream.java" 349]
#9 1176.8 [java.io.BufferedOutputStream
#9 1176.8 flushBuffer
#9 1176.8 "BufferedOutputStream.java"
#9 1176.8 81]
#9 1176.8 [java.io.BufferedOutputStream write "BufferedOutputStream.java" 127]
#9 1176.8 [java.util.zip.DeflaterOutputStream
#9 1176.8 deflate
#9 1176.8 "DeflaterOutputStream.java"
#9 1176.8 255]
#9 1176.8 [java.util.zip.ZipOutputStream
#9 1176.8 closeEntry
#9 1176.8 "ZipOutputStream.java"
#9 1176.8 269]
#9 1176.8 [java.util.zip.ZipOutputStream finish "ZipOutputStream.java" 374]
#9 1176.8 [java.util.zip.DeflaterOutputStream
#9 1176.8 close
#9 1176.8 "DeflaterOutputStream.java"
#9 1176.8 238]
#9 1176.8 [java.util.zip.ZipOutputStream close "ZipOutputStream.java" 391]
#9 1176.8 [uberdeps.api$package$fn__1689 invoke "api.clj" 298]
#9 1176.8 [uberdeps.api$package invokeStatic "api.clj" 290]
#9 1176.8 [uberdeps.api$package invoke "api.clj" 283]
#9 1176.8 [uberdeps.uberjar$_main$fn__1705 invoke "uberjar.clj" 42]
#9 1176.8 [uberdeps.uberjar$_main invokeStatic "uberjar.clj" 40]
#9 1176.8 [uberdeps.uberjar$_main doInvoke "uberjar.clj" 17]
#9 1176.8 [clojure.lang.RestFn applyTo "RestFn.java" 137]
#9 1176.8 [clojure.lang.Var applyTo "Var.java" 705]
#9 1176.8 [clojure.core$apply invokeStatic "core.clj" 667]
#9 1176.8 [clojure.main$main_opt invokeStatic "main.clj" 514]
#9 1176.8 [clojure.main$main_opt invoke "main.clj" 510]
#9 1176.8 [clojure.main$main invokeStatic "main.clj" 664]
#9 1176.8 [clojure.main$main doInvoke "main.clj" 616]
#9 1176.8 [clojure.lang.RestFn applyTo "RestFn.java" 137]
#9 1176.8 [clojure.lang.Var applyTo "Var.java" 705]
#9 1176.8 [clojure.main main "main.java" 40]],
#9 1176.8 :cause "No space left on device"}}
#9 1176.8
#9 1176.8 Execution error (IOException) at java.io.FileOutputStream/writeBytes (FileOutputStream.java:-2).
#9 1176.8 No space left on device
Is it reasonable to assume that the change in your last commit addresses this?
I see. No, this will be the same with my change, afaiu. It might take me a while to figure out an alternative way. Want me to close this PR until I have something that might fix it?
No, that's not necessary as far as I'm concerned. This is just a review finding that needs to be followed up.
The latest commit has a chance to fix the problem. It also downloads dependencies, but hopefully not as many.
@PEZ CI now succeeds. Can I proceed with merging this?
Nice! Yes, then this is good to go from my perspective.
Description
Build an exectutable jar and run that, instead of compiling and running the program four times.
Contributing requirements
drag-race
as the target branch.