PlummersSoftwareLLC / Primes

Prime Number Projects in C#/C++/Python
https://plummerssoftwarellc.github.io/PrimeView/
2.46k stars 573 forks source link

Clojure solution 3 build executable jar #825

Closed PEZ closed 2 years ago

PEZ commented 2 years ago

Description

Build an exectutable jar and run that, instead of compiling and running the program four times.

Contributing requirements

PEZ commented 2 years ago

So, same here as with #824. We will see the bitset variant flicker in its performance, until we have figured out a way to allow for pinning the heap allocation to the same core as the JVM gets scheduled to.

rbergen commented 2 years ago

Same question and comment as posted in #824.

PEZ commented 2 years ago

I decided to remove all NUMA stuff, since I couldn't get the predictable behaviour I wanted through them. Again, I will try use the Discussion forum to raise the awareness for this problem with the benchmark setup. Maybe we can find a way to start the container in a JVM friendly way for the solutions that need it...

However, in perusing the NUMA options path I changed the way the solution is run. Compiling it in the Docker step and then just running it with the java command. (As opposed to using the clojure command in the run step, which compiles and runs. (And four times for the four different versions.)

PEZ commented 2 years ago

I couldn't see what was going wrong in the build step in CI. I've now moved the commands to a script and separated the commands in the chain. Might give me some more level of diagnostics to fix the issue...

rbergen commented 2 years ago

CI logging shows that the uberdeps packaging step runs for almost 20 minutes and then runs out of disk space:

#9 8.203 Downloading: uberdeps/uberdeps/1.1.2/uberdeps-1.1.2.jar from clojars
#9 10.65 [uberdeps] Packaging target/primes.jar...
#9 10.65 + ./**
#9 1176.8 {:clojure.main/message
#9 1176.8  "Execution error (IOException) at java.io.FileOutputStream/writeBytes (FileOutputStream.java:-2).\nNo space left on device\n",
#9 1176.8  :clojure.main/triage
#9 1176.8  {:clojure.error/class java.io.IOException,
#9 1176.8   :clojure.error/line -2,
#9 1176.8   :clojure.error/cause "No space left on device",
#9 1176.8   :clojure.error/symbol java.io.FileOutputStream/writeBytes,
#9 1176.8   :clojure.error/source "FileOutputStream.java",
#9 1176.8   :clojure.error/phase :execution},
#9 1176.8  :clojure.main/trace
#9 1176.8  {:via
#9 1176.8   [{:type java.io.IOException,
#9 1176.8     :message "No space left on device",
#9 1176.8     :at
#9 1176.8     [java.io.FileOutputStream writeBytes "FileOutputStream.java" -2]}],
#9 1176.8   :trace
#9 1176.8   [[java.io.FileOutputStream writeBytes "FileOutputStream.java" -2]
#9 1176.8    [java.io.FileOutputStream write "FileOutputStream.java" 349]
#9 1176.8    [java.io.BufferedOutputStream
#9 1176.8     flushBuffer
#9 1176.8     "BufferedOutputStream.java"
#9 1176.8     81]
#9 1176.8    [java.io.BufferedOutputStream write "BufferedOutputStream.java" 127]
#9 1176.8    [java.util.zip.DeflaterOutputStream
#9 1176.8     deflate
#9 1176.8     "DeflaterOutputStream.java"
#9 1176.8     255]
#9 1176.8    [java.util.zip.ZipOutputStream
#9 1176.8     closeEntry
#9 1176.8     "ZipOutputStream.java"
#9 1176.8     269]
#9 1176.8    [java.util.zip.ZipOutputStream finish "ZipOutputStream.java" 374]
#9 1176.8    [java.util.zip.DeflaterOutputStream
#9 1176.8     close
#9 1176.8     "DeflaterOutputStream.java"
#9 1176.8     238]
#9 1176.8    [java.util.zip.ZipOutputStream close "ZipOutputStream.java" 391]
#9 1176.8    [uberdeps.api$package$fn__1689 invoke "api.clj" 298]
#9 1176.8    [uberdeps.api$package invokeStatic "api.clj" 290]
#9 1176.8    [uberdeps.api$package invoke "api.clj" 283]
#9 1176.8    [uberdeps.uberjar$_main$fn__1705 invoke "uberjar.clj" 42]
#9 1176.8    [uberdeps.uberjar$_main invokeStatic "uberjar.clj" 40]
#9 1176.8    [uberdeps.uberjar$_main doInvoke "uberjar.clj" 17]
#9 1176.8    [clojure.lang.RestFn applyTo "RestFn.java" 137]
#9 1176.8    [clojure.lang.Var applyTo "Var.java" 705]
#9 1176.8    [clojure.core$apply invokeStatic "core.clj" 667]
#9 1176.8    [clojure.main$main_opt invokeStatic "main.clj" 514]
#9 1176.8    [clojure.main$main_opt invoke "main.clj" 510]
#9 1176.8    [clojure.main$main invokeStatic "main.clj" 664]
#9 1176.8    [clojure.main$main doInvoke "main.clj" 616]
#9 1176.8    [clojure.lang.RestFn applyTo "RestFn.java" 137]
#9 1176.8    [clojure.lang.Var applyTo "Var.java" 705]
#9 1176.8    [clojure.main main "main.java" 40]],
#9 1176.8   :cause "No space left on device"}}
#9 1176.8 
#9 1176.8 Execution error (IOException) at java.io.FileOutputStream/writeBytes (FileOutputStream.java:-2).
#9 1176.8 No space left on device

Is it reasonable to assume that the change in your last commit addresses this?

PEZ commented 2 years ago

I see. No, this will be the same with my change, afaiu. It might take me a while to figure out an alternative way. Want me to close this PR until I have something that might fix it?

rbergen commented 2 years ago

No, that's not necessary as far as I'm concerned. This is just a review finding that needs to be followed up.

PEZ commented 2 years ago

The latest commit has a chance to fix the problem. It also downloads dependencies, but hopefully not as many.

rbergen commented 2 years ago

@PEZ CI now succeeds. Can I proceed with merging this?

PEZ commented 2 years ago

Nice! Yes, then this is good to go from my perspective.