Closed borkdude closed 2 years ago
Here's a screenshot with puget + the above changes made in a local checkout of fipp:
Note that [zprint]() already works with bb:
(ns zprint
(:require [babashka.deps :as deps]))
(deps/add-deps '{:deps {org.babashka/spec.alpha {:git/url "https://github.com/babashka/spec.alpha"
:git/sha "1a841c4cc1d4f6dab7505a98ed2d532dd9d56b78"}
zprint/zprint {:mvn/version "1.2.3"}}})
(require '[zprint.core :as zp])
(println (zp/zprint-file-str (slurp *file*) "yolo" {:color? true :style :dark-color-map}))
but some users would like to use fipp as well.
Submitted a PR to address point 2: https://github.com/brandonbloom/fipp/pull/82 I think that change would be good in general, not only for bb.
Hey @borkdude, thanks for the detailed writeup and PR #82 (merged!).
In general, I'm willing to make improvements (cleanup, fixes, etc) that also extend compatibility, but would like to avoid accreting compatibility cruft. On that front, I'm disinclined to eliminate the dependency on rrb vectors & would instead encourage you to upstream the compatibility fixes there.
Perhaps worth mentioning is that Fipp used to use finger-trees for ClojureScript (See a2bd18a92a8480bd84a0410ae9041ed6f2fedcf0), but rrb-vectors grew support there. I might also consider an alternative dependency that provides a dequeue interface more abstractly.
On that front, I'm disinclined to eliminate the dependency on rrb vectors
Just to be clear, I'm not asking to eliminate the dependency, the change would be hidden behind reader conditionals for bb. clj + cljs would still be using rrb. But I understand that you would not like to deal with compatibility for more than clj and cljs.
I haven't looked too deeply into this library, but Java also comes with a Deque implementation:
https://docs.oracle.com/javase/7/docs/api/java/util/Deque.html
Perhaps you've already considered that.
Needs a functional/persistent dequeue.
I didn't mean to cause you to close this issue. Still open to small changes to support Babashka, but my hope is that can be as easy as bumping an rrb-vector dependency.
A few thoughts:
I don't think it makes a lot of sense to make rrb compatible from source so we can run rrb from source with babashka. Performance of code executed with babashka is several times slower than executed with the JVM. Falling back on regular vector will likely make things faster in babashka than executing rrb from source to optimize concatenation of vectors. Of course I haven't measured this and I might be wrong.
Babashka has built-in dependencies which execute as fast (since they're just using the compiled code). Babashka could decide to include rrb as a built-in dep. But so far, I haven't seen many other libraries that need rrb, other than those depending on puget / fipp.
Maintaining a fork of fipp is still an option. Maintaining a separate repo that creates a "polyfill" for rrb that falls back on clojure.core is another option. The latter option seems to be the least intrusive for fipp. I could give that a try first.
Babashka could decide to include rrb as a built-in dep.
Looking at this patch here: https://github.com/babashka/babashka/issues/1241#issue-1204914375
I don't know anything about Babashka, but do you have some sort of library/namespace/function stub/patch mechanism? Seems like you could stub either fipp.deque or clojure.core.rrb-vector and you only need that one catvec function?
@brandonbloom Babashka is able to "fake" any Clojure function via the SCI (interpreter) configuration, so that's certainly an option to try. I will try that soon.
I have a development version of babashka here (for macOS):
which can run fipp (the current version from master).
What I've done is include rrb vector proper, but only the catvec
function. This adds 350kb to the binary. Might be worth it, but I'll wait for feedback in https://github.com/greglook/puget/issues/56, to see if I can keep it at that for compatibility for fipp + puget.
$ rlwrap ./bb
Babashka v0.8.2-SNAPSHOT REPL.
Use :repl/quit or :repl/exit to quit the REPL.
Clojure rocks, Bash reaches.
user=> (babashka.deps/add-deps '{:deps {fipp/fipp {:deps/manifest :deps :git/url "https://github.com/brandonbloom/fipp" :git/sha "efdf87f22a6fcc08cef5017c14769df29fc37850"}}})
nil
user=> (require '[fipp.edn :as fipp])
nil
user=> (fipp/pprint (range 45))
(0
1
2
3
4
Finally got to the point that hashp is working in babashka (current master version)
;; add fipp from master
(babashka.deps/add-deps '{:deps {fipp/fipp {:deps/manifest :deps :git/url "https://github.com/brandonbloom/fipp" :git/sha "efdf87f22a6fcc08cef5017c14769df29fc37850"}}})
;; add newest zprint which is already compatible with bb (hashp also loads zprint)
(babashka.deps/add-deps '{:deps {zprint/zprint {:mvn/version "1.2.3"}}})
;; add babashka fork of clojure spec
(babashka.deps/add-deps '{:deps {org.babashka/spec.alpha {:git/url "https://github.com/babashka/spec.alpha"
:git/sha "1a841c4cc1d4f6dab7505a98ed2d532dd9d56b78"}}})
;; add hashp
(babashka.deps/add-deps '{:deps {hashp/hashp {:mvn/version "0.2.1"}}})
(require '[hashp.core])
(set! *data-readers* {'p hashp.core/p*})
If you can release a new version of fipp, then we don't need the git/sha stuff anymore and then I think we can close this issue.
What I've done is include rrb vector proper, but only the catvec function. This adds 350kb to the binary.
I'm not sure I follow, but I am curious: What does it mean to add it "proper" but only one function?
deployed: https://clojars.org/fipp/versions/0.6.26
@brandonbloom It means I've only exposed the catvec
function to the interpreter:
https://github.com/babashka/babashka/blob/master/feature-rrb-vector/babashka/impl/rrb_vector.clj
GraalVM native-image will analyze what is used and will not include other vars in the native image, if they are not reachable.
Thanks!
It means I've only exposed the catvec function to the interpreter
Since catvec returns an object of the rrb vector class, presumably that means a fair amount of code is then included?
@brandonbloom Yes, that fair amount of code yields about 350kb.
Hi @brandonbloom!
Babashka is a Clojure interpreter which is suitable for scripting and has (very) fast startup. It has the ability to execute Clojure libraries from source. Sometimes minor changes are needed to avoid things that babashka cannot (currently) do in a similar way how you would make a library compatible with ClojureScript: through reader conditionals. A recent example of this is the commit in specter.
Recently I've gotten questions about whether fipp and puget (which uses fipp) can run with babashka and I started investigating. I found that with a few tweaks, it can work. Details here: https://github.com/babashka/babashka/issues/1241.
Some things can be fixed in babashka (or the Clojure interpreter library it uses: SCI). Some things are better solved in the library. I will outline those here. Note that these changes can be made in such a way that they only affect behavior for bb while not changing anything for clj/cljs.
in
dequeue.cljc
fall back on clojure.core vector operations instead of using rrb-vector. I realize this is worse performance-wise but for babashka this may be a reasonable compromise to get things working without pulling in rrb-vector.in
format-hack
in ednize.clj start adoptingjava.time
rather then relying on ThreadLocals inclojure.instant
. Babashka has full support forjava.time
whileclojure.instant
is based on classes prior tojava.time
.java.time
is the modern way to do stuff with dates and times in Java and is available since Java 1.8.My question to you: are you open to receiving PRs for (some of) the above issues, or should users of babashka maintain a bb-compatible fork somewhere else?