Closed josh-7t closed 2 years ago
@josh-7t Thanks for submitting this PR to improve byte-streams. I appreciate the effort and the interest.
However, there's two issues, one big and one small.
The small issue is the fn as written violates the Comparable contract. If the objects compared aren't equal, it will always return -1, which would mean the objects are always before each other, which is impossible. This is at least fixable by picking an artificial ordering based on type
and wrapper
. There's no inherent ordering of Types, but as long as it's consistent for TreeBin, it might be ok.
The bigger issue is the ConcurrentHashMap behind fast-memoize is using reflection, which is not exactly friendly to GraalVM. In addition to Java code, Zach's code wasn't shy about doing stuff like this either, so even if we change this here, you're likely to run into it again elsewhere. I suspect supporting Graal is a much bigger endeavor than a few PRs will solve.
That being said, I'm not convinced fast-memoize is worth it over plain memoize, if it's causing issues. If you want to replace it, I'd support that. We'd need a performance comparison between the two. As long as it's not egregious, they should be swappable.
Thanks for responding! I think I would be interested in replacing fast-memoize
since Implementing Comparable
on this type seems very arbitrary.
Would you be amenable to me using https://github.com/clojure/core.memoize instead of the default memoize? It seems like it's implemented to work better for multithreaded apps and it has different cache strategies available so that the cache doesn't just grow forever.
superseded by https://github.com/clj-commons/byte-streams/pull/50
After compiling aleph with graal native-image calling
manifold.stream.SplicedStream.put
appears to pass abyte_stream.graph.Type
tobyte_streams.utils$fast_memoize
The following stck trace suggests that in the graal environment
fast-memoize
needs it's arguments to implementjava.lang.Coparable
interface: