Fibers interfere with Clojure's Reflector

[mhluongo]

lein exec -p this snippet

(import java.io.ByteArrayOutputStream
        java.awt.image.BufferedImage
        javax.imageio.ImageIO)
(require '[co.paralleluniverse.pulsar.actors :refer [spawn]]
         '[co.paralleluniverse.pulsar.core :refer [defsfn]])
(defsfn test-screen [image]
  (let [baos (ByteArrayOutputStream.)]
    (ImageIO/write image "png" baos)))
(let [image (BufferedImage. 1 1 BufferedImage/TYPE_INT_ARGB)]
  (test-screen image)
  (spawn #(test-screen %) image))

and you get this beauty.

Exception in Fiber "fiber-10000001" java.lang.IllegalArgumentException: wrong number of arguments
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at clojure.lang.Reflector.invokeMatchingMethod(Reflector.java:93)
    at clojure.lang.Reflector.invokeStaticMethod(Reflector.java:207)
    at user$test_screen.invoke(test.clj:8)
    at co.paralleluniverse.pulsar.InstrumentedIFn.invoke(InstrumentedIFn.java:36)
    at user$eval2252$fn__2255$fn__2256.invoke(test.clj:11)
    at clojure.lang.AFn.applyToHelper(AFn.java:154)
    at clojure.lang.AFn.applyTo(AFn.java:144)
    at clojure.core$apply.invoke(core.clj:624)
    at user$eval2252$fn__2255.invoke(test.clj:11)
    at co.paralleluniverse.pulsar.InstrumentedIFn.invoke(InstrumentedIFn.java:32)
    at co.paralleluniverse.pulsar.ClojureHelper.suspendableInvoke(ClojureHelper.java:185)
    at co.paralleluniverse.pulsar.ClojureHelper$3.run(ClojureHelper.java:172)
    at co.paralleluniverse.actors.PulsarActor.doRun(PulsarActor.java:90)
    at co.paralleluniverse.actors.Actor.run0(Actor.java:667)
    at co.paralleluniverse.actors.ActorRunner.run(ActorRunner.java:51)
    at co.paralleluniverse.actors.Actor.run(Actor.java:236)
    at co.paralleluniverse.fibers.Fiber.run(Fiber.java:1003)
Exception in Fiber "fiber-10000001" java.lang.IllegalArgumentException: wrong number of arguments
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at clojure.lang.Reflector.invokeMatchingMethod(Reflector.java:93)
    at clojure.lang.Reflector.invokeStaticMethod(Reflector.java:207)
    at user$test_screen.invoke(test.clj:8)
    at co.paralleluniverse.pulsar.InstrumentedIFn.invoke(InstrumentedIFn.java:36)
    at user$eval2252$fn__2255$fn__2256.invoke(test.clj:11)
    at clojure.lang.AFn.applyToHelper(AFn.java:154)
    at clojure.lang.AFn.applyTo(AFn.java:144)
    at clojure.core$apply.invoke(core.clj:624)
    at user$eval2252$fn__2255.invoke(test.clj:11)
    at co.paralleluniverse.pulsar.InstrumentedIFn.invoke(InstrumentedIFn.java:32)
    at co.paralleluniverse.pulsar.ClojureHelper.suspendableInvoke(ClojureHelper.java:185)
    at co.paralleluniverse.pulsar.ClojureHelper$3.run(ClojureHelper.java:172)
    at co.paralleluniverse.actors.PulsarActor.doRun(PulsarActor.java:90)
    at co.paralleluniverse.actors.Actor.run0(Actor.java:667)
    at co.paralleluniverse.actors.ActorRunner.run(ActorRunner.java:51)
    at co.paralleluniverse.actors.Actor.run(Actor.java:236)
    at co.paralleluniverse.fibers.Fiber.run(Fiber.java:1003)

relevant parts of my project.clj

:dependencies [[org.clojure/clojure "1.6.0"]
                 [co.paralleluniverse/pulsar "0.6.2"]]
:java-agents [[co.paralleluniverse/quasar-core "0.6.2"]]
:jvm-opts ["-Dco.paralleluniverse.fibers.detectRunawayFibers=false"]

Running (clojure.reflect/reflect ImageIO) with a variety of prints and filters both inside and outside an actor don't show any obvious differences.

Am I missing something? I hope this is just misconfiguration- sorry in advance if so!

EDIT:

Running the above with spawn-fiber throws a similar exception, but spawn-thread works fine. As a temporary work-around, maybe there's a way to run the actor on a thread instead of a fiber?

JVM details:

java version "1.7.0_55"
OpenJDK Runtime Environment (IcedTea 2.4.7) (7u55-2.4.7-1ubuntu1~0.13.10.1)
OpenJDK 64-Bit Server VM (build 24.51-b03, mixed mode)

I also tried Oracle, same thing.

java version "1.7.0_65"
Java(TM) SE Runtime Environment (build 1.7.0_65-b17)
Java HotSpot(TM) 64-Bit Server VM (build 24.65-b04, mixed mode)

[circlespainter]

I think this has something to do with Clojure not being able to figure out the correct method to call, as ImageIO/write has 3 overloads with 3 args (at least here on JDK Mac OS X 1.8.0_25-b17).

So by annotating arguments with types it seems to work:

(import java.io.ByteArrayOutputStream
        java.io.OutputStream
        java.awt.image.BufferedImage
        java.awt.image.RenderedImage
        javax.imageio.ImageIO)

(require '[co.paralleluniverse.pulsar.actors :refer [spawn]]
         '[co.paralleluniverse.pulsar.core :refer [defsfn]])

(defsfn test-screen [^RenderedImage image]
  (let [^OutputStream baos (ByteArrayOutputStream.)
        ^String filename "png"]
    (ImageIO/write image ^String filename baos)))

(let [image (BufferedImage. 1 1 BufferedImage/TYPE_INT_ARGB)]
  (test-screen image)
  (spawn test-screen image))

Please confirm this works for you too.

[mhluongo]

That did it, thanks for the quick response!

[mhluongo]

It still seems strange that it works fine via thread, though, and not via fiber without the annotations... maybe worth mentioning in the docs?

[mhluongo]

Ah, now I'm running into this using other Clojure libraries (eg clj-webdriver).

[mhluongo]

Okay. So I've run into this issue with a number of 3rd-party Clojure libraries that at some point do Java interop.

My short-term solution is to break out the offending piece of code into a thread, and communicate the result of the code via channels. I've written a handy-dandy macro to do just that. Of course, this is a shim, since it obviates the benefits of fibers.

(require '[co.paralleluniverse.pulsar.core :refer [spawn-thread channel snd rcv])
(defmacro in-thread [& body]
  (let [chan-name (gensym)
        func-name (gensym)
        thread-name (gensym)
        results-name (gensym)]
    `(let [~chan-name (channel)
           ~func-name (fn []
                        (let [~results-name (do ~@body)]
                          (snd ~chan-name
                               (if (nil? ~results-name)
                                 :nil
                                 ~results-name))))
           ~thread-name (spawn-thread ~func-name)]
       (rcv ~chan-name))))

I think the trouble here is related to another issue I had the other day. I had to re-write a piece of a library we maintain (https://github.com/cardforcoin/shale) to avoid heavy use of reflection, since it was causing issues with pulsar. Basically, I was using reflection to change the accessibility of protected members, etc- and these changes didn't seem to be reflected via fiber.

More and more this is sounding like the instrumentation somehow interferes with reflection. I've updated the title of the issue to reflect the more general nature, though I suspect a fix might involve some changes in quasar.

[circlespainter]

Thanks for additional investigation, I did a few more experiments and the issue seems indeed a bit more articulate, I'm going to look deeper into it.

[circlespainter]

Verified with the the sample provided here by @mhluongo (and both Pulsar's and Comsat's full test suites run clean): closing https://github.com/puniverse/quasar/issues/73 via https://github.com/puniverse/quasar/commit/38be1d7c24348070c9f23e9c4d2cda394598f9e2 closes this one too.

[circlespainter]

@mhluongo If you could verify that the other issue you had is solved too (the one that caused you to rewrite some code in order to avoid Clojure reflection) it would be really helpful. Thanks for support!

[mhluongo]

@circlespainter I'll give it a try today and report back.

[mhluongo]

I'm pretty sure this is still an issue, though the above example is fixed. It might take me a while to pithily reproduce.

[mhluongo]

A quick bit from working with clj-webdriver before I have a full reproduction, in case that helps.

Exception in Fiber "fiber-10000002" java.lang.IllegalArgumentException: No implementation of method: :current-url of protocol: #'clj-webdriver.core/IDriver found for class: nil

Sorry I can't post the rest of the stacktrace, but I should have a reproduction soon. The above runs fine using thread-based actors using the below macro, but chokes on fibers.

  (defmacro spawn-in-thread
    "Our own version of pulsar's `spawn` that runs an actor in a thread.

    Note the this macro only works with functions, not anything defined with eg
    `defactor`."
    {:arglists '([:name? :trap? f & args])}
    [& args]
    (let [[{:keys [^String name
                   ^Boolean trap]
            :or {trap false}} body] (kps-args args)
          b   (gensym 'b)
          cls (gensym 'cls)]
      `(let [args#  (list ~@(rest body))
             ~b     (first ~body)
             ~cls   (fn [] (apply ~b args#))
             nme#   (when ~name (clojure.core/name ~name))
             f#     (suspendable! ~(if (== (count body) 1) b cls))
             mailbox-config# (->MailboxConfig -1 nil)
             ^Actor actor# (PulsarActor. nme# ~b ~trap mailbox-config# nil f#)
             thread# (.spawnThread actor#)]
         (.ref actor#))))

[circlespainter]

The first issue was caused by a subtle Quasar instrumentation problem triggered by a specific code pattern; actually it was not even related to reflection but Clojure's Reflector just happened to use that very code pattern.

My guess is that this last exception might be a different issue, so for now I'll leave the present one closed but a reproduction would help a lot in problem determination when you have the chance. Thanks!

[pron]

@mhluongo Hi. Do you have any more information you can share about the new issue you ran into?

[mhluongo]

I wasn't able to replicate it outside our code base. I could put together a minimal project and tar it up, but I'm not sure I could eliminate the dependencies.

[pron]

Whatever added information you can give us about the problem (partial stack traces), workarounds you've been able to find -- would be very helpful.

[mhluongo]

I don't think I'll be able to get to this until Monday but I'll see what I can do.