circlespainter
commented
9 years ago Hi, core.async's go blocks give you a limited form of sequential, blocking (so, easier and more expressive) programming model for channels operations that performs on par with async APIs. It is limited because it is implemented by means of the go macro and meant to be used only with accompanying, special core.async channel operations.
Pulsar uses Quasar's fibers (i.e. extremely lightweight threads implemented through bytecode instrumentation) to provide a compatible core.async implementation that will allow you to use a sequential, blocking programming model not limited to go blocks but to all fiber-blocking operations, while still retaining async-like efficiency.
Of course if you call existing thread-blocking APIs from go blocks they will block a whole thread, not just the macro-built continuation created by the go block, and similarly if you call thread-blocking operations from a fiber they will block a whole thread and not just the fiber (the warning you show is provided by an useful check done by Quasar that will let you know if fibers are "monopolizing" threads by means of blocking ops, like I/O, or long computations).
In order to avoid losing efficiency you should use async I/O in go blocks.
You can do the same inside fibers of course, but async is less expressive than blocking, so Quasar and Pulsar provide easy methods to transform any async APIs into fiber-blocking APIs (and that's a good portion of the scope of the Comsat project). Or you can still easily delegate the thread-blocking call to a dedicated thread pool if no async API is available for the I/O operation you wish to perform. Have a look here: http://docs.paralleluniverse.co/pulsar/#transforming-any-asynchronous-callback-to-a-fiber-blocking-operation , here http://docs.paralleluniverse.co/quasar/#transforming-any-asynchronous-callback-to-a-fiber-blocking-operation and here http://blog.paralleluniverse.co/2014/08/12/noasync/ .
There's no specific core.async benchmark as of yet, but in the meanwhile you can find entry points to our Quasar/Pulsar fibers (which are the building block of our core.async implementation too) performance measuring work and benchmarks in several blog posts, such as http://blog.paralleluniverse.co/2014/05/29/cascading-failures/ and http://blog.paralleluniverse.co/2014/02/06/fibers-threads-strands/.
As a last note, our core.async implementation is being upgraded to the latest core.async API version, including the new "ops" upper layer and we'd like to keep it in sync timely as the core.async API evolves.
cocodrino
Hi!!...sorry for the super noob question..but I was recently reading this article
http://martintrojer.github.io/clojure/2013/07/07/coreasync-and-blocking-io/
seems than clojure core.async go blocks can be blocked and I would need use threads and a thread pool, this is not optimal...in a comment (I don't know if who comment works for paralleluniverse) says than fiber hasn't that problem, is it true?...
I try replicate the code but I get almost the same result than using cor async and this warning
WARNING: fiber Fiber@10000028[task: ParkableForkJoinTask@176cd783(Fiber@10000028), target: co.paralleluniverse.pulsar.ClojureHelper$3@3592b423, scheduler: co.paralleluniverse.fibers.FiberForkJoinScheduler@6d25f36b] is hogging the CPU or blocking a thread. at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:150) at java.net.SocketInputStream.read(SocketInputStream.java:121) at org.apache.http.impl.io.AbstractSessionInputBuffer.read(AbstractSessionInputBuffer.java:198) at org.apache.http.impl.io.ContentLengthInputStream.read(ContentLengthInputStream.java:178) at org.apache.http.conn.EofSensorInputStream.read(EofSensorInputStream.java:137) at java.io.FilterInputStream.read(FilterInputStream.java:133) at clj_http.core.proxy$java.io.FilterInputStream$ff19274a.read(Unknown Source) at java.io.FilterInputStream.read(FilterInputStream.java:107) at clj_http.core.proxy$java.io.FilterInputStream$ff19274a.read(Unknown Source) at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1792) at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1769) at org.apache.commons.io.IOUtils.copy(IOUtils.java:1744) at org.apache.commons.io.IOUtils.toByteArray(IOUtils.java:462) at clj_http.util$force_byte_array.invoke(util.clj:63) at clj_http.client$eval11476$fn11479.invoke(client.clj:386) at clojure.lang.MultiFn.invoke(MultiFn.java:231) at clj_http.client$wrap_output_coercion$fn11484.invoke(client.clj:400) at clj_http.client$wrap_exceptions$fn11327.invoke(client.clj:164) at clj_http.client$wrap_accept$fn__11524.invoke(client.clj:522) at clj_http.client$wrap_accept_encoding$fn11530.invoke(client.clj:536) at clj_http.client$wrap_content_type$fn11519.invoke(client.clj:512) at clj_http.client$wrap_form_params$fn__11610.invoke(client.clj:683) at clj_http.client$wrap_nested_params$fn11627.invoke(client.clj:707) at clj_http.client$wrap_method$fn11570.invoke(client.clj:624) at clj_http.cookies$wrap_cookies$fn7993.invoke(cookies.clj:122) at clj_http.links$wrap_links$fn9872.invoke(links.clj:50) at clj_http.client$wrap_unknown_host$fn11635.invoke(client.clj:726) at clj_http.client$get.doInvoke(client.clj:829) at clojure.lang.RestFn.invoke(RestFn.java:410) at alepthbenchmark.bmch$blocking_get.invoke(form-init7142513807440439850.clj:2) at alepthbenchmark.bmch$eval11770$fn11775.invoke(form-init7142513807440439850.clj:6) at co.paralleluniverse.pulsar.InstrumentedIFn.invoke(InstrumentedIFn.java:32) at co.paralleluniverse.pulsar.async$f__GT_chan$fn7817$fn7818.invoke(async.clj:213) at co.paralleluniverse.pulsar.async$fGT_chan$fn__7817.invoke(async.clj:213) at co.paralleluniverse.pulsar.InstrumentedIFn.invoke(InstrumentedIFn.java:32) at co.paralleluniverse.pulsar.ClojureHelper.suspendableInvoke(ClojureHelper.java:185) at co.paralleluniverse.pulsar.ClojureHelper$3.run(ClojureHelper.java:172) at co.paralleluniverse.fibers.Fiber.run(Fiber.java:1003)
do you have any benchmark of clojure core.async with respect to the paralleluniverse implementation?...