dependency hell

[awb99]

aleph - Asynchronous streaming communication for Clojure - web server, web client, and raw TCP/UDP

aleph/aleph {:mvn/version "0.3.0-beta8"} aleph/aleph {:mvn/version "0.6.4"} ; current clojars [aleph "0.7.0-alpha2"] ;; current github => the aleph version used is very old.

https://github.com/clj-commons/aleph Aleph exposes data from the network as a Manifold stream, which can easily be transformed into a java.io.InputStream, core.async channel, Clojure sequence, or many other byte representations. It exposes simple default wrappers for HTTP, TCP, and UDP, but allows access to full performance and flexibility of the underlying Netty library.

[awb99]

https://github.com/ztellman/lamina This repository has been archived by the owner on Nov 9, 2017. It is now read-only. Repo has been split up to multiple libraries. I guess this is related to threading. lamina/lamina {:mvn/version "0.5.0-beta8"} ;lamina/lamina {:mvn/version "0.5.6"} ; 0.5.6 does not work.

lamina.core, lamina.cache, lamina.time -> Manifold I guess we need to switch to manifold.

[awb99]

gloss/gloss {:mvn/version "0.2.2-beta3"} org.clj-commons/gloss {:mvn/version "0.3.6"} https://github.com/clj-commons/gloss https://cljdoc.org/d/org.clj-commons/gloss/0.3.6/doc/readme speaks in bytes, so that you don't have to 476 github stars - so likely well maintained just one function call used.

[awb99]

edw/ordered {:mvn/version "1.3.2"} https://github.com/edw/ordered ... this link does not work. Pure-clojure implementation of ruby's ordered hash and set types - instead of sorting by key, these collections retain insertion order. Features less bloated syntax vis a vis the original flatland implementation. DEPENDENCY REMOVED. AS FIX-ENGINE IS NOT USING IT.

[awb99]

clj -Stree

 . aleph/aleph 0.3.0-beta8
    X org.clojure/tools.logging 0.2.3 :superseded
    . org.clojure/data.xml 0.0.6
    . io.netty/netty 3.5.8.Final
    X lamina/lamina 0.5.0-beta7 :older-version
    . gloss/gloss 0.2.2-beta3
    X cheshire/cheshire 4.0.1 :older-version
    . criterium/criterium 0.3.0
    . commons-codec/commons-codec 1.7
    . org.apache.commons/commons-compress 1.4.1
      . org.tukaani/xz 1.0
  . lamina/lamina 0.5.0-beta8
    . org.clojure/tools.logging 0.2.4 :newer-version
    . useful/useful 0.8.2
      . org.clojure/tools.macro 0.1.1
    . potemkin/potemkin 0.1.5
    . criterium/criterium 0.3.0
    . com.yammer.metrics/metrics-core 2.1.0
  . gloss/gloss 0.2.2-beta3
    X lamina/lamina 0.5.0-beta7 :older-version
    . potemkin/potemkin 0.1.5

[awb99]

failed to do ANY updates in aleph/gloss/lamina.

[awb99]

aleph 0.3.0-beta depends on

• gloss - byte-encoding
• lamina - threading

aleph 0.6.4 depends on

• manifold 0.3.0 - threading
• org.clj-commons/byte-streams 0.3.1] - byte encoding

[awb99]

lamina used functions: l/enqueue -> manifold.stream/put! (not sure, but would make sense) l/closed? l/receive-all -> manifold.stream/take! (not sure, but would make sense) l/close

[awb99]

In trateg, it is not possible to use fix-engine at the moment. It is because lamina uses potemkin which defines defprotocol-once and the later versions of potemkin seem to no longer have this function/protocol.

[awb99]

(:require [manifold.deferred :as d]
              [manifold.stream :as s]
              [aleph.tcp :as tcp])

    (def aleph-client (atom nil)) 

    ;store my initial connection into the atom
    (reset! aleph-client @(tcp/client {:host (:HOST CONFIG) :port (:PORT CONFIG)}))

    (defn msg! 
      "A very simple request response method"
      [ bytebuf ]
       (d/let-flow [status (s/put! @aleph-client (.array bytebuf))
                    reply  @(s/take! @aleph-client)]
                 reply))

[awb99]

decode-stream for tcp client 121 views Subscribe Vladimir Lebedev's profile photo Vladimir Lebedev unread, 11/27/14 to alep...@googlegroups.com Hi,

I'm trying to make decode-stream work in new version of aleph (0.4.0), but without any success. Here is the code:

(ns avro-producer.core (:use [lamina.core] [gloss.core] [gloss.io :only (decode-stream)]) (:require [aleph.tcp :as tcp] [manifold.stream :as s] [byte-streams :as bs]))

(def tcp-stream @(tcp/client {:host "bravo" :port 5353}))

(def messages (decode-stream tcp-stream (string :utf-8 :delimiters ["\n"])))

(defn -main [& args] (s/consume bs/print-bytes messages))

If I substitute 'messages' with 'tcp-stream' in the last line of code, I get nice printout of my incoming data. If this line left intact, my program just hangs and doesn't print anything. Here is the data which is pumped into the client, it is just JSON strings separated by a "\n" (0x0A):

7B 22 69 64 22 3A 22 34 34 37 36 36 34 33 64 2D {"id"."4476643d- 32 30 63 32 2D 34 30 65 32 2D 37 36 30 37 2D 37 20c2-40e2-7607-7 39 63 37 30 30 65 64 61 38 66 64 22 2C 22 75 72 9c700eda8fd","ur 6C 22 3A 22 68 74 74 70 73 3A 2F 2F 77 77 77 2E l"."https.//www. 61 76 69 74 6F 2E 72 75 2F 61 6E 61 70 61 2F 70 avito.ru/anapa/p 72 65 64 6C 6F 7A 68 65 6E 69 79 61 5F 75 73 6C redlozheniyausl 75 67 2F 6D 61 6E 69 6B 79 75 72 6C 61 6B 2E 5F ug/manikyurlak. 6D 61 6E 69 6B 79 75 72 67 65 6C 5F 70 65 64 69 manikyurgel_pedi 6B 79 75 72 67 65 6C 5F 69 5F 6C 61 6B 5F 76 79 kyurgel_i_lak_vy 5F 34 36 37 34 31 37 32 31 35 22 2C 22 69 70 22 _467417215","ip" 3A 22 38 35 2E 31 37 35 2E 31 33 32 2E 35 36 22 ."85.175.132.56" 2C 22 75 61 22 3A 22 4D 6F 7A 69 6C 6C 61 2F 35 ,"ua"."Mozilla/5 2E 30 20 28 57 69 6E 64 6F 77 73 20 4E 54 20 36 .0 (Windows NT 6 2E 31 3B 20 72 76 3A 33 33 2E 30 29 20 47 65 63 .1; rv.33.0) Gec 6B 6F 2F 32 30 31 30 30 31 30 31 20 46 69 72 65 ko/20100101 Fire 66 6F 78 2F 33 33 2E 30 22 2C 22 74 73 22 3A 31 fox/33.0","ts".1 34 31 37 30 39 31 39 38 38 39 33 33 7D 0A 417091988933}. 7B 22 69 64 22 3A 22 66 65 38 33 38 30 32 63 2D {"id"."fe83802c- 39 33 39 38 2D 34 66 34 36 2D 63 66 37 39 2D 36 9398-4f46-cf79-6 63 30 34 39 61 30 33 34 32 32 64 22 2C 22 75 72 c049a03422d","ur 6C 22 3A 22 68 74 74 70 3A 2F 2F 79 61 6E 64 65 l"."http.//yande [.......]

The goal is to have each individual JSON string in a separate message on the 'messages' stream.

Many thanks in advance,

Vladimir Vladimir Lebedev's profile photo Vladimir Lebedev unread, 11/27/14 to alep...@googlegroups.com I've managed to figure it out by myself, using the gist from Jonatan Jacob's Oct 27th post. My problem was that I did not convert original tcp stream into stream of byte-buffers. Here is solution which worked:

(ns avprod.core (require [aleph.tcp :as tcp] [gloss.core :as gloss] [gloss.io :as gio] [byte-streams :as bs] [manifold.stream :as s] [manifold.deferred :as d]) (:import [java.nio ByteBuffer]) (:gen-class))

(defn- to-bb [value] (bs/convert value ByteBuffer))

(def json-frame (gloss/string :ascii :delimiters ["\n"]))

(defn main-loop [] (d/let-flow [src (tcp/client {:host "bravo" :port 5353}) dst (s/stream)] (s/connect-via src #(s/put! dst (to-bb %)) dst) (s/consume #(println "msg =>" %) (gio/decode-stream dst json-frame))))

(defn -main [& args] (main-loop))

[awb99]

this is working now.