search

brandonbloom / fipp

Fast Idiomatic Pretty Printer for Clojure
525 stars 44 forks source link

fipp.edn/pprint is slow #47

Closed tiye closed 7 years ago

tiye commented 7 years ago

My code:

(defn persist! [db]
  (let [start-time (now!)]
    (fs.writeFileSync
     (:storage-key schema/configs)
     (with-out-str (fipp/pprint (-> db (assoc :sessions {}) (assoc :saved-files {})))))
    (println (str "Took " (- (now!) start-time) "ms to wrote!"))))

And the result are <900 lines, 94k+ characters. Took 2500ms to write.

I tried to format with pr-str, and is <500ms estimated from experience.

Is there any room to make this function faster?

Using this file as example: https://github.com/mvc-works/coworkflow/blob/master/coir.edn

brandonbloom commented 7 years ago

1) with-out-str and pr-str will realize the entire output in memory. Try redirecting out to your file.

2) It's well expected that fipp will be slower than pr. Let me know if there are particular data structures or something that are pathological, but 3 to 5 times slower is not unexpected.

3) fipp is plenty fast enough for interactive use, but I'm willing to entertain patches if you can make reliable/measured improvements. I probably won't accept any churn unless the gains are measurable and the changes low risk. If you do come up with something, feel free to open a PR (however unfinished) to discuss it.

tiye commented 7 years ago

It's sometimes 20 times slower compared to pr-str.

Since it's in Node.js , could you point me an example piping fipp.edn/print to a stream?

brandonbloom commented 7 years ago

Redirecting out is unrelated to fipp. I have no special resources for that. You'll have to conduct your own research.

I don't use fipp aggressively from JS runtimes. Profiling results and potential improvement patches welcome for consideration.

Sent from my iPhone

On Jul 19, 2017, at 5:58 PM, jiyinyiyong notifications@github.com wrote:

It's sometimes 20 times slower compared to pr-str.

Since it's in Node.js , could you point me an example piping fipp.edn/print to a stream?

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub, or mute the thread.

tiye commented 7 years ago

Dark side of ClojureScript, I don't read Writer of Java, then I can't understand *out* in Node.js .

brandonbloom commented 7 years ago

Sorry. This is the wrong forum for that kind of help. Maybe somebody on IRC, Slack, or Stack Overflow can provide some advice.