search

retrogradeorbit / bootleg

Simple template processing command line tool to help build static websites
Eclipse Public License 2.0
255 stars 12 forks source link

Exception in thread "main" java.lang.NullPointerException #2

Closed chr15m closed 4 years ago

chr15m commented 4 years ago

I'm probably doing something dumb here.

site.clj

(-> (html "index.template.html") (enlive/at [:section#content] (enlive/content (markdown "README.md"))))

Exception

Exception in thread "main" java.lang.NullPointerException
    at bootleg.utils$convert_to.invokeStatic(utils.clj:153)
    at bootleg.utils$convert_to.invoke(utils.clj:150)
    at bootleg.namespaces$at__2990.invokeStatic(namespaces.clj:135)
    at bootleg.namespaces$at__2990.doInvoke(namespaces.clj:125)
    at clojure.lang.RestFn.applyTo(RestFn.java:139)
    at clojure.core$apply.invokeStatic(core.clj:665)
    at clojure.core$apply.invoke(core.clj:660)
    at sci.impl.interpreter$do_recur_BANG_.invokeStatic(interpreter.cljc:123)
    at sci.impl.interpreter$do_recur_BANG_.doInvoke(interpreter.cljc:121)
    at clojure.lang.RestFn.applyTo(RestFn.java:139)
    at clojure.core$apply.invokeStatic(core.clj:667)
    at clojure.core$apply.invoke(core.clj:660)
    at sci.impl.interpreter$apply_fn.invokeStatic(interpreter.cljc:132)
    at sci.impl.interpreter$apply_fn.invoke(interpreter.cljc:130)
    at sci.impl.interpreter$eval_call.invokeStatic(interpreter.cljc:306)
    at sci.impl.interpreter$eval_call.invoke(interpreter.cljc:271)
    at sci.impl.interpreter$interpret.invokeStatic(interpreter.cljc:320)
    at sci.impl.interpreter$interpret.invoke(interpreter.cljc:310)
    at sci.impl.interpreter$eval_do$fn__707.invoke(interpreter.cljc:62)
    at sci.impl.interpreter$eval_do.invokeStatic(interpreter.cljc:62)
    at sci.impl.interpreter$eval_do.invoke(interpreter.cljc:57)
    at sci.impl.interpreter$eval_string.invokeStatic(interpreter.cljc:370)
    at sci.impl.interpreter$eval_string.invoke(interpreter.cljc:352)
    at sci.core$eval_string.invokeStatic(core.cljc:35)
    at sci.core$eval_string.invoke(core.cljc:5)
    at bootleg.hiccup$process_hiccup_data.invokeStatic(hiccup.clj:54)
    at bootleg.hiccup$process_hiccup_data.invoke(hiccup.clj:22)
    at bootleg.hiccup$process_hiccup.invokeStatic(hiccup.clj:68)
    at bootleg.hiccup$process_hiccup.invoke(hiccup.clj:64)
    at bootleg.core$process.invokeStatic(core.clj:34)
    at bootleg.core$process.invoke(core.clj:32)
    at bootleg.core$_main.invokeStatic(core.clj:66)
    at bootleg.core$_main.doInvoke(core.clj:49)
    at clojure.lang.RestFn.applyTo(RestFn.java:137)
    at bootleg.core.main(Unknown Source)
retrogradeorbit commented 4 years ago

Recreating this I get a different exception:

Exception in thread "main" java.lang.IllegalArgumentException: Don't know how to create ISeq from: clojure.lang.Keyword

Which is coming from the markdown output being a hiccup-seq being passed to enlive/contentwhich expects a string.

You could use:

(-> (html "index.template.html")
    (enlive/at [:section#content] (enlive/content (markdown "README.md" :html))))

but you will get escaped html.

enlive/html-content is for inserting html content. But that is exploding for me with 

Exception in thread "main" java.lang.IllegalArgumentException: No matching clause: 
    at hickory.convert$hickory_to_hiccup.invokeStatic(convert.cljc:36)
    at hickory.convert$hickory_to_hiccup.invoke(convert.cljc:28)
    at clojure.core$map$fn__5851.invoke(core.clj:2753)
    at clojure.lang.LazySeq.sval(LazySeq.java:42)
    at clojure.lang.LazySeq.seq(LazySeq.java:51)
    at clojure.lang.Cons.next(Cons.java:39)
    at clojure.lang.RT.boundedLength(RT.java:1788)
    at clojure.lang.RestFn.applyTo(RestFn.java:130)
    at clojure.core$apply.invokeStatic(core.clj:669)
    at clojure.core$apply.invoke(core.clj:660)
    at hickory.convert$hickory_to_hiccup.invokeStatic(convert.cljc:52)
    at hickory.convert$hickory_to_hiccup.invoke(convert.cljc:28)
    at bootleg.utils$hickory__GT_hiccup.invokeStatic(utils.clj:51)

And that is probably a bug. Investigating...

retrogradeorbit commented 4 years ago

enlive's html-content produces a hickory structure that is missing a :type key that hickory's converter expects to find:

bootleg -e "(convert-to {:tag :blink, :attrs nil, :content [\"FOO\"]} :hiccup)"

Fails with NPE. while:

bootleg -e "(convert-to {:type :element :tag :blink, :attrs nil, :content [\"FOO\"]} :hiccup)"

succeeds.

This will require fixing. But you can work around this conversion incompatabilty at the moment by converting to hickory-seq first, then using enlive's conversion tools to convert back to html, like:

(-> (html "index.template.html")
    (convert-to :hickory-seq)
    (enlive/at [:section#content] (enlive/html-content "<blink>FOO</blink>"))
    net.cgrand.enlive-html/emit*
    (->> (apply str)))
chr15m commented 4 years ago

@retrogradeorbit will wait for the fix to html-content. Thanks!

Once that's working it would be good to document that as a common example. Most of the examples are command line things to demonstrate functionality and reveal what is going on under the hood, whereas a user wants to see practical examples they would use. Will create a new ticket for this.

retrogradeorbit commented 4 years ago

Fixed in v0.1.2

bootleg -e '(-> (html "index.template.html") (enlive/at [:section#content] (enlive/content (markdown "README.md" :html))))'
chr15m commented 4 years ago

@retrogradeorbit this is still segfaulting for me:

$ bootleg -e '(-> (html "index.template.html") (enlive/at [:section#content] (enlive/content (markdown "README.md" :html))))'
Exception in thread "main" java.lang.NullPointerException
    at bootleg.utils$convert_to.invokeStatic(utils.clj:166)
    at bootleg.utils$convert_to.invoke(utils.clj:163)
    at bootleg.namespaces$at.invokeStatic(namespaces.clj:26)
    at bootleg.namespaces$at.doInvoke(namespaces.clj:16)
    at clojure.lang.RestFn.applyTo(RestFn.java:139)
    at clojure.core$apply.invokeStatic(core.clj:665)
    at clojure.core$apply.invoke(core.clj:660)
    at sci.impl.interpreter$do_recur_BANG_.invokeStatic(interpreter.cljc:123)
    at sci.impl.interpreter$do_recur_BANG_.doInvoke(interpreter.cljc:121)
    at clojure.lang.RestFn.applyTo(RestFn.java:139)
    at clojure.core$apply.invokeStatic(core.clj:667)
    at clojure.core$apply.invoke(core.clj:660)
    at sci.impl.interpreter$apply_fn.invokeStatic(interpreter.cljc:132)
    at sci.impl.interpreter$apply_fn.invoke(interpreter.cljc:130)
    at sci.impl.interpreter$eval_call.invokeStatic(interpreter.cljc:306)
    at sci.impl.interpreter$eval_call.invoke(interpreter.cljc:271)
    at sci.impl.interpreter$interpret.invokeStatic(interpreter.cljc:320)
    at sci.impl.interpreter$interpret.invoke(interpreter.cljc:310)
    at sci.impl.interpreter$eval_do$fn__707.invoke(interpreter.cljc:62)
    at sci.impl.interpreter$eval_do.invokeStatic(interpreter.cljc:62)
    at sci.impl.interpreter$eval_do.invoke(interpreter.cljc:57)
    at sci.impl.interpreter$eval_string.invokeStatic(interpreter.cljc:370)
    at sci.impl.interpreter$eval_string.invoke(interpreter.cljc:352)
    at sci.core$eval_string.invokeStatic(core.cljc:35)
    at sci.core$eval_string.invoke(core.cljc:5)
    at bootleg.hiccup$process_hiccup_data.invokeStatic(hiccup.clj:54)
    at bootleg.hiccup$process_hiccup_data.invoke(hiccup.clj:22)
    at bootleg.core$_main.invokeStatic(core.clj:62)
    at bootleg.core$_main.doInvoke(core.clj:49)
    at clojure.lang.RestFn.applyTo(RestFn.java:137)
    at bootleg.core.main(Unknown Source)
$ bootleg --version
Version: 0.1.2
retrogradeorbit commented 4 years ago

Can you give me the md and html files you are using. Or minimal versions that recreate this problem for me to test with?

chr15m commented 4 years ago

Here you go: bad.zip

chr15m commented 4 years ago

Could there be a dep call missing from the build? at bootleg.utils$convert_to.invokeStatic(utils.clj:166) Maybe enlive has not been correctly inlined? So it would work on your local (using external library ref) but not on mine where it's not installed.

retrogradeorbit commented 4 years ago

at bootleg.utils$convert_to.invokeStatic(utils.clj:166)

It will probably be another enlive/hickory incompatibility. That's my hunch. Something in your markup is exposing it.

retrogradeorbit commented 4 years ago

OK. Thanks for the zipfile. You are actually getting hit by #7

retrogradeorbit commented 4 years ago

v0.1.3 released. Give that a try and close this if it fixes your NPE.

chr15m commented 4 years ago 
$ bootleg -v
Version: 0.1.3

Traceback (same files):

$ bootleg bad.clj 
Exception in thread "main" java.lang.Exception: java.lang.Character cannot be cast to clojure.lang.Named [at line 1, column 1]
    at sci.impl.utils$re_throw_with_location_of_node.invokeStatic(utils.cljc:64)
    at sci.impl.utils$re_throw_with_location_of_node.invoke(utils.cljc:55)
    at sci.impl.interpreter$eval_do$fn__707.invoke(interpreter.cljc:64)
    at sci.impl.interpreter$eval_do.invokeStatic(interpreter.cljc:62)
    at sci.impl.interpreter$eval_do.invoke(interpreter.cljc:57)
    at sci.impl.interpreter$eval_string.invokeStatic(interpreter.cljc:370)
    at sci.impl.interpreter$eval_string.invoke(interpreter.cljc:352)
    at sci.core$eval_string.invokeStatic(core.cljc:35)
    at sci.core$eval_string.invoke(core.cljc:5)
    at bootleg.hiccup$process_hiccup_data.invokeStatic(hiccup.clj:54)
    at bootleg.hiccup$process_hiccup_data.invoke(hiccup.clj:22)
    at bootleg.hiccup$process_hiccup.invokeStatic(hiccup.clj:68)
    at bootleg.hiccup$process_hiccup.invoke(hiccup.clj:64)
    at bootleg.core$process.invokeStatic(core.clj:34)
    at bootleg.core$process.invoke(core.clj:32)
    at bootleg.core$_main.invokeStatic(core.clj:66)
    at bootleg.core$_main.doInvoke(core.clj:49)
    at clojure.lang.RestFn.applyTo(RestFn.java:137)
    at bootleg.core.main(Unknown Source)
Caused by: java.lang.ClassCastException: java.lang.Character cannot be cast to clojure.lang.Named
    at clojure.core$name.invokeStatic(core.clj:1595)
    at clojure.core$name.invoke(core.clj:1589)
    at bootleg.utils$hiccup_seq__GT_hickory_seq.invokeStatic(utils.clj:47)
    at bootleg.utils$hiccup_seq__GT_hickory_seq.invoke(utils.clj:46)
    at bootleg.utils$convert_to.invokeStatic(utils.clj:142)
    at bootleg.utils$convert_to.invoke(utils.clj:139)
    at bootleg.namespaces$at.invokeStatic(namespaces.clj:18)
    at bootleg.namespaces$at.doInvoke(namespaces.clj:16)
    at clojure.lang.RestFn.applyTo(RestFn.java:139)
    at clojure.core$apply.invokeStatic(core.clj:665)
    at clojure.core$apply.invoke(core.clj:660)
    at sci.impl.interpreter$do_recur_BANG_.invokeStatic(interpreter.cljc:123)
    at sci.impl.interpreter$do_recur_BANG_.doInvoke(interpreter.cljc:121)
    at clojure.lang.RestFn.applyTo(RestFn.java:139)
    at clojure.core$apply.invokeStatic(core.clj:667)
    at clojure.core$apply.invoke(core.clj:660)
    at sci.impl.interpreter$apply_fn.invokeStatic(interpreter.cljc:132)
    at sci.impl.interpreter$apply_fn.invoke(interpreter.cljc:130)
    at sci.impl.interpreter$eval_call.invokeStatic(interpreter.cljc:306)
    at sci.impl.interpreter$eval_call.invoke(interpreter.cljc:271)
    at sci.impl.interpreter$interpret.invokeStatic(interpreter.cljc:320)
    at sci.impl.interpreter$interpret.invoke(interpreter.cljc:310)
    at sci.impl.interpreter$eval_do$fn__707.invoke(interpreter.cljc:62)
    ... 16 more
retrogradeorbit commented 4 years ago 
$ bootleg -v
Version: 0.1.4
$ bootleg bad.clj
<!DOCTYPE html>
<html>
  <head>
    <meta charset="utf-8">
    <meta content="width=device-width, initial-scale=1" name="viewport">
    <title>ABC</title>
  </head>
  <body>
    <main>
      <h1>ABC</h1>
      <section id="content"></section>
    </main>
  </body>
</html>
chr15m commented 4 years ago

Ok so if I use the html version of the markdown slurp it works great:

(-> (html "t.html") (enlive/at [:section#content] (enlive/html-content (markdown "r.md" :html))))

However if I try to pull it through enlive it segfaults (see below). From my perspective this is totally fine - I can just use the enlive/html-content. I don't think it's necessary to support every different user flow, just to make the most common ones that work simple to use.

Segfaulting version:

(-> (html "t.html") (enlive/at [:section#content] (enlive/content (markdown "r.md"))))
$ bootleg bad.clj 
Exception in thread "main" java.lang.IllegalArgumentException: Don't know how to create ISeq from: clojure.lang.Keyword
    at clojure.lang.RT.seqFrom(RT.java:553)
    at clojure.lang.RT.seq(RT.java:533)
    at clojure.core$seq__5387.invokeStatic(core.clj:137)
    at clojure.core$seq__5387.invoke(core.clj:137)
    at net.cgrand.enlive_html$flatten_nodes_coll$flat_STAR___2231.invoke(enlive_html.clj:228)
    at net.cgrand.enlive_html$flatten_nodes_coll$flat__2237$fn__2238.invoke(enlive_html.clj:233)
    at clojure.lang.LazySeq.sval(LazySeq.java:42)
    at clojure.lang.LazySeq.seq(LazySeq.java:51)
    at clojure.lang.RT.seq(RT.java:531)
    at clojure.core$seq__5387.invokeStatic(core.clj:137)
    at clojure.core$map$fn__5851.invoke(core.clj:2746)
    at clojure.lang.LazySeq.sval(LazySeq.java:42)
    at clojure.lang.LazySeq.seq(LazySeq.java:51)
    at clojure.lang.RT.seq(RT.java:531)
    at clojure.core$seq__5387.invokeStatic(core.clj:137)
    at clojure.core$dorun.invokeStatic(core.clj:3133)
    at clojure.core$doall.invokeStatic(core.clj:3148)
    at clojure.walk$walk.invokeStatic(walk.clj:47)
    at clojure.walk$postwalk.invokeStatic(walk.clj:53)
    at clojure.walk$postwalk.invoke(walk.clj:53)
    at clojure.core$partial$fn__5824.invoke(core.clj:2624)
    at clojure.walk$walk.invokeStatic(walk.clj:46)
    at clojure.walk$postwalk.invokeStatic(walk.clj:53)
    at clojure.walk$postwalk.invoke(walk.clj:53)
    at clojure.core$partial$fn__5824.invoke(core.clj:2624)
    at clojure.core$map$fn__5851.invoke(core.clj:2755)
    at clojure.lang.LazySeq.sval(LazySeq.java:42)
    at clojure.lang.LazySeq.seq(LazySeq.java:51)
    at clojure.lang.Cons.next(Cons.java:39)
    at clojure.lang.RT.next(RT.java:709)
    at clojure.core$next__5371.invokeStatic(core.clj:64)
    at clojure.core.protocols$fn__8144.invokeStatic(protocols.clj:169)
    at clojure.core.protocols$fn__8144.invoke(protocols.clj:124)
    at clojure.core.protocols$fn__8099$G__8094__8108.invoke(protocols.clj:19)
    at clojure.core.protocols$seq_reduce.invokeStatic(protocols.clj:31)
    at clojure.core.protocols$fn__8131.invokeStatic(protocols.clj:75)
    at clojure.core.protocols$fn__8131.invoke(protocols.clj:75)
    at clojure.core.protocols$fn__8073$G__8068__8086.invoke(protocols.clj:13)
    at clojure.core$reduce.invokeStatic(core.clj:6828)
    at clojure.core$into.invokeStatic(core.clj:6895)
    at clojure.walk$walk.invokeStatic(walk.clj:50)
    at clojure.walk$postwalk.invokeStatic(walk.clj:53)
    at clojure.walk$postwalk.invoke(walk.clj:53)
    at clojure.core$partial$fn__5824.invoke(core.clj:2624)
    at clojure.core$map$fn__5851.invoke(core.clj:2753)
    at clojure.lang.LazySeq.sval(LazySeq.java:42)
    at clojure.lang.LazySeq.seq(LazySeq.java:51)
    at clojure.lang.RT.seq(RT.java:531)
    at clojure.core$seq__5387.invokeStatic(core.clj:137)
    at clojure.core.protocols$seq_reduce.invokeStatic(protocols.clj:24)
    at clojure.core.protocols$fn__8131.invokeStatic(protocols.clj:75)
    at clojure.core.protocols$fn__8131.invoke(protocols.clj:75)
    at clojure.core.protocols$fn__8073$G__8068__8086.invoke(protocols.clj:13)
    at clojure.core$reduce.invokeStatic(core.clj:6828)
    at clojure.core$into.invokeStatic(core.clj:6895)
    at clojure.walk$walk.invokeStatic(walk.clj:50)
    at clojure.walk$postwalk.invokeStatic(walk.clj:53)
    at clojure.walk$postwalk.invoke(walk.clj:53)
    at clojure.core$partial$fn__5824.invoke(core.clj:2624)
    at clojure.walk$walk.invokeStatic(walk.clj:46)
    at clojure.walk$postwalk.invokeStatic(walk.clj:53)
    at clojure.walk$postwalk.invoke(walk.clj:53)
    at clojure.core$partial$fn__5824.invoke(core.clj:2624)
    at clojure.core$map$fn__5851.invoke(core.clj:2755)
    at clojure.lang.LazySeq.sval(LazySeq.java:42)
    at clojure.lang.LazySeq.seq(LazySeq.java:51)
    at clojure.lang.Cons.next(Cons.java:39)
    at clojure.lang.RT.next(RT.java:709)
    at clojure.core$next__5371.invokeStatic(core.clj:64)
    at clojure.core.protocols$fn__8144.invokeStatic(protocols.clj:169)
    at clojure.core.protocols$fn__8144.invoke(protocols.clj:124)
    at clojure.core.protocols$fn__8099$G__8094__8108.invoke(protocols.clj:19)
    at clojure.core.protocols$seq_reduce.invokeStatic(protocols.clj:31)
    at clojure.core.protocols$fn__8131.invokeStatic(protocols.clj:75)
    at clojure.core.protocols$fn__8131.invoke(protocols.clj:75)
    at clojure.core.protocols$fn__8073$G__8068__8086.invoke(protocols.clj:13)
    at clojure.core$reduce.invokeStatic(core.clj:6828)
    at clojure.core$into.invokeStatic(core.clj:6895)
    at clojure.walk$walk.invokeStatic(walk.clj:50)
    at clojure.walk$postwalk.invokeStatic(walk.clj:53)
    at clojure.walk$postwalk.invoke(walk.clj:53)
    at clojure.core$partial$fn__5824.invoke(core.clj:2624)
    at clojure.core$map$fn__5851.invoke(core.clj:2753)
    at clojure.lang.LazySeq.sval(LazySeq.java:42)
    at clojure.lang.LazySeq.seq(LazySeq.java:51)
    at clojure.lang.RT.seq(RT.java:531)
    at clojure.core$seq__5387.invokeStatic(core.clj:137)
    at clojure.core.protocols$seq_reduce.invokeStatic(protocols.clj:24)
    at clojure.core.protocols$fn__8131.invokeStatic(protocols.clj:75)
    at clojure.core.protocols$fn__8131.invoke(protocols.clj:75)
    at clojure.core.protocols$fn__8073$G__8068__8086.invoke(protocols.clj:13)
    at clojure.core$reduce.invokeStatic(core.clj:6828)
    at clojure.core$into.invokeStatic(core.clj:6895)
    at clojure.walk$walk.invokeStatic(walk.clj:50)
    at clojure.walk$postwalk.invokeStatic(walk.clj:53)
    at clojure.walk$postwalk.invoke(walk.clj:53)
    at clojure.core$partial$fn__5824.invoke(core.clj:2624)
    at clojure.walk$walk.invokeStatic(walk.clj:46)
    at clojure.walk$postwalk.invokeStatic(walk.clj:53)
    at clojure.walk$postwalk.invoke(walk.clj:53)
    at clojure.core$partial$fn__5824.invoke(core.clj:2624)
    at clojure.core$map$fn__5851.invoke(core.clj:2755)
    at clojure.lang.LazySeq.sval(LazySeq.java:42)
    at clojure.lang.LazySeq.seq(LazySeq.java:51)
    at clojure.lang.Cons.next(Cons.java:39)
    at clojure.lang.RT.next(RT.java:709)
    at clojure.core$next__5371.invokeStatic(core.clj:64)
    at clojure.core.protocols$fn__8144.invokeStatic(protocols.clj:169)
    at clojure.core.protocols$fn__8144.invoke(protocols.clj:124)
    at clojure.core.protocols$fn__8099$G__8094__8108.invoke(protocols.clj:19)
    at clojure.core.protocols$seq_reduce.invokeStatic(protocols.clj:31)
    at clojure.core.protocols$fn__8131.invokeStatic(protocols.clj:75)
    at clojure.core.protocols$fn__8131.invoke(protocols.clj:75)
    at clojure.core.protocols$fn__8073$G__8068__8086.invoke(protocols.clj:13)
    at clojure.core$reduce.invokeStatic(core.clj:6828)
    at clojure.core$into.invokeStatic(core.clj:6895)
    at clojure.walk$walk.invokeStatic(walk.clj:50)
    at clojure.walk$postwalk.invokeStatic(walk.clj:53)
    at clojure.walk$postwalk.invoke(walk.clj:53)
    at clojure.core$partial$fn__5824.invoke(core.clj:2624)
    at clojure.core$map$fn__5851.invoke(core.clj:2753)
    at clojure.lang.LazySeq.sval(LazySeq.java:42)
    at clojure.lang.LazySeq.seq(LazySeq.java:51)
    at clojure.lang.RT.seq(RT.java:531)
    at clojure.core$seq__5387.invokeStatic(core.clj:137)
    at clojure.core.protocols$seq_reduce.invokeStatic(protocols.clj:24)
    at clojure.core.protocols$fn__8131.invokeStatic(protocols.clj:75)
    at clojure.core.protocols$fn__8131.invoke(protocols.clj:75)
    at clojure.core.protocols$fn__8073$G__8068__8086.invoke(protocols.clj:13)
    at clojure.core$reduce.invokeStatic(core.clj:6828)
    at clojure.core$into.invokeStatic(core.clj:6895)
    at clojure.walk$walk.invokeStatic(walk.clj:50)
    at clojure.walk$postwalk.invokeStatic(walk.clj:53)
    at clojure.walk$postwalk.invoke(walk.clj:53)
    at clojure.core$partial$fn__5824.invoke(core.clj:2624)
    at clojure.walk$walk.invokeStatic(walk.clj:46)
    at clojure.walk$postwalk.invokeStatic(walk.clj:53)
    at clojure.walk$postwalk.invoke(walk.clj:53)
    at clojure.core$partial$fn__5824.invoke(core.clj:2624)
    at clojure.core$map$fn__5851.invoke(core.clj:2755)
    at clojure.lang.LazySeq.sval(LazySeq.java:42)
    at clojure.lang.LazySeq.seq(LazySeq.java:51)
    at clojure.lang.Cons.next(Cons.java:39)
    at clojure.lang.RT.next(RT.java:709)
    at clojure.core$next__5371.invokeStatic(core.clj:64)
    at clojure.core.protocols$fn__8144.invokeStatic(protocols.clj:169)
    at clojure.core.protocols$fn__8144.invoke(protocols.clj:124)
    at clojure.core.protocols$fn__8099$G__8094__8108.invoke(protocols.clj:19)
    at clojure.core.protocols$seq_reduce.invokeStatic(protocols.clj:31)
    at clojure.core.protocols$fn__8131.invokeStatic(protocols.clj:75)
    at clojure.core.protocols$fn__8131.invoke(protocols.clj:75)
    at clojure.core.protocols$fn__8073$G__8068__8086.invoke(protocols.clj:13)
    at clojure.core$reduce.invokeStatic(core.clj:6828)
    at clojure.core$into.invokeStatic(core.clj:6895)
    at clojure.walk$walk.invokeStatic(walk.clj:50)
    at clojure.walk$postwalk.invokeStatic(walk.clj:53)
    at clojure.walk$postwalk.invoke(walk.clj:53)
    at bootleg.utils$munge_hickory_tags.invokeStatic(utils.clj:56)
    at bootleg.utils$munge_hickory_tags.invoke(utils.clj:55)
    at bootleg.utils$hickory__GT_hiccup.invokeStatic(utils.clj:106)
    at bootleg.utils$hickory__GT_hiccup.invoke(utils.clj:103)
    at clojure.core$map$fn__5851.invoke(core.clj:2753)
    at clojure.lang.LazySeq.sval(LazySeq.java:42)
    at clojure.lang.LazySeq.seq(LazySeq.java:51)
    at clojure.lang.RT.seq(RT.java:531)
    at clojure.core$seq__5387.invokeStatic(core.clj:137)
    at clojure.core$map$fn__5851.invoke(core.clj:2746)
    at clojure.lang.LazySeq.sval(LazySeq.java:42)
    at clojure.lang.LazySeq.seq(LazySeq.java:51)
    at clojure.lang.RT.seq(RT.java:531)
    at clojure.core$seq__5387.invokeStatic(core.clj:137)
    at clojure.core$dorun.invokeStatic(core.clj:3133)
    at clojure.core$doall.invokeStatic(core.clj:3148)
    at clojure.walk$walk.invokeStatic(walk.clj:47)
    at clojure.walk$postwalk.invokeStatic(walk.clj:53)
    at clojure.walk$postwalk.invoke(walk.clj:53)
    at bootleg.hiccup$process_hiccup_data.invokeStatic(hiccup.clj:62)
    at bootleg.hiccup$process_hiccup_data.invoke(hiccup.clj:22)
    at bootleg.hiccup$process_hiccup.invokeStatic(hiccup.clj:68)
    at bootleg.hiccup$process_hiccup.invoke(hiccup.clj:64)
    at bootleg.core$process.invokeStatic(core.clj:34)
    at bootleg.core$process.invoke(core.clj:32)
    at bootleg.core$_main.invokeStatic(core.clj:66)
    at bootleg.core$_main.doInvoke(core.clj:49)
    at clojure.lang.RestFn.applyTo(RestFn.java:137)
    at bootleg.core.main(Unknown Source)
retrogradeorbit commented 4 years ago

What is happening here is (markdown "r.md") returns a hiccup-seq by default. enlive knows nothing about hiccup. And html-content is expecting arguments of strings or hickory structures.

content doesn't work because it flattens the nodes? https://github.com/cgrand/enlive/blob/master/src/net/cgrand/enlive_html.clj#L641

html-content applies snippet processing across each argument: https://github.com/cgrand/enlive/blob/master/src/net/cgrand/enlive_html.clj#L658

To return the markdown as html for embedding use the :html option. ie. (markdown "r.md" :html)

content may work if you pass in :hickory-seq too... maybe. Not sure if enlive will accept a sequence. Might expect a series of args of hickory. If :hickory-seq doesn't work you could try a (apply content (markdown "r.md" :hickory-seq))

Check the enlive documentation for more info.

There are always going to be these conversions when mixing different templating frameworks and processors.

I would like better automatic conversions and error reporting. But I still have to think about how to implement them. Use :html for now.

retrogradeorbit commented 4 years ago 
$ cat bad.clj
(-> (html "t.html")
    (enlive/at [:section#content] (enlive/html-content (markdown "r.md" :html))))
$ bootleg bad.clj
<!DOCTYPE html>
<html>
  <head>
    <meta charset="utf-8">
    <meta content="width=device-width, initial-scale=1" name="viewport">
    <title>ABC</title>
  </head>
  <body>
    <main>
      <h1>ABC</h1>
      <section id="content"><p>HELLO.</p></section>
    </main>
  </body>
</html>
$ cat bad.clj
(-> (html "t.html")
    (enlive/at [:section#content] (apply enlive/content (markdown "r.md" :hickory-seq))))
$ bootleg bad.clj
<!DOCTYPE html>
<html>
  <head>
    <meta charset="utf-8">
    <meta content="width=device-width, initial-scale=1" name="viewport">
    <title>ABC</title>
  </head>
  <body>
    <main>
      <h1>ABC</h1>
      <section id="content"><p>HELLO.</p></section>
    </main>
  </body>
</html>
retrogradeorbit commented 4 years ago

btw, you can see what something returns like:

$ bootleg -d -e '(markdown "r.md")'
([:p {} "HELLO."])
$ bootleg -d -e '(markdown "r.md" :hickory)'
{:type :element, :attrs nil, :tag :p, :content ["HELLO."]}
$ bootleg -d -e '(markdown "r.md" :hickory-seq)'
({:type :element, :attrs nil, :tag :p, :content ["HELLO."]})
chr15m commented 4 years ago

I would just make clear in the documentation that if you want to push it through enlive it needs to come out as hickory.

retrogradeorbit commented 4 years ago

Will add that to the docs.

retrogradeorbit commented 4 years ago

There's additional complexity here too. Because this test markdown is only one line. But if it was many lines, you'd get a sequence of hickory. If you returned :hickory you'd only get the last form. Which is technically correct, but you know someone is going to be really confused by that at some point.

eg

$ cat r.md
# HEADING
This is some markdown with multiple forms

ok?
$ bootleg -d -e '(markdown "r.md" :hickory)'
{:type :element, :attrs nil, :tag :p, :content ["ok?"]}
$ cat bad.clj
(-> (html "t.html")
    (enlive/at [:section#content] (enlive/content (markdown "r.md" :hickory))))
$ bootleg bad.clj
<!DOCTYPE html>
<html>
  <head>
    <meta charset="utf-8">
    <meta content="width=device-width, initial-scale=1" name="viewport">
    <title>ABC</title>
  </head>
  <body>
    <main>
      <h1>ABC</h1>
      <section id="content"><p>ok?</p></section>
    </main>
  </body>
</html>

That is going to confuse someone at some point

chr15m commented 4 years ago

Yes it will.

retrogradeorbit commented 4 years ago

My present thought is to print to stderr a big red "WARNING: markup forms lost!" whenever a lossy conversion is done and data is lost. Your thoughts on this? Any better ideas?

chr15m commented 4 years ago

Could you make markdown just always return a seq?

As a user building a static site I don't want to have to care about these details. I generally want everything to work as strings. So have markdown come in and turn into an HTML string and get inserted into the document.

Because you've included the possibility to get the hiccup form that introduces a second way of working which is with Clojure data structures. Again though I don't want to know all the details about what particular kind of structure it is - I just want to plug stuff together like lego bricks.

I would say wherever possible don't expose gnarly edges to the user and just do what you think they would expect in the most common use-case, then give them options where they can get super technical if they want. Sensible defaults.

Sorry, I know that introduces a lot of complexity under the hood but I'm just speaking as a dumb user who simply wants to build a static site quickly and efficiently without learning too much new stuff.

chr15m commented 4 years ago

I guess what I want is wrapped versions of the enlive fns which take Hiccup not Hickory. The latter format is confusing to me as an end user.

chr15m commented 4 years ago

Or maybe something like enliven would be sufficient:

(-> (html "t.html")
    (enlive/at [:section#content] (enlive/content (enliven (markdown "r.md")))))

As long as there is some simple rule I need to follow when working with enlive.

chr15m commented 4 years ago

I guess what I am saying is :hickory-seq and :hickory is one option too many. Just make it :hickory and always return a seq, even if there is only one element.

retrogradeorbit commented 4 years ago

Could you make markdown just always return a seq?

As a user building a static site I don't want to have to care about these details. I generally want everything to work as strings. So have markdown come in and turn into an HTML string and get inserted into the document.

I know what you are saying, but this is not completely true. Initially you wanted hiccup. Now strings.

Because you've included the possibility to get the hiccup form that introduces a second way of working which is with Clojure data structures. Again though I don't want to know all the details about what particular kind of structure it is - I just want to plug stuff together like lego bricks.

It needs a default. Is that hiccup? or strings?

I would say wherever possible don't expose gnarly edges to the user and just do what you think they would expect in the most common use-case, then give them options where they can get super technical if they want. Sensible defaults.

That's what I've done. I've picked a sequence of hiccup as the default. So you can go [:div (markdown "mine.md")]. If strings were default, this would not work. You'd get a big encoded html string as content.

Sorry, I know that introduces a lot of complexity under the hood but I'm just speaking as a dumb user who simply wants to build a static site quickly and efficiently without learning too much new stuff.

It does. It's actually complecting things together. You are asking for an easy, complex solution.

retrogradeorbit commented 4 years ago

I guess what I want is wrapped versions of the enlive fns which take Hiccup not Hickory. The latter format is confusing to me as an end user.

I agree with this and I plan to do it at some point. This just requires writing auto coercing binding fuctions that wrap every enlive call. It's just a lot of work. I've done it for the at function (which is why you can pass hiccup to at and it doesnt blow up) but that is all. This will be done in time.

retrogradeorbit commented 4 years ago

I guess what I am saying is :hickory-seq and :hickory is one option too many. Just make it :hickory and always return a seq, even if there is only one element.

I also thought about that naming. And while that will help the users who have never used hickory before, it will totally confuse those who have used hickory. As they try and use their standard (maybe preexisting) hickory code and it all blows up, because it's not actually hickory. It's a sequence of hickory.

It all stems from the underlying problem that both hiccup and hickory were never intended to hold template snippets. They were intended to contain a single, well formed HTML node.

chr15m commented 4 years ago

You are asking for an easy, complex solution.

So a typical user in other words. ;)

It needs a default. Is that hiccup?

Yep I think so.

What you have working already is perfectly sufficient already. I can merge HTML, Markdown, and JSON into a nice Hiccup form. :+1:

retrogradeorbit commented 4 years ago

I'll close this as it resolves the NPE exception (which were type conversion bugs). Opened three enhancement issues to improve the situation at a later date.