nbNewCode + Generate javascript from Nim

[HugoGranstrom]

As the scope of this grew beyond just nbFile it might as well get its own issue. Continuation on discussion in #34.

ping @ajusa saw you had an interest in something like this in the #28. Feel free to weigh in. If we could use client-side karax using this as well it would be really cool as we would be able to create interactive elements without having to leave Nim-land.

This is the API suggestion thus far:

script = nbNewCode:
  var myVar = 0

script.addCode:
  inc myVar

script.addToDocAsJs

But there are still things lacking like being able to capture variables from the compiled to the javascript version. We will essentially have to inline the values in the code and my current best idea is something like:

var nimVar = 1
script.addCode(captures=[nimVar]):
    echo nimVar

# Turns above into:

var nimVar = 1
let serialized_nimVar = nimVar.toJson()
let code = fmt"echo fromJson({serialized_nimVar})" 

We serialize the variable on the C-side and do some sort of string interpolation to replace all occurrences of the variable by fromJson("json value of the variable"). And for this, I think we will eventually need a preprocessing macro to for example normalize all variable names and check that the user doesn't do anything like var nimVar = nimVar in which case we shouldn't replace it with the serialized variable anymore. But this can be added at the very end so let's not focus on it right now.

[pietroppeter]

being able to capture variables from the compiled to the javascript version

can you expand a little bit more the example you give? I am still not sure what the use case would be and what the code is supposed to do.

[HugoGranstrom]

can you expand a little bit more the example you give? I am still not sure what the use case would be and what the code is supposed to do.

Let's say you want a block that is a counter (button + label) that increases every time you press it. In psuedo-HTML it would be:

<p id="label">0</p>
<button id="btn">Click me</button>

<script>
const btn = document.getElementById("btn")
const label = document.getElementById("label")
let count = 0
btn.addEventlistener("click", () => {
  count++
  label.innerHtml = count
})
</script>

Now if we want to implement this kind of block in nimib we would first use nbRawOutput to generate the <p> and <button> tags and then nbJsCode to generate the <script> tag:

template counterButton:
  nbRawOutput: """
<p id="label">0</p>
<button id="btn">Click me</button>
"""
  nbJsCode:
    import std/dom
    var counter = 0
    # I don't know how std/dom work so this is very psuedo:
    var btn = document.getElementById("btn")
    var label = document.getElementById("label")
    btn.addEventListener("click",
        proc() =
        counter += 1
        label.innerHtml = $counter
    )       

So far everything works just fine. But what if I create two of these buttons?

counterButton()
counterButton()

Because the button and label tags in both blocks have the same ids we will be in trouble. So then we create unique ids for each one. For example by passing in the id as a string:

template counterButton(id: string):
  nbRawOutput: """
<p id=fmt"label{id}">0</p>
<button id=fmt"btn{id}">Click me</button>
"""
  nbJsCode:
    import std/dom
    var counter = 0
    # I don't know how std/dom work so this is very psuedo:
    var btn = document.getElementById(fmt"btn{id}")
    var label = document.getElementById(fmt"label{id}")
    btn.addEventListener("click",
        proc() =
            counter += 1
            label.innerHtml = $counter
    )

This is where we get into trouble though! Because how can the code in the nbJsCode block get access to our id variable which is defined in the C-code? Do you see the problem? The code in nbJsCode doesn't know anything about the code around it when it is compiled in a separate file. What we would need here is to be able to capture the value of id from the C-code and then when we reach the nbJsCode (the code of which is never run in C-land) insert the current value of id into all places it is used.

Please tell me if anything was unclear 😄

[pietroppeter]

perfectly clear thanks! Now I understand why this can be an issue in some cases, especially when we want to make reusable widgets. Let's say that this is the case where you want the Js code to change according to some context.

I would say that the better solution for this could be to abandon the untyped version of nbJSCode template and directly replace the value of id in the code string, something like:

template counterButton(id: string):
  nbRawOutput: fmt"""
<p id=fmt"label{id}">0</p>
<button id=fmt"btn{id}">Click me</button>
"""
  nbJsCode: fmt"""
    import std/dom
    var counter = 0
    # I don't know how std/dom work so this is very psuedo:
    var btn = document.getElementById(fmt"btn{id}")
    var label = document.getElementById(fmt"label{id}")
    btn.addEventListener("click",
        proc() =
            counter += 1
             #label.innerHtml = $counter
    )
"""

here you lose the advantage of the untyped version, but not being executed the only change is that syntax is checked and highlighted (and if this is a library element it is not a big deal to specify the string instead of the code).

Alternatively, a more hacky solution could be to call the untyped version and then call something like nb.blk.code.replace("{id}", id) (or a more refined version of this that does at runtime what fmt does at compile time).

[HugoGranstrom]

Writing Nim code in a string is nearly as bad as writing javascript in a string to me 😅 Plus simple string interpolation won't work for objects. If we define a type like this:

type
  Context = object
    field: float
    count: int

Then the string version of it isn't valid Nim-code for creating it:

echo $Context()
# (field: 0.0, count: 0)

echo Context().repr
# [field = 0.0,
# count = 0]

So we will have to do some sort of serialization to get it into a suitable string format. And sure we could do fmt"fromJson({ctx.toJson})" to handle those cases but then it starts to become quite verbose and could equally well be automated in a macro IMO to keep the code non-stringy. Losing autocomplete is quite devastating for me when writing code 🤣

But the same underlying structure could be used for the string-version and the untyped-version as the untyped code will become a string sooner or later either way. So I'm insisting on having both of them available although we could start with just the string version to get going. I'll happily write the required macro so don't worry about that 😜

I guess I'll have to add a hlNim to nimiBoost now, how ironic 🤣

[HugoGranstrom]

A working Proof of concept:

template nbJsCode*(code: string) =
  writeFile("code.nim", code)
  discard execShellCmd("nim js -d:danger -o:code.js code.nim")
  let jscode = readFile("code.js")
  nbRawOutput: "<script>\n" & jscode & "\n</script>"

nbText: "Counter POC"

nbRawOutput: """
<p id="label">0</p>
<button id="btn">Click me</button>
"""

nbJsCode: """
import std/dom
echo "Hello world!"
var label = getElementById("label".cstring)
var button = getElementById("btn".cstring)
var counter: int = 0
button.addEventListener("click",
  proc (ev: Event) =
    counter += 1
    label.innerHtml = ($counter).cstring
)
"""

The next step I guess is to implement a proper "system" á la nbNewCode to base nbJsCode on. Do you have any preferences on how the script type should be structured?

[HugoGranstrom]

The name nbCodeScript and fields are still up for discussion but this went rather smoothly. I'm surprised how simple this was tbh, expected it to be more work. I still have left to make the files in a temp folder but other than that it is basically functioning 🤯

type
  NbCodeScript* = ref object
    code*: string

template nbNewCode*(codeString: string): NbCodeScript =
  NbCodeScript(code: codeString)

template addCode*(script: NbCodeScript, codeString: string) =
  script.code &= "\n" & codeString

template addToDocAsJs*(script: NbCodeScript) =
  writeFile("code.nim", script.code)
  discard execShellCmd("nim js -d:danger -o:code.js code.nim")
  let jscode = readFile("code.js")
  nbRawOutput: "<script>\n" & jscode & "\n</script>"

template nbJsCode*(code: string) =
  let script = nbNewCode(code)
  script.addToDocAsJs

[HugoGranstrom]

I've finally gotten the untyped version to work (a lot thanks to @Vindaar's macro magic). He also had an idea, but let's start with the problem. As it stands I need the macro to be untyped because otherwise, we will get undeclared identifier errors if we use "global" javascript variables. So the string and untyped variants can't have the same name nbNewCode because it would require the compiler to type-check the code block to decide which one to choose. So we could use different names, for example, nbNewCodeStr and nbNewCode for the string and untyped versions. Vindaars idea then was, we are in a macro and we can the whatever we want so why not check if we get a string as the input to our untyped version and if then dispatch it to nbNewCodeStr. This way we can use nbNewCode for both strings and untyped code-blocks seamlessly.

I modified this idea a bit into this:

template nbNewCode*(args: varargs[untyped]): untyped =
  let code = nimToJsString(args)
  NbCodeScript(code: code)

where nimToJsString just returns the string if it was a string that was passed in and otherwise it runs through the complex macro machinery to parse a code block into a string.

And here is a working example which prints out 4 in the javascript console.

let k = 3
let script = nbNewCode(k):
  let a = 1
  let b = 2
  let c = a + k
  echo c

script.addToDocAsJs

What do you think about this approach of unifying nbNewCode into a single macro to avoid having two different names for the different options?

[pietroppeter]

Amazing! Looks great!

[HugoGranstrom]

Nice, I'll do some cleanup and create a PR later tonight then 👍 And then It's just a matter of trying it out and finding the bugs!

[pietroppeter]

closed by #88 (different api name, not nbNewCode but nbCodeToJs)