COMPILE-APPLICATION and macro exposure

[mmontone]

I'm working on an application that uses JSCL, and I'm using COMPILE-APPLICATION after JSCL:BOOTSTRAP, to compile it. It compiles and works fine, but there's a problem, macros are not exposed to the runtime, and cannot be accessed from the REPL. I think the environment needs to be dumped for that, as I understand from this comment here:

https://github.com/jscl-project/jscl/blob/29885f79a236f71d38e1fe2ea73c3158a8324291/src/compiler/compiler.lisp#L518

We record the macro definitions as lists during the bootstrap. Once everything is compiled, we want to dump the whole global environment to the output file to reproduce it in the run-time.

I tried calling DUMP-GLOBAL-ENVIRONMENT after COMPILE-APPLICATION, but it fails. Am I right about this? Do you have any idea how I can fix this and continue compiling with this approach?

[mmontone]

In the mean time, I've managed to implement a bundle-application function:

(defun bundle-application (files output-pathname &key verbose)
  (let ((*features* (list* :jscl :jscl-xc *features*))
        (*package* (find-package "JSCL"))
        (*default-pathname-defaults* jscl::*base-directory*))
    (setq jscl::*environment* (jscl::make-lexenv))
    (jscl::with-compilation-environment
      (with-open-file (out output-pathname
                           :direction :output
                           :if-exists :supersede)
        (format out "(function(){~%")
        (format out "'use strict';~%")
        (write-string (jscl::read-whole-file (jscl::source-pathname "prelude.js")) out)
        (jscl::do-source input :target
          (jscl::!compile-file input out :print verbose))

    ;; NOTE: This file must be compiled after the global
        ;; environment. Because some web worker code may do some
        ;; blocking, like starting a REPL, we need to ensure that
        ;; *environment* and other critical special variables are
        ;; initialized before we do this.
        (jscl::!compile-file "src/toplevel.lisp" out :print verbose)

    ;; Compile application files

    (dolist (file files)
      (jscl::!compile-file file out :print nil))

    (write-string (jscl::compile-toplevel '(cl:in-package :cl-user)))

        (jscl::dump-global-environment out)        

        (format out "})();~%")))

    (jscl::report-undefined-functions)))

Takes the list of application files and outputs a self-contained js file, with jscl and the application. One of the caveats is that I'm having to wrap in-package defintions with (eval-when (:compile-toplevel :load-toplevel :execute)) in application files.

This bundling method allows me to work with files, packages and expose macros in my JSCL application.

Reference: https://codeberg.org/mmontone/interactive-lang-tools/src/branch/master/backends/jscl

[VitoVan]

Hi @mmontone , thanks for your work, I managed to use your work to expose custom my functions.

I want to post a minimum working example here for anyone else who might also need this, since I rummaged around for quite a lot of hours (because of my dumbness).

1. create a file:

main.lisp

(eval-when (:compile-toplevel :load-toplevel :execute)
  (cl:defpackage :c
    (:use cl)
    (:export
     :hello))
  (in-package :c))

(defun hello (name)
    (#j:console:log "Hello" name))

(in-package :cl-user)

2. load jscl and bootstrap it

sbcl --load jscl.lisp --eval '(jscl:bootstrap)'

3. copy and paste the bundle-application code above

...
...
...
BUNDLE-APPLICATION
*

4. bundle main.lisp

(bundle-application (list "main.lisp") "main.js")

5. in your index.html, replace jscl.js with main.js

    <script>
      var jqconsole = $("#console").jqconsole("", "");
    </script>
-     <script src="jscl.js" type="text/javascript"></script>
+     <script src="main.js" type="text/javascript"></script>
    <script src="jscl-web.js" type="text/javascript"></script>

6. open your browser, refresh and type this

[mmontone]

It is not because of your dumbness. It is not clear how to do this with JSCL, and you need to understand the compiler internals. I came up with this but still I think some things still didn't work (I don't remember the details now). I'm glad that this worked for you, if I understand correctly. Adding a demo to the repository would be nice, I do that in my projects too, djula and cl-forms for example. That's valuable.

[mmontone]

Also I remember there was a problem with packages, like in-package not working some times, or not as I would expect.

[VitoVan]

It is not because of your dumbness. It is not clear how to do this with JSCL, and you need to understand the compiler internals. I came up with this but still I think some things still didn't work (I don't remember the details now). I'm glad that this worked for you, if I understand correctly. Adding a demo to the repository would be nice, I do that in my projects too, djula and cl-forms for example. That's valuable.

Thank you Mariano, I'm thinking about adding a Wiki page like "JSCL HOW-TO" to include some newbie questions, like:

• how to alert?
• how to access JS object in JSCL?
• how to access JSCL function in JS?
• how to add JSCL function to DOM event listener as callback?
• ......

I haven't figured all of them out yet.

[mmontone]

Those would be great to have IMO; some docs live in issues at the moment and are difficult to find. Also include this temporary recipe for compiling application code to the wiki.

[VitoVan]

Those would be great to have IMO; some docs live in issues at the moment and are difficult to find. Also include this temporary recipe for compiling application code to the wiki.

WIKI: https://github.com/jscl-project/jscl/wiki/JSCL-HOW-TO

Not finished, will continue to write once I got time.