Loading arbitrary text into modules

guregu commented 1 month ago

Vaguely related to #581.

The WebAssembly ports can be run in environments where the filesystem is read-only or doesn't exist at all. To give users the ability to add code, I hacked in a $load_chars system predicate that sometimes relied on the looser module behavior.

// trealla-js api
pl.consultText("foo(bar).", "mymodule")

Runs a query like this:

mymodule:'$load_chars'("foo(bar).").

There's a quick fix:

module(mymodule), mymodule:'$load_chars'("foo(bar).").

But I was wondering if having a non-hacky way to load arbitrary text into a (potentially new) module might be useful? Even as an internal system predicate. Perhaps it could take the module as an argument instead of relying on the old implicit behavior.

There are a couple other places where I use this:

To let people write predicates in Javascript/Go, it loads a shim into user which looks kind of like: user:'$load_text'("my_cool_predicate(A) :- '$host_call'(my_cool_predicate(A)).") ← this could just be an assertz instead, but it would be nice to keep it non-dynamic.
I have a predicate http_consult(Module:URL) which loads code from a URL into the given (new) module, this is the main thing that broke from the module changes. To be honest, the usefulness of this one is questionable, I mostly just put it in for fun.

As for prior art, SWI's load_files/2 can take module(Name) as one of the options, but I'm not sure if you can give it strings of code to load. Pengines lets you load code as a query parameter, so it probably exists somewhere.

Here's the code I have for my internal predicate, it's pretty simple. I'm happy to just tack on a module name and create it here, but if we had a better way to do this it would be one step closer to move away from needing a wasm fork.

static bool fn_sys_load_chars_1(query *q)
{
    GET_FIRST_ARG(p1,any);
    size_t len;
    module *m = NULL;

    if (is_cstring(p1)) {
        const char *src = C_STR(q, p1);
        m = load_text(q->st.m, src, q->st.m->filename);
    } else if (scan_is_chars_list(q, p1, p1_ctx, true) > 0) {
        char *src = chars_list_to_string(q, p1, p1_ctx);
        m = load_text(q->st.m, src, q->st.m->filename);
        free(src);
    } else if (is_nil(p1)) {
        return false;
    } else
        return throw_error(q, p1, p1_ctx, "type_error", "chars");

    check_heap_error(m);
    return true;
}

infradig commented 1 month ago

I think module/1 should not create a module, rather just be a test for existence.

On Fri, 30 Aug 2024, 21:40 guregu, @.***> wrote:

Vaguely related to #581 https://github.com/trealla-prolog/trealla/issues/581.

The WebAssembly ports can be run in environments where the filesystem is read-only or doesn't exist at all. To give users the ability to add code, I hacked in a $load_chars system predicate that sometimes relied on the looser module behavior.

// trealla-js apipl.consultText("foo(bar).", "mymodule")

Runs a query like this:

mymodule:'$load_chars'("foo(bar).").

There's a quick fix:

module(mymodule), mymodule:'$load_chars'("foo(bar).").

But I was wondering if having a non-hacky way to load arbitrary text into a (potentially new) module might be useful? Even as an internal system predicate. Perhaps it could take the module as an argument instead of relying on the old implicit behavior.

There are a couple other places where I use this:

To let people write predicates in Javascript/Go, it loads a shim into user which looks kind of like: user:'$load_text'("my_cool_predicate(A) :- '$host_call'(my_cool_predicate(A)).") ← this could just be an assertz instead, but it would be nice to keep it non-dynamic.

I have a predicate http_consult(Module:URL) which loads code from a URL into the given (new) module, this is the main thing that broke from the module changes. To be honest, the usefulness of this one is questionable, I mostly just put it in for fun.

As for prior art, SWI's load_files/2 can take module(Name) as one of the options, but I'm not sure if you can give it strings of code to load. Pengines lets you load code as a query parameter, so it probably exists somewhere.

Here's the code I have for my internal predicate, it's pretty simple. I'm happy to just tack on a module name and create it here, but if we had a better way to do this it would be one step closer to move away from needing a wasm fork.

static bool fn_sys_load_chars_1(query q) { GET_FIRST_ARG(p1,any); size_t len; module m = NULL;

if (is_cstring(p1)) { const char src = C_STR(q, p1); m = load_text(q->st.m, src, q->st.m->filename); } else if (scan_is_chars_list(q, p1, p1_ctx, true) > 0) { char src = chars_list_to_string(q, p1, p1_ctx); m = load_text(q->st.m, src, q->st.m->filename); free(src); } else if (is_nil(p1)) { return false; } else return throw_error(q, p1, p1_ctx, "type_error", "chars");

check_heap_error(m); return true; }

— Reply to this email directly, view it on GitHub https://github.com/trealla-prolog/trealla/issues/582, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFNKSERD5WTP6AHFARFUP7LZUBK3JAVCNFSM6AAAAABNMJ5YROVHI2DSMVQWIX3LMV43ASLTON2WKOZSGQ4TMOJZHA2DIMI . You are receiving this because you are subscribed to this thread.Message ID: @.***>

guregu commented 1 month ago

I think module/1 should not create a module, rather just be a test for existence.

I agree that way makes more sense, the current behavior creates a module though. A different blessed way to create a module would solve my problem.

guregu commented 1 month ago

At least the module part that is. The text loading part is the other bit. One alternative would be a C API for it, working kind of like how consult(user) does. Having a predicate is nice though.

pmoura commented 1 month ago

I think module/1 should not create a module, rather just be a test for existence.

The standard predicate to test module existence, or enumerate existing modules by backtracking, is current_module/1. No reason to deviate from that. The module/1 predicate is used by some systems to set the "type-in" module, usually at the top-level.

infradig commented 1 month ago

Do you think module/1 should create modules then?

On Sat, 31 Aug 2024, 17:55 Paulo Moura, @.***> wrote:

I think module/1 should not create a module, rather just be a test for existence.

The standard predicate to test module existence, or enumerate existing modules by backtracking, is current_module/1. No reason to deviate from that. The module/1 predicate is used by some systems to set the "type-in" module, usually at the top-level.

— Reply to this email directly, view it on GitHub https://github.com/trealla-prolog/trealla/issues/582#issuecomment-2322819830, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFNKSEUZV5HZHNGR654C5QTZUFZHPAVCNFSM6AAAAABNMJ5YROVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMRSHAYTSOBTGA . You are receiving this because you commented.Message ID: @.***>

pmoura commented 1 month ago

Do you think module/1 should create modules then?

The module/1 predicate do create modules (when the argument is not an already existing module) as a side-effect when changing the default top-level module in SWI-Prolog and YAP. See e.g. https://www.swi-prolog.org/pldoc/doc_for?object=module/1

guregu commented 1 month ago

Everything looks good to me, thanks!

trealla-prolog / trealla

Loading arbitrary text into modules #582