ability to run lua program line-by-line / pause execution and return to caller

[jvantuyl]

I'm trying to write a distributed consensus based sandbox. This would be absolutely perfect if I could somehow step through the program and do consensus on the VM state. Unfortunately, I can't figure out any way to pause execution. Ideally, the requested feature would work something like:

InterpState = luerl:call_func(Code, {enable_breakpoints, [{explicit, true}, {reductions, 100}])
{Status, State} = luerl:run(InterpState)

Or something like the above. Basically some way to say "run like normal but break out and give me a status code and a paused interpreter state (that's opaque but can be serialized) when something explicitly triggers a breakpoint or every 100 'reductions'".

I recognize this is a big ask and I suspect the answer will be "not likely to happen", but I thought I'd at least ask. ^_^

[linux2647]

I have a similar need, and it seems like the tracing functions could give you what you need. Instead of every 100 reductions, it calls the tracing function (and pauses the VM execution) on every line, function call, and return. I haven't used it yet, but it seems promising. See this commit for more (I don't see any other documentation on it). It was also included in the v1.0 release

[rvirding]

Yes @jvantuyl, one way to do this would be to use the tracing function. It is as @linux2647 mentioned call every line, function and return with the traced op and the current luerl state. It returns the potentially updated luerl state. Note be very careful if you modify the state as this is what is then used in the interpreter. There is trace_data field in the luerl state which is reserved for the tracer and is not used by anything else. Here is a very simple example file (unfortunately not in the in the commit, sorry about that) trace_test.erl.zip. If you can't read it I will just paste the content.

[jvantuyl]

Thanks for this.

Hmmmm... I'm not entirely sure if tracing will work. Ideally I'd have something more like a continuation.

Sorry if my use-case isn't super clear. What I'm trying to do is build a sandbox that can be run on multiple hosts and then use consensus to provide fault-tolerance. I figured I could just run up to the point that it does some kind of IO and that would be my sync point between the various nodes.

What I need is essentially:

• A way to start an interpreter from scratch (this isn't a problem today).
• A way to have a call in the interpreter bail out and give me a ball of state.
• A way to resume that ball of state providing a return value.
• A way to serialize that state and send it between nodes.

I'm not 100% sure if the last one works with the current state representation. I see that there are captured functions in there. I'm not exactly clear how that would serialize (but I am interested in finding out).

By way of example, let's say I wanted to have a sandboxed program that runs the following script:

function heartbeat ()
  while true
    -- sleep 5 seconds
    sandbox.sleep(5000) 
    -- log a heartbeat
    sandbox.print("heartbeat")
  end
end

Ideally, it'd look like this from the Erlang side:

state0 = luerl:init().
%% agree on state via consensus
%% agree on startup via consensus
{state1, info_about_escape}  = luerl:do_until_escape(heartbeat_script).
%% agree on new state via consensus
%% recognize that it's a sleep
%% agree on when to continue execution and return value of nil from sleep
{state2, info_about_escape} = luerl:do_until_escape(nil, state1).
%% recognize that it's a print
%% agree on what it printed and return nil value from print
%% loop on this, doing consensus until the program is done

And presumably the sandbox library code would look something like this:

sandbox = {}

-- the serialization below is just for example sake
-- if I was implementing something real, there'd be more sophisticated escaping

function sandbox.sleep(time)
  return magic_luerl_internals.escape_hatch("sleep:" .. type(time) .. "=" .. time)
end

function sandbox.print(text)
  escaped_text = escape_text_somehow(text)
  return magic_luerl_internals.escape_hatch("print:" .. type(escaped_text) .. "=" .. escaped_text)
end

The idea is to be able to break out at any point, serialize what I've got, send it somewhere else, and then be able to resume it.

Tracing seems to be kind of inside-out from what I need. I could probably use it to capture state, but I'm not entirely sure how I'd take that serialized state and send it somewhere else.

I know that this is probably not an expected use-case. Given Erlang's functionalness and Lua being such a widely understood language, it would be really great if I could make this work. I've already tried this with Python and it didn't go so well.

If this isn't something that you want to implement, could I possibly spike something out and see what you think of it? Luerl has solved all of the hard parts of what I need to do. I just need the right control surface.

[jvantuyl]

I'm still tinkering with this but there doesn't appear to be much value in keeping this issue open.

I'll close it for now and reopen if I get anywhere. Thanks for your attention!