dmolina / DaemonMode.jl

Client-Daemon workflow to run faster scripts in Julia
MIT License
272 stars 16 forks source link

Add a REST HTTP API #5

Open NightMachinery opened 4 years ago

NightMachinery commented 4 years ago

This should also solve the remote execution problem, and it'll also make everything much faster, as currently you need to julia --startup-file=no -e 'using DaemonMode; runargs()' program.jl <arguments> which has huge overhead compared to an HTTP request:

❯ time julia --startup-file=no --print "pi"
π
julia --startup-file=no --print "pi"  0.30s user 1.22s system 25% cpu 6.014 total

❯ time (print -nr -- '{"cmd":"echo hi from zsh","stdin":"\n","verbose":"0"}' | curl --fail --silent --location --header 'Content-Type: application/json' --request POST --data @- http://127.0.0.1:7230/zsh/
) # my own version of this package for the zsh language
hi from zsh
( print -nr -- '{"cmd":"echo hi from zsh","stdin":"\n","verbose":"0"}' | curl)  0.01s user 0.01s system 36% cpu 0.042 total

I think the HTTP API can be a separate package. I have implemented my own zsh version using two Python packages, Brish which is similar to this DaemonMode.jl, and BrishGarden which uses Brish to implement an HTTP API with multithreading. I'm open to implementing an HTTP API over DaemonMode.jl, too, but I don't know when I'll have the time.

dmolina commented 4 years ago

Thank you for the suggestion. I copy the idea. I had chosen the socket option because it is in the standard library, but actually rest api could be simpler. After making possible that the server and the client are in different computers, I will add it.

zahachtah commented 3 years ago

This would be extremely useful. Is it implemented already? didn't see it described in the readme.

NightMachinery commented 3 years ago

I created a basic Genie app to try and do this: https://github.com/NightMachinary/RestingDemon

The file https://github.com/NightMachinary/RestingDemon/blob/master/routes.jl is where the logic should be defined:

using Genie, Genie.Router, Genie.Requests, Genie.Renderer.Json
using DaemonMode

route("/jsonpayload", method = POST) do
  @show jsonpayload()
  out = runexpr(jsonpayload()["code"])
  json(out)
end

The problem is that I need runexpr to return stdout, stderr, and the resulting Julia object to me, so that I can put them in the JSON response; But runexpr just dumps stdout (etc) to the Genie app's stdout, and doesn't return anything.

Using

route("/jsonpayload", method = POST) do
  @show jsonpayload()
  code = jsonpayload()["code"]
  out = eval(Meta.parse(code))
  json(out)
end

I can at least do the easiest stuff:

❯ echo '{"code": "1+(8*4)"}' | curl --fail --silent --location --header "Content-Type: application/json" --request POST --data '@-' localhost:8000/jsonpayload

33
dmolina commented 3 years ago

Thank you for your interest. I am going to add the REST API, these days I am a bit busy with the starting of second semester, but I hope to do it for next week.