Hot module reloading

[Dale-Black]

Is there a way to restart the server on file changes within the directory?

I was thinking something like BetterFileWatcher.jl might be useful for something like this but I'm not sure how to implement it

[frankier]

I tried to take a little look at this myself. What I found was that Revise and includet of my main module alone were not sufficient. Possibly invokelatest needs to be used somewhere in the request/response loop? Or else listen for the revise event and restart the server altogether?

You might get some idea by looking at Genie's code:

https://github.com/search?q=repo%3AGenieFramework%2FGenie.jl%20revise&type=code

Happy to help test/review a PR.

[ndortega]

Hi @Dale-Black & @frankier

Thanks for opening this ticket and for the suggestions. This is a good next issue to tackle which would definitely help speed up local development.

I'll definitely try to follow what Genie.jl is doing to get this working, but I'm open to any and all suggestions. I did find a similar approach in this issue: https://github.com/JuliaWeb/HTTP.jl/issues/587

Also I'll let you know when I have something worth testing & reviewing @frankier, thanks for the help!

[Dale-Black]

That http.jl code looks great. So something like HTTP.listen with Revise should work?

[JanisErdmanis]

Is there any progress in this direction?

[ndortega]

@JanisErdmanis,

There was some progress made in a PR & discussion about a month ago - but I haven't worked on it since. I've been very busy this December.

[walterl]

@ndortega I see in #134 you mentioned a "server flapping issue". Where is that, and is there anything we can help with?

[asjir]

I use Revise and to do it I have:

include("routes.jl")
__revise_mode__ = :eval
Revise.track("routes.jl")
function ReviseHandler(handle)
    req -> begin
        Revise.revise()
        invokelatest(handle, req)
    end
end

[Dale-Black]

Hmm, and that works? Can you share a bit more of your code and structure? Where are you serving the app from?

[ndortega]

Hi guys,

Sorry about the inactivity on this issue. This issue is now on the top of my todo list. As far as direction and approaches, I'm all ears. Like most things, this is new territory for me and I'm open to any and all suggestions on how to tackle this problem.

@asjir thanks for your snippet. Id also like to hear more about your project setup and workflow. I really like how you embedded this feature as a middleware function, which would be a super clean way to integrate behind the scenes

[asjir]

Hmm, and that works? Can you share a bit more of your code and structure? Where are you serving the app from?

Ofc, though it's very basic: in my repo I have sveltekit projects and one folder for julia server X, in X I have 5 files: Project.toml, Manifest.toml, main.jl, routes.jl, util.jl at the top of main.jl I have cd(@__DIR__) (so imports run from REPL) and then include("routes.jl") etc. at the bottom of main.jl I have:

run(; port=2227, async=true) = (!isdefined(Main, :s) || !isopen(Main.s)) && (Main.s = serve(middleware=[CorsHandler, ReviseHandler]; port, async))
run()

this function starts an async server that I can close(s) but reruns of it don't try to start a new server which would overwrite my s variable

and inside routes.jl there's

include("util.jl")
Revise.track("util.jl")

so changes to util are also immediately reflected.

While I'm at it I'll show an example route:

post("/make-qch") do req
    data = json(req, MyStruct)
    saved[] = data
    f(data)  # returns NamedTouple
end

which allows me to call f(saved[]) from the REPL and Infiltrator.@inflitrate inside this function (can't use it in an async call of server response).

[Dale-Black]

It looks like Fons' new project has built in hot module reloading. I haven't worked with it yet but something to look into https://github.com/JuliaPluto/PlutoPages.jl

[JanisErdmanis]

I can report that I have had some success with ReviseHandler, which use as follows:

using ModuleA
using Oxygen

function ReviseHandler(handle)
    req -> begin
        Revise.revise()
        invokelatest(handle, req)
    end
end

server = ModuleA.serve(port=2345; middleware=[ReviseHandler])

The ModuleA is defined as:

module ModuleA

using Oxygen; @oxidise

@get "/inside" function(req::Request)
    return "Hello from Inside"
end

end 

where the @oxidise macro will be available after the pull request https://github.com/OxygenFramework/Oxygen.jl/pull/158 will be merged.

[Dale-Black]

That looks clean

[JanisErdmanis]

The pull request is now merged on master. It would be great to get a feedback on the Revise workflow before the release.

[Dale-Black]

Amazing, works very nicely! (https://github.com/Dale-Black/HTMLStrings.jl/tree/main/examples/TodoApp)

Now I have more motivation to keep exploring "full stack" Julia. Excited to build out a better TodoApp with pure Julia as an example

[JanisErdmanis]

@Dale-Black, is there any reason why you have chosen to register routes into the todo() function? For me, it looks like a bizarre thing to do, as the routes are registered at runtime.

[Dale-Black]

No there isn't. I just changed that

[ahjulstad]

I am following this with great interest. Unfortunately I cannot get the approach from https://github.com/OxygenFramework/Oxygen.jl/issues/122#issuecomment-1939386137 to work, possibly because I don't understand how to setup folder structures and modules properly. Specifically, I am doing:

server.jl:

includet("submodule.jl")

using ..ModuleA
using Oxygen

function ReviseHandler(handle)
    req -> begin
        Revise.revise()
        invokelatest(handle, req)
    end
end

server = ModuleA.serve(port=8080; middleware=[ReviseHandler])

and submodule.jl:

module ModuleA

using Oxygen; @oxidise

@get "/greet" function(req::Request)
    return "Hello from Inside, changed"
end

end 

Server works, but doesn't reload. What is strange, though, is that I need to restart Julia for changes to submodule.jl to register. A simple Ctrl-C and rerun of the server.jl (using run in REPL from in vscode) is not sufficient.

Is it required to have ModuleA in a proper package that is added as a dev dependency?

[JanisErdmanis]

The approach I outlined earlier works only when the module is loaded statically, which Revise then picks up. To get the setup working, use ] generate ModuleA, put the code in src/ModuleA.jl, use @oxidise macro and activate the project with ] activate . which makes ModuleA available. Then, in the REPL, start the service with ReviseHandler. I haven’t managed to get an includet approach to work. Perhaps it is related to the way globals are revised.

[ahjulstad]

Thanks. I got bit by not understanding modules vs packages and the functionality of ] generate.

I also managed to get it working using PkgTemplates, but this is maybe just more work for little gain, compared to your approach.

using PkgTemplate
t = Template(dir=".")
t("ServerPackage")

The development then happens in this ServerPackage.

Separate from this, a new server.jl resides in a project that has ServerPackage as a development dependency. (]dev ./ServerPackage) I have it in the root folder.

using Revise

import ServerPackage

function ReviseHandler(handle)
    req -> begin
        Revise.revise()
        invokelatest(handle, req)
    end
end

server = ServerPackage.serve(port=8080; middleware=[ReviseHandler])

And then, (just because it is possible), I run a client in separate process that repeatedly pings the API.