Simple server in multi-threaded julia

[h4rm]

Describe the bug I like to run a server on the main thread in a multi-threaded julia (e.g. julia -p4). I don't need to handle requests on multiple threads, however, I need the threads for some calculations.

I get the following error:

From worker 2:    2019-07-19 08:51:57:ERROR:Main: / 404
From worker 2: 

To Reproduce

Run the following code with julia -p 2

using Genie, Genie.Router, Genie.Renderer, Genie.Requests
using HTTP
using Distributed

println("Started julia with $(nworkers()) threads.")

Genie.config.run_as_server = true

route("/") do
    (:result => "Hello") |> json
end

Genie.startup()

Open the site.

Expected behavior The request is handled by the main thread.

[essenciary]

@h4rm Thanks for reporting this - this will take a bit of digging. Basically, it's about adding support for it.

Isn't it simple to start Julia on a single worker and spawn the computations within the server thread (in the controller or model)?

[h4rm]

@essenciary Hi Adrian,

thx for the quick response. At some point I need to add threads before Genie.startup() and I need to also load code into these threads with @everywhere. Therefore, as soon as startup is called, the threads have been created.

[Ankur-deDev]

Hi,

I bumped into a similar issue (when I start extra processes with Distributed.addprocs after launching the server). A quick workaround (that has not been tested much) is to replace in the file Genie.jl/src/AppServer.jl the Distributed.@fetch ... calls by something like Distributed.@fetch(Distributed.@spawnat 1 ...) to make sure this get executed on master and not on the slaves that do not have the server running.

Do you expect this to break anything?

Thanks.

[essenciary]

@Ankur-deDev tbh I don't think it will break things. To be sure we can start with the tests we have and maybe devise some new ones for parallel execution?

[carstenbauer]

At some point I need to add threads before Genie.startup() and I need to also load code into these threads with @everywhere.

Note that there seems to be a confusion here. The command julia - p4 doesn't start Julia with 4 threads but with 4 worker processes. That you need to load code with @everywhere is because the workers are different processes. For multi-threading (julia -t4) @everywhere is completely unnecessary. Also, note that nworkers() gives you the number of worker processes whereas it is Threads.nthreads() that counts the number of threads.

So the title of this thread (no pun intended) should be changed. It appears to have nothing to do with multi-threading.

Further reading in the Julia doc: multi-processing vs multi-threading

[jvo203]

Count me too, I would like to replace the bog-standard HTTP.jl + WebSockets.jl with Genie.jl. However, the lack of full support for Distributed.jl with #workers > 1 is a show-stopper. Have just tested Genie.jl. As of September 2021 running the test code with "julia -p 4" (anything greater than 1) produces "Error: GET / 404". End of the story...

[essenciary]

@jvo203 Thanks for surfacing this - we might have a bit of bandwidth to tackle this soon. Contributions would be more than welcomed (and I'd be happy to support and join any efforts around this issue).

[jvo203]

Thanks but no thanks! I need to prioritise the scarce development time, there are deadlines at work. As it stands the hurdles in adopting Genie.jl are pretty high:

1. Fixing the Genie.jl multi-processing (Distributed.jl #workers > 1).
2. Changing my existing client-side JavaScript to use Genie.jl channels instead of plain vanilla WebSockets. Ripping out the existing normal WebSocket code from JavaScript and replacing it with custom channels is a huge undertaking.
3. Learning how to do HTTP chunked streaming responses in Genie.jl.

Had Genie.jl worked out-of-the-box with #workers > 1 I might have given it a try, and even tried to convert the JavaScript WebSockets code to work with channels. Is there an easy way to use vanilla WebSockets in Genie.jl, not channels? The application I am working on uses custom HTTP handlers (including low-level HTTP chunking) + real-time WebSockets handlers. Can one easily do low-level HTTP chunking in Genie.jl? It is dead easy in HTTP.jl. And as Genie.jl uses HTTP.jl under the hood, I guess chunked streaming of HTTP responses should be possible in Genie.

[essenciary]

The choice is yours - you're welcome.

[shrimpceviche]

You could have just used multi-threading Threads.@threads that would run on multiple cores.

[jemferraz]

Hi, I am currently facing the same issue refered to by @jvo203 (https://github.com/GenieFramework/Genie.jl/issues/151#issuecomment-918724790), namely that of Distributed.jl with #workers > 1. Do you know if there has been any advance in that area, or in case not, can you give me some advice on how to proceed (just for reference, that is the first time I am building an API, and the reason I've chosen Genie, is because it is so simple to do with it...)? Thanks,

[garibarba]

@essenciary I'm willing to do a simple pull request for fixing this, but I need to understand your rationale for using @fetch to run as a process instead of a thread, or an example/test showing the expected usage of addprocs(N)/julia -procs N within Genie. Otherwise I'd propose to completely remove the usage of Distributed.@fetch 🤔

Alternatively we could use Distributed.@fetchfrom 1 ... so that it always run on the master process, but I don't see the point (I might be missing something) of running "distributed" if it always runs on the master process.

[essenciary]

I've been working on this recently with good success, managed to load and serve a simple app on multiple workers. However, some more advanced things are broken so it needs a bit more work. This will land soon.

[cjproud]

Hi @essenciary, I wonder if there has been any update on this? I'm currently using Oxygen, which supports multi-threading, and could find some time to give you a hand with this.

[essenciary]

@cjproud yes, did some experiments - it's not a big deal. The issue is really code loading (loading dependencies and resources on all workers) so I ended up with loading the app on each worker. From there it just works. What's needed: 1/ benchmark the effects of loading the app on every worker for RAM usage. At first sight the app's RAM usage seems minor when compared to the RAM usage of Julia itself, so it should be ok, but proper measuring is needed. 2/ the more complicated one is that it breaks WebChannels/WebSockets functionality as one websockets data push is generated for each worker, resulting in multiple identical data pushes.

[rori4]

yeah I would highly recommend anyone who wants to use multi-threading to switch to https://github.com/ndortega/Oxygen.jl

Genie doesn't support that out of the box as Oxygen does.

[jvo203]

Whilst Oxygen does seem to have a lot of functionality covered, the one major thing that's missing is WebSockets.

[rori4]

@jvo203 well I think you can handle them with custom middleware. Here is an issue/PR that has added that this summer https://github.com/ndortega/Oxygen.jl/pull/117

[jvo203]

Great!

[essenciary]

We have this ready to roll out in Genie 6.

[Thiago-Simoes]

Any estimates on the release date for version 6? I use Genie.jl a lot, this will be very useful.

[essenciary]

In the end, most likely we'll release the Genie 6 features into Genie 5 - as some of the breaking changes didn't provide the performance gains I was expecting. That would be ideal anyway.

[essenciary]

We could use help with porting features from the v6 branch into main if anybody is interested - starting with this one maybe. Should be straightforward. As we're a bit busy currently.

[mzy2240]

hey any updates on this? What’s the best practice now to run genie with some multi-thread computation-intensive function?