Worker statistics

[wlandau]

For the controller and controller group, it would be nice to combine the output of mirai::daemons()$daemons with information about the load balancing of tasks in a nice data frame.

[wlandau]

This means the crew_eval() and monad will probably have to report which worker it ran on. That's easy.

[wlandau]

Should also record launch attempts in the launcher.

[wlandau]

Extra custom load balancing info should include the number of tasks on each worker and the total amount of time each worker spent on tasks. Here, each worker is labeled according to its mirai websocket. In the case of transient workers, mirai websockets may be reused a lot.

[wlandau]

• [x] The router should have a daemons field to stash the output of mirai::daemons()$daemons. It should get updated on initialization, then each call to router$poll(), then on router$terminate().
• [x] The launcher should count the number of launches of each worker.
• [x] The monad class should include a field for the mirai websocket the worker ran on, taken from Sys.getenv("CREW_SOCKET_MIRAI").
• [x] crew_eval() should get Sys.getenv("CREW_SOCKET_DATA") and Sys.getenv("CREW_SOCKET_SESSION")and return them in the monad.
• [x] crew_worker() should set env vars CREW_SOCKET_DATA and CREW_SOCKET_SESSION before calling mirai::server().
• [ ] The controller should put all the above info together in a nice data frame that can be polled even after termination.

[wlandau]

FYI @shikokuchuo, this feature may delay the CRAN release of crew by a little bit.

[shikokuchuo]

FYI @shikokuchuo, this feature may delay the CRAN release of crew by a little bit.

It will take some time after the Windows binary is built for it to replicate to the win-devel test machine in any case for your package to pass pretest - as you are depending on mirai >= 0.8.0.

On the other hand - the groundwork at nanonext is pretty much done for the next version of mirai, just needs a lot of testing. Then a re-design of the internals at dispatcher(). This also won't be quick. But in the meantime have a look over this and see if you're not excited: https://shikokuchuo.net/nanonext/reference/cv.html

[wlandau]

On the other hand - the groundwork at nanonext is pretty much done for the next version of mirai, just needs a lot of testing. Then a re-design of the internals at dispatcher(). This also won't be quick. But in the meantime have a look over this and see if you're not excited: https://shikokuchuo.net/nanonext/reference/cv.html

Very exciting indeed!