selfint / rust-ml

Rust machine learning library
0 stars 0 forks source link

Implement a rust-ml Errand #34

Open selfint opened 3 years ago

selfint commented 3 years ago

A 'mini-batch' of training for any sl/ul/rl algorithm, that is meant to be completed asynchronously from the main training program.

selfint commented 3 years ago

The idea is:

  1. The main server will generate a To Do queue of Tasks that need to be done in order to train the algorithm.
  2. Anyone (server/client) can take a Task from the To Do queue.
  3. Once a Task is taken from the To Do queue it is moved to the In Progress queue automagically.
  4. Once a Task is completed it is moved to the Completed queue.
  5. Once all relevant Tasks are In the Completed queue, the server performs the relevant actions - update network parameters, and/or anything else that was waiting for these Tasks to be completed.

Add checks to make sure that:

  1. If a client disconnected then his Tasks are moved back to the To Do column.
  2. If all Tasks in the To Do column are done then the Server will start working on the Tasks in the In Progress queue, in case it finishes before the client, since the server is waiting anyway.
selfint commented 3 years ago

This should probably be broken down into smaller issues

selfint commented 3 years ago

Implement async functionality using tokio