wilk / microjob

A tiny wrapper for turning Node.js worker threads into easy-to-use routines for heavy CPU loads.
https://wilk.github.io/microjob/
MIT License
2.02k stars 47 forks source link

docs: async job #24

Closed larshp closed 5 years ago

larshp commented 5 years ago

From GUIDE.md: "An asynchronous job is a task with at least one async call: for instance, a query to a DB, a HTTP request, a file system call, etc, and of course plus a heavy CPU load"

from https://nodejs.org/api/worker_threads.html#worker_threads_worker_threads "do not use them for I/O, since Node.js’s built-in mechanisms for performing operations asynchronously already treat it more efficiently than Worker threads can"

so suggest removing "a file system call" in GUIDE.md

(sorry about the spam today)

wilk commented 5 years ago

In this case async job refers to async microjob. Inside a job (executed in a separated thread) microjob allows you to use the async/await pattern because you could need to perform I/O bound with CPU bound ops.

Take a look to the example, taken from the docs:

(async () => {
  const { job } = require('microjob')

  try {
    // this function will be executed in another thread
    const res = await job(async () => {
      let i = 0
      for (i = 0; i < 1000000; i++) {
        for (let j = 0; j < 1000000; j++) {
          for (let k = 0; k < 1000000; k++) {
            await http.get('www.google.it')
          }
        }
      }

      return i
    })

    console.log(res) // 1000000
  } catch (err) {
    console.error(err)
  }
})()

Maybe it's not so clear but you should be able to perform also async ops and not just sync ones.

What about to improve the example with mixed sync/async ops?

larshp commented 5 years ago

yes, it is possible to do I/O inside a thread, however the Node documentation recommends not doing this. Reading the microjob GUIDE, it is possible to get the impression that it is okay to do I/O in a thread(which sometimes might be needed)

But to avoid conflict with the node recommendations I suggest just removing the "file system call" from the list of examples, or alternatively leave a note that this is not recommended

wilk commented 5 years ago

I didn't express myself well. Node.js does not recommend to use worker threads as a replacement of the async/await pattern. In fact, the event-loop is enough to sustain I/O bound ops. However, inside a worker thread, you could need to perform also I/O ops and not just CPU ops and that's ok: you're executing a heavy background operation but at some point you need to perform a fs call (for instance, reading/writing a file) and so you should be able to use async/await instead of the old callback system.

larshp commented 5 years ago

👍