search

andywer / threads.js

🧵 Make web workers & worker threads as simple as a function call.
https://threads.js.org/
MIT License
3.05k stars 163 forks source link

[QUESTION] Node.js 8 to 11 #131

Closed simplecommerce closed 5 years ago

simplecommerce commented 5 years ago

Stupid question,

Is the expose function supposed to work in the worker when dealing with Node.JS 8 to 11?

I am not able to get it working, it always keeps giving me this error:

Error: Timeout: Did not receive an init message from worker after 10000ms. Make sure the worker calls expose().

Even if I copy and paste the exact same code from the docs.

But I am able to get it working if I use the same examples from tiny-worker. But in this fashion, I am not able to expose modules from the worker, etc.

Hope I am clear, sorry if its dumb.

andywer commented 5 years ago

Hey @simplecommerce. No, it’s not dumb at all! It should definitely work.

I think you just found a bug in the tiny-worker fallback implementation and/or the worker thread's error reporting code 😉

Did you install the tiny-worker package? And can you provide some sample code reproducing the issue?

Also: What operating system are you running the code on?

simplecommerce commented 5 years ago

@andywer Hi thanks for the quick response, I installed threads alone and I just did a test by installing tiny-worker also on top of it, and it still gives me the same error.

Os I am using it on is: Distributor ID: Ubuntu Description: Ubuntu 18.04.1 LTS Release: 18.04 Codename: bionic

NodeJS version 8.10.0 Npm 6.9.0

I literally tried this sample https://github.com/andywer/threads.js#basics and it gives me that error.

What do you recommend?

andywer commented 5 years ago

So I found a small issue in the sample code: It should be Thread.terminate(add) (#132). That doesn't explain the issue you see, though.

Worked on my machine, even though the process kept running after completing main():

/tmp/threads-test via ⬢ v8.16.0 took 10s 
➜ node --version
v8.16.0

/tmp/threads-test via ⬢ v8.16.0 
➜ cat master.js 
const { spawn, Thread, Worker } = require("threads")

async function main() {
  const add = await spawn(new Worker("./workers/add"))
  const sum = await add(2, 3)

  console.log(`2 + 3 = ${sum}`)

  await Thread.terminate(add)
}

main().catch(console.error)

/tmp/threads-test via ⬢ v8.16.0 
➜ cat workers/add.js 
const { expose } = require("threads/worker")

expose(function add(a, b) {
  return a + b
})

/tmp/threads-test via ⬢ v8.16.0 
➜ node ./master.js
Node worker_threads not available. Trying to fall back to tiny-worker polyfill...
2 + 3 = 5
^C^C^C

/tmp/threads-test via ⬢ v8.16.0 took 10s 
➜

Can you spot anything that looks unlike your code?

simplecommerce commented 5 years ago

I keep seeing this in my server logs error: unknown option--abort-on-uncaught-exception' But when I do aconsole.log` of the worker, I see this: 

Worker {
  child:
   ChildProcess {
     domain: null,
     _events:
      { internalMessage: [Function],
        error: [Function],
        message: [Function] },
     _eventsCount: 3,
     _maxListeners: undefined,
     _closesNeeded: 2,
     _closesGot: 0,
     connected: true,
     signalCode: null,
     exitCode: null,
     killed: false,
     spawnfile: '/usr/bin/node',
     _handle: Process { owner: [Circular], onexit: [Function], pid: 24142 },
     spawnargs:
      [ '/usr/bin/node',
        '/root/develop/node_modules/@babel/node/lib/_babel-node.js',
        '--abort-on-uncaught-exception',
        '/root/develop/node_modules/tiny-worker/lib/worker.js' ],
     pid: 24142,
     stdin: null,
     stdout: null,
     stderr: null,
     stdio: [ null, null, null, null ],
     channel:
      Pipe {
        writeQueueSize: 0,
        buffering: false,
        onread: [Function],
        sockets: [Object] },
     _channel: [Getter/Setter],
     _handleQueue: null,
     _pendingMessage: null,
     send: [Function],
     _send: [Function],
     disconnect: [Function],
     _disconnect: [Function] },
  onerror: undefined,
  onmessage: undefined }

I don't see much difference in your code, this is really puzzling to me. I think it might be related to the error I have in my log?

simplecommerce commented 5 years ago

Oh and also I am using @babel/node does it make a difference, I assume so? 7.4.5

andywer commented 5 years ago

Are you sure you are running node v8.10? --abort-on-uncaught-exception was already present in node v8.9.

Oh and also I am using @babel/node does it make a difference, I assume so?

Maybe. What's your babel configuration?

simplecommerce commented 5 years ago

Here is my package.json file, I just did an npm update.

{
  "name": "graphql-test-server",
  "version": "1.0.0",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1",
    "start": "babel-node index.js --presets @babel/env --plugins @babel/plugin-transform-destructuring,@babel/plugin-syntax-throw-expressions --abort-on-uncaught-exception"
  },
  "author": "",
  "license": "ISC",
  "dependencies": {
    "apollo-server": "^2.6.8",
    "apollo-server-caching": "^0.3.1",
    "applicationinsights": "^1.4.0",
    "bcrypt": "^3.0.6",
    "cron": "^1.7.1",
    "crypto-js": "^3.1.9-1",
    "filestack-js": "^1.14.6",
    "graphql": "^14.4.2",
    "graphql-middleware": "^3.0.2",
    "graphql-shield": "^5.7.3",
    "graphql-tag": "^2.10.1",
    "graphql-tools": "^4.0.5",
    "hashids": "^1.2.2",
    "i18n-nodejs": "^3.0.0",
    "i18next": "^15.1.3",
    "immutability-helper": "^3.0.1",
    "intl": "^1.2.5",
    "intl-locales-supported": "^1.3.4",
    "jsonwebtoken": "^8.5.0",
    "lodash": "^4.17.13",
    "merge-graphql-schemas": "^1.5.8",
    "moment": "^2.24.0",
    "moment-range": "^4.0.2",
    "moment-timezone": "^0.5.26",
    "mongodb": "^3.2.7",
    "node-sass": "^4.12.0",
    "nodemailer": "^6.2.1",
    "nodemailer-smtp-transport": "^2.7.4",
    "path-to-regexp": "^3.0.0",
    "request": "^2.88.0",
    "simple-recur": "^0.2.0",
    "speakingurl": "^14.0.1",
    "stimulsoft-reports-js": "^2019.3.2",
    "taxjar": "^2.3.0",
    "threads": "^1.0.0-beta.2",
    "tiny-worker": "^2.2.0",
    "uglify-js": "^3.6.0"
  },
  "devDependencies": {
    "@babel/cli": "^7.5.0",
    "@babel/core": "^7.5.4",
    "@babel/node": "^7.5.0",
    "@babel/plugin-syntax-throw-expressions": "^7.2.0",
    "@babel/plugin-transform-destructuring": "^7.5.0",
    "@babel/polyfill": "^7.4.4",
    "@babel/preset-env": "^7.5.4"
  },
  "description": ""
}

What is odd is that I even use it in my start command.

simplecommerce commented 5 years ago

image

simplecommerce commented 5 years ago

Ok, so I removed --abort-on-uncaught-exception from my package.json file and it fixed the issue for tiny-worker not working. (What a giant waste of my time and yours)

I am still trying to get it working with your api.

simplecommerce commented 5 years ago

Ok it works now, it was that invalid argument the issue, your sample code works perfectly, its just now a problem with my code.

Am I supposed to spawn a worker and kill the worker process on every request? Or do I keep one running to handle all requests?

Thanks for your help!!

andywer commented 5 years ago

Keep the worker alive until you don’t need it anymore, then kill it. Spawning a new worker is a quite expensive operation 😉

andywer commented 5 years ago

But that’s great news!