wilk / microjob

A tiny wrapper for turning Node.js worker threads into easy-to-use routines for heavy CPU loads.
https://wilk.github.io/microjob/
MIT License
2.02k stars 47 forks source link

ts-node support/example? #54

Closed kirillgroshkov closed 4 years ago

kirillgroshkov commented 4 years ago

Great library! Love the minimalistic API and 0 dependencies!

I just tried to use it for a little bit more complex example and it failed immediately with such code:

await job(async accountId => {
  const account = await accountDao.getById(accountId)
  ...
}, {data: accountId})

What's happening here is that it's using some accountDao to fetch some thing from DB. This accountDao is imported above as:

import {accountDao} from '@src/accountDao'

2 problems:

  1. It can't find this module by such path (probably because @src is achieved using tsconfig-paths)
  2. Even if I change it to some relative path like ./src/accountDao or ../../src/accountDao - no luck.

My next try to is to:

require('ts-node/register')
const {accountDao} = require('./src/accountDao')

And then it's loaded! But then all its dependencies fail (cause they're still written in form of @src/....

Any advice? Is it just me or someone else also having such issue?

I just cannot imaging the use of this module without using some dependent files, which doesn't work right now for a typescript project..

jsalonen commented 4 years ago

I can confirm the same issue. Code loaded with require seems to work in ts-node, but import ... from doesn't.

wilk commented 4 years ago

Mmm this is weird. The scope of microjob's callback is not shared with its parent, meaning that you shouldn't be able to use accountDao in any form (both using import or require). You're saying it is working with something like this?

require('ts-node/register')
const {accountDao} = require('./src/accountDao')

await job(async accountId => {
  const account = await accountDao.getById(accountId)
  ...
}, {data: accountId})
kirillgroshkov commented 4 years ago

Mmm this is weird. The scope of microjob's callback is not shared with its parent, meaning that you shouldn't be able to use accountDao in any form (both using import or require). You're saying it is working with something like this?

No, sorry, wasn't clear enough. It was working like this:

await job(async accountId => {
  require('ts-node/register')
  const {accountDao} = require('./src/accountDao')
  const account = await accountDao.getById(accountId)
  ...
}, {data: accountId})

But such usage kills all the advantage of multithreading, since it takes significant time to transpile the code for each job invocation.

Probably I should just instantiate worker once and then stream input object there and receive a stream of outputs from it. But that's not the API that microjob offers, am I right?

wilk commented 4 years ago

@kirillgroshkov Using ts-node in production is not recommended exactly for performance degradation. Said that, you can do it both with require (and then compile) and by sending/receiving data to/from the worker. microjob has a worker pool mechanism that preallocates a set of worker to increase run-time performance, so don't worry and go ahead with your project 😄

wilk commented 4 years ago

I'm closing this. If you think that the issue is still unsolved, please feel free to reopen it.