K-FOSS / TS-ESNode

Node.JS Loader hook to transform typescript source and files as node loads them TS-ESNode. TS-Node for ESNext & ESModule Node.JS.
MIT License
38 stars 4 forks source link

Yarn2 support #1

Open armarti opened 4 years ago

armarti commented 4 years ago

Hey there, this is a neat project and could hopefully someday do away with needing ts-node for everything.

I want to open this as more of a TO-DO for when I or someone else finds time. Right now I'm using yarn2, which uses a virtual filesystem and doesn't have a node_modules folder. All packages are instead stored in .zip files and extracted when needed.

A few takeaways from my hour trying to get this to work with yarn2.0.0-rc.29 and node 13.9:

So maybe sometime later this month I'll have some bandwidth to tackle this. But compliments again for finding a way to leverage these new node features to potentially make typescript less painful.

EntraptaJ commented 4 years ago

Sorry I missed this earlier. I think I know why this is failing and I'm not sure how it could be resolved without finding how to remove my hack for supporting commonJS modules, because without my dynamic module type all imported node modules not ESNext format give a XYZ does not export ABC, if you had something like this:

import { ABC } from 'XYZ'

So to solve that and allow for compat with older modules or TypeScript projects with ESModule compat enabled I have to load all node_modules as dynamic modules with a dynamicinstanciate hook that manually imports with a createRequire and then returns the values on the object and spreads out anything withing imported.default. It's super hacky but it was the only way to achieve my goal of drop in support in most projects without any refactoring.

Here is how I load all "node_modules"

/**
 * This dynamically imports the `node_modules` module and creates a dynamic module with all the same exports.
 * @param url fileURL given by Node.JS
 */
export async function dynamicInstantiate(url: string) {
  // Create a Node.JS Require using the `node_modules` folder as the base URL.
  const require = createRequire(
    `${url.split('/node_modules/')[0].replace('file://', '')}/node_modules/`,
  );

  // Import the module file path
  let dynModule = require(url.replace(/.*\/node_modules\//, ''));

  /**
   * This is needed to allow for default exports in CommonJS modules.
   */
  if (dynModule.default)
    dynModule = {
      ...dynModule.default,
      ...dynModule,
    };

  const linkKeys = Object.keys(dynModule);

  return {
    exports: [...linkKeys, 'default'],
    execute: (module: any) => {
      module.default.set(dynModule);
      // For all elements in the import set the module's key.
      for (const linkKey of linkKeys) module[linkKey].set(dynModule[linkKey]);
    },
  };
}
EntraptaJ commented 4 years ago

That hack is the only part of this codebase I hate/want to get rid of. If anyone has any solutions or alternative ways of supporting destructing of legacy node_modules please submit a PR or let me know and I can attempt it.

EntraptaJ commented 4 years ago

I will attempt to refactor the file and module loader functions this weekend.

EntraptaJ commented 4 years ago

@armarti Can you test to see if the issue is still present with the changes made in #28 I changed how I require in Common.JS modules, which could possibly allow the Yarn2 hack to redirect require to work, although I really don't want to setup Yarn myself.

armarti commented 4 years ago

Ha just seeing this again, impressive progress you've made. With luck I'll get back to the project I was working on in the next week or two and I could try this out.

EntraptaJ commented 4 years ago

Yeah, when I started this project it was meant as a proof of concept to be integrated into TS-Node now I've morphed it into its own ecosystem.

EntraptaJ commented 3 years ago

Wait, I wonder if my new getSource based CJS module compat/(TOTALLY NOT A HACK) makes Yarn2 Work now...

EntraptaJ commented 3 years ago

Doesn't work magically but I do think that this will be easier to be done

EntraptaJ commented 3 years ago

The core of this issue is that Yarn2 doesn't support NodeJS loader hooks, because their hack to allow for the "compressed" modules assumes that ESModules don't exist in Node.JS. This can for sure be done. I'm not sure as to how complex this will be. I myself don't use Yarn2. And as much as I'd love to solve this myself, unless I decide to understand the internals of Yarn2 I don't think I can do this myself.

EntraptaJ commented 3 years ago

One way I could see this working is if TS-ESNode is loaded from disk instead of attempting to load the installed version from Yarn

EntraptaJ commented 3 years ago

Just thought of a potential workaround, using a bin script that loads TS-ESNode. Turns out that even that won't work because the virtual filesystem appears to not include the package.json file that includes the "type": "module JSON