Add hook after all files have been processed?

[vedantroy]

For typecheck.macro, I am trying to support the following feature:

file1.ts:

import { register } from 'typecheck.macro'
export interface Example {}
register('Example')

file2.ts:

import createValidator from 'typecheck.macro'
import type { Example } from "./file1.ts"
createValidator<Example>()

What's going on here is that the register macro is being called to "register" the Example type. Then when createValidator is called inside "file2.ts", it looks up the registered type and generates the validator.

This utilizes the fact that macros can have global inter-file state, like so:

let numberOfFiles = 0;

function macroHandler({ references, state, babel}) {
     numberOfFiles++;
     console.log(numberOfFiles)
}

export default createMacro(macroHandler)

Problem description:

The issue is that I need to wait until all files have been macro-ed. Then I can process all register paths at once (to generate a global map of all the types), and then I can process all instances of createValidator.

This would require a hook that is called once after all files have been processed. The hook wouldn't have to have any parameters or anything complicated.

Suggested solution:

The api could like this:

let numberOfFiles = 0;

function macroHandler({ references, state, babel}) {
     numberOfFiles++;
     console.log(numberOfFiles)
}

function afterAllFilesHaveBeenProcessed() {
 // do something with numberOfFiles
// because we now know that all macro paths have been processed/encountered
}

export {afterAllFilesHaveBeenProcessed as hook}
export default createMacro(macroHandler)

I would be very willing to make a PR and implement this feature because it's pretty crucial to typecheck.macro/I'm pretty sure typecheck.macro is only useful if it is a macro.

[kentcdodds]

Unfortunately I don't think this is possible. Babel plugins (like babel-plugin-macros) operate on a single file and aren't made aware of any other files that are being processed.

[haltcase]

@kentcdodds if @vedantroy does the tracking themselves as proposed (e.g. by using their register macro) then there is no requirement for Babel or babel-plugin-macros to be aware of other files at all. From the OP:

let numberOfFiles = 0;

function macroHandler({ references, state, babel}) {
     numberOfFiles++;
     console.log(numberOfFiles)
}

export default createMacro(macroHandler)

Theoretically, babel-plugin-macros could have several hooks forming a lifecycle. Whether the hooks are provided anything would be up for debate, but I think if they were to be added at all they should be kept simple and leave state tracking up to the macro developer. For example:

const macroState = {
  files: []
}

export const beforeAll = () => {
  // do something before any uses of this macro are processed
}

export const afterAll = () => {
  // do something after all uses of this macro have been processed
  for (const file of macroState.files) {
    console.log(`File at path ${file} used 'register'`)
  }
}

export default createMacro(({ references, state }) => {
  const { filename } = state.file.opts

  if (references.register?.length > 0) {
    macroState.files.push(filename)
  }

  macroState
})

[kentcdodds]

Sorry, I'm really confused. When/how would beforeAll and afterAll be called if babel-plugin-macros doesn't have any concept of "all" 🤔 I'm pretty confident this is impossible.

[haltcase]

Yeah, after taking a look at the implementation again you're right of course — this is all processed by file, in import order. The sequence in my head (by macro in import order, then file) makes no sense in hindsight 😄

[vedantroy]

Yeah, this isn't possible, which is unfortunate but inevitable due to the way Babel works.

[nettybun]

Thought I'd mention you can actually get around this by leveraging the process.on('exit') hook of Node since everything runs in one process. It might be "dirty" but hey - works.

import { createMacro } from 'babel-plugin-macros';
import * as fs from 'fs';
import type { MacroHandler } from 'babel-plugin-macros';

const thingsFromAllFiles = [];

// Replaced each time the macro is run, so process.exit only runs _once_
let processExitHook = () => {};
process.on('exit', () => processExitHook());

const yourMacro: MacroHandler = ({ references, state }) => {
  references.default.forEach(referencePath => {
    // Here's where you'd do work. Maybe extract a string from the AST
    thingsFromAllFiles.push("...");
  });
  // Replace. Not set! Else you'll stack exit hooks
  processExitHook = () => {
    console.log(`Found ${thingsFromAllFiles.length} things in all files`);
    fs.writeFileSync("output.txt", thingsFromAllFiles.join('\n'));
  };
};

export default createMacro(yourMacro);

A real example is here in my repo: https://github.com/heyheyhello/stayknit/blob/fbc56a9d1f9ca15e4078af1b320b9d130bcc3190/contrib/babel-style-takeout/style-takeout.macro.ts I pull out CSS-in-JS to its own CSS file and replace the node with a generated CSS classname

Works OK. The only trouble is babel --watch doesn't work because the process never exits 😅 Maybe I'll think of something... I'm hoping to not just need to debounce() it...