vercel / modelfusion

The TypeScript library for building AI applications.
https://modelfusion.dev
MIT License
1.01k stars 76 forks source link

experimental-vm-modules bug introduced in v0.134.0 #278

Closed jakedetels closed 5 months ago

jakedetels commented 5 months ago

I updated to the latest version of Model Fusion (0.135.0) and found that my previously passing tests were now failing with this error:

    TypeError: A dynamic import callback was invoked without --experimental-vm-modules

      at ensureLoaded (node_modules/src/core/getRun.ts:14:35)
      at getRun (node_modules/src/core/getRun.ts:25:9)
      at executeStandardCall (node_modules/src/model-function/executeStandardCall.ts:42:21)
      at generateText (node_modules/src/model-function/generate-text/generateText.ts:77:30)

I've tested in the latest version of node.js and the LTS, and a few recent older versions: same results each time.

I isolated the error to be something that was introduced in v0.134.0 (as the same test suite passes up until v0.133.0). I tried examining the Model Fusion diffs between those two versions (https://github.com/lgrammel/modelfusion//compare/v0.133.0...v0.134.0#), but I couldn't identify the source of the issue.

Please advise. I didn't see anything in the release notes indicating breaking changes in v0.134.0. Thanks in advance.

lgrammel commented 5 months ago

Thanks for the bug report. I suspect it is because I switched bundling to tsup. Can you share some code that can help me reproduce the issue?

lgrammel commented 5 months ago

@jakedetels I did some digging and it might be related to jest specifically (see https://jestjs.io/docs/ecmascript-modules ). Are you using jest by any chance? Does this also happen outside of tests?

lgrammel commented 5 months ago

@jakedetels can you try https://github.com/lgrammel/modelfusion/releases/tag/v0.135.1 ?

It should fix the issue. If you still run into it, please re-open.

jakedetels commented 5 months ago

@lgrammel, thanks for the quick attention on this. I tested with 0.135.1, and confirm that it fixed the previously reported issue and my tests are now passing again. However, the patch release introduced a bug/side effect related to the schema of the event objects that are emitted (which I leverage for logging purposes). In 133, event.input was an array of messages (see first screenshot). Now in 0.135.1, to access the same array that was at event.input earlier, code must be revised as event.input.input (see second screenshot). How do you advise proceeding? Is the new event object schema locked in, or will you restore to the pre-v0.135.1 version?

0.133.0

image

0.135.1

image
lgrammel commented 5 months ago

@lgrammel, thanks for the quick attention on this. I tested with 0.135.1, and confirm that it fixed the previously reported issue and my tests are now passing again. However, the patch release introduced a bug/side effect related to the schema of the event objects that are emitted (which I leverage for logging purposes). In 133, event.input was an array of messages (see first screenshot). Now in 0.135.1, to access the same array that was at event.input earlier, code must be revised as event.input.input (see second screenshot). How do you advise proceeding? Is the new event object schema locked in, or will you restore to the pre-v0.135.1 version?

0.133.0 image

0.135.1 image

@jakedetels this is expected (a result of introducing prompt functions). In general, the event structure is an area that will likely change a bit more going forward. Are you using the events / input information or is this caused by snapshot tests?

jakedetels commented 5 months ago

@lgrammel, I've been using the event object within a logging service I'm using. (Issue is unrelated to snapshot testing.) No worries if the event schema will change a bit further, but it would be nice to lock things in at some point soon, since the event object is part of MF's public API for function observers.

lgrammel commented 5 months ago

@jakedetels totally - it's good to know that you are using the event, so I'll keep that in mind when there are updates.