Extension build chain not working

asteinarson commented 3 years ago

1) The issue and what you expected to happen Building an extension (display, module,...) should work

2) Exact steps to reproduce this issue In a blank node.js project, follow the instructions here: https://docs.directus.io/guides/displays/

3) Your environment: Directus: 9.0.0:-rc.44 DB: Postgres 12, Deployment: WSL2 Browser: Chrome

4) Any other relevant information we might need to reproduce this issue The dependencies (or resolution of) of some Vue/Rollup modules have changed. If I download an existing Directus 9 plugin (such as directus-extension-editorjs-interface) - with a package-lock.json. Then I can build with npm install. And generate a functional extension with npx rollup -c.

However if I regenerate _nodemodules and package-lock.json (from package.json) - then newer dependencies are used. And I get this result (on feeding rollup the example interface):

$ npx rollup -c
src/index.js → dist/index.js...
[!] (plugin commonjs) SyntaxError: Unexpected token (2:1) in /home/arst/proj/dir-exts/if-test2/src/interface.vue?vue&type=template&id=64969d30&lang.js
src/interface.vue?vue&type=template&id=64969d30&lang.js (2:1)
1:
2:   <div>My Custom Interface</div>

It's like the .vue file is not sent to the right parser at all.

The only way I can get the extension examples (from Directus docs) to work, is to manually copy a package-lock.json from another Directus extension on Github.

After quite some looking around I made some observations:

When it works (old package-lock.json), compiling of *.vue files seem to happen via vue-template-compiler. However this is for Vue 2.x - and in theory it shouldn't work with Vue 3 (which relies on @vue/compiler-sfc instead). Here is a discussion on some of it: https://stackoverflow.com/questions/64868632/vuejs-3-problem-with-vue-template-compiler
The key dependency update seem to be rollup-plugin-vue@5.0.0 to rollup-plugin-vue@6.0.0 (this removes the dependency on vue-template-compiler) (Note: To be changed in docs I think).
But when I have those new dependencies... unfortunately rollup starts to have problems (and I get the Unexpected token above).
Another update is that rollup-plugin-commonjs has been deprecated and replaced by @rollup/plugin-commonjs (however same error (not parsing the .vue file) if I go back to it). Again I guess Directus docs could reflect that.

I've not been able to make progress beyond this. It seems to me, if Directus extension building works for some now, it is because they happen to have a slightly aged package-lock.json - and that is not a very stable situation.

I attach my package.json and rollup.config.js. package.json rollup.config.js

Also, I went over docs here: https://v3.vuejs.org/guide/single-file-component.html#building-with-rollup But trying to build src/index.js + src/interface.vue based on that gave more or less the same error.

So... a bit stuck here...

rijkvanzanten commented 3 years ago

Yup! Seeing that both Rollup and Vue are going through a major version change, this'll be a little rocky for now. Long story short, as long as Directus itself is still running Vue 2, you can't use Vue 3 in your extensions.

I'd recommend locking the versions used to the ones that are compatible with Vue 2 (including vue-template-compiler) and using that for now

asteinarson commented 3 years ago

I wasn't clear if Directus was on Vue 2 or 3 - but assumed the later, since I saw use of composition API (I also assumed so, given that Directus 9 is quite new).

I found it now (Vue: 2.6.12) in top level package.json. It would be nice to have something about that, in the docs, maybe in the section about extension building.

I'll try that approach tomorrow :-)

On Fri, Mar 5, 2021 at 9:34 PM Rijk van Zanten notifications@github.com wrote:

Yup! Seeing that both Rollup and Vue are going through a major version change, this'll be a little rocky for now. Long story short, as long as Directus itself is still running Vue 2, you can't use Vue 3 in your extensions.

I'd recommend locking the versions used to the ones that are compatible with Vue 2 (including vue-template-compiler) and using that for now

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/directus/directus/issues/4401#issuecomment-791671527, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACDJGUURS37S6ITB2RMOMFLTCE56ZANCNFSM4YV3B52A .

asteinarson commented 3 years ago

OK - so I think it's sorted out. But I'd say some updates in docs would be fine, to save others from stumbling into this.

Short version: All recent examples I find out there, of building Vue single file templates, on the server, pulls in both @vue/compiler-sfc (which is only for Vue 3.0) AND vue-template-compiler (which is only for Vue 2.x).

For some mysterious reason (until recently), Node has resolved packages so that one got away with this contradiction (undefined whether one targets Vue2 or Vue 3). But internally it did the Vue2 compilation.

I don't know why all examples are pulling in @vue/compiler-sfc. But that pulls in Vue 3.0 as a dependency and there we are.

Solution:

Remove @vue/compiler-sfc from package.json
Make Vue 2.6.12 an explicit dev dependency from package.json
- Not sure it is needed
Downgrade rollup-plugin-vue (from 6.0 to to 5.x)
- Note: This one was important.
Make vue-template-compiler an explicit dependency in package.json

Then it worked for me.

I think it would be great if someone else can try these steps, verify they work (for a real world plugin). And that extension docs could mirror this.

Mrmiffo commented 3 years ago

I was facing issues with getting my module to work, though I was not having the same errors:

PS>npx rollup -c

src/index.js → dist/index.js...
[@vue/compiler-sfc] compileScript now requires passing the `id` option.

[@vue/compiler-sfc] compileTemplate now requires the `id` option.`.

(!) Unresolved dependencies
https://rollupjs.org/guide/en/#warning-treating-module-as-external-dependency
vue (imported by src\module.vue?vue&type=template&id=b52a34f6)
(!) Unused external imports
createVNode imported from external module 'vue' but never used
created dist/index.js in 371ms

I tried your solution above:

Remove @vue/compiler-sfc
Downgrade rollup-plugin-vue to "^5.x"
Add "vue-template-compiler": "^2.6.12" to devDependencies.

I then ran npm i -D rollup rollup-plugin-commonjs rollup-plugin-node-resolve rollup-plugin-terser rollup-plugin-vue@5.0.0 @vue/compiler-sfc rollup-plugin-vue@next as per the documentation. This gave me a high severity vulnerability, which I examined with npm audit:

npm ERR! code ELOCKVERIFY
npm ERR! Errors were found in your package-lock.json, run  npm install  to fix them.
npm ERR!     Missing: vue-template-compiler@^2.6.12

Running npm install resolved this issue, but resulted in a new high severity Remote Code Execution vulnerability in rollup-plugin-vue version 5, recommended solution is to upgrade to version 6 (which we can't as mentioned above). In addition, it modified the devDependencies in package.json to the following:

    "@vue/compiler-sfc": "^3.0.7",
    "rollup": "^2.41.2",
    "rollup-plugin-commonjs": "^10.1.0",
    "rollup-plugin-node-resolve": "^5.2.0",
    "rollup-plugin-terser": "^7.0.2",
    "rollup-plugin-vue": "^5.0.0",
    "vue-template-compiler": "^2.6.12"

In any case, I finally ran npx rollup -c and moved the index file to my module extensions and now the module works as expected. As such it seems like the required changes are to add vue-template-compiler and downgrade rollup-plugin-vue. However, this comes with a Remote Code execution vulnerability.

chomamateusz commented 3 years ago

OK - so I think it's sorted out. But I'd say some updates in docs would be fine, to save others from stumbling into this.

Short version: All recent examples I find out there, of building Vue single file templates, on the server, pulls in both @vue/compiler-sfc (which is only for Vue 3.0) AND vue-template-compiler (which is only for Vue 2.x).

For some mysterious reason (until recently), Node has resolved packages so that one got away with this contradiction (undefined whether one targets Vue2 or Vue 3). But internally it did the Vue2 compilation.

I don't know why all examples are pulling in @vue/compiler-sfc. But that pulls in Vue 3.0 as a dependency and there we are.

Solution:

Remove @vue/compiler-sfc from package.json

Make Vue 2.6.12 an explicit dev dependency from package.json

Not sure it is needed

Downgrade rollup-plugin-vue (from 6.0 to to 5.x)

Note: This one was important.

Make vue-template-compiler an explicit dependency in package.json

Then it worked for me.

I think it would be great if someone else can try these steps, verify they work (for a real world plugin). And that extension docs could mirror this.

This worked for me! Thanks. But it is ultra weird and not well documented :/

rijkvanzanten commented 3 years ago

It's a little finicky right now with all the tooling upgrading from Vue 2 to Vue 3... Some packages already published the Vue 3 versions as latest, while others keep Vue 2 as latest and publish Vue 3 under next.. We'll have to do some research and lock the packages on the latest Vue 2 compatible ones, until we move Directus itself over to Vue 3

rijkvanzanten commented 3 years ago

Closed in https://github.com/directus/directus/commit/726420576b00e2fa8e15bd01567a8a5b1601d402#diff-46b42b4229cd7a39c564e780bb665a8bde4fdf722007e8473f167fe53ed4b995

directus / directus

Extension build chain not working #4401