heavenkiller2018 commented 2 years ago

Describe the bug

some problems about branch:refactor/mv3 until 2022/11/22

I downloaded the latest source code:

❯ npx degit antfu/vitesse-webext#refactor/mv3 antfu@vitesse-webext_mv3_202211

problem 1

there is a ERR_PNPM_FETCH_404 error when using pnpm to install.

❯ pnpm i
Lockfile is up-to-date, resolution step is skipped
Packages: +893
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 WARN  GET https://registry.npmjs.org/@nodelib/fs.scandir/-/@nodelib/fs.scandir-2.1.5.tgz error (ERR_PNPM_FETCH_404). Will retry in 10 seconds. 2 retries left.
 WARN  GET https://registry.npmjs.org/@nodelib/fs.scandir/-/@nodelib/fs.scandir-2.1.5.tgz error (ERR_PNPM_FETCH_404). Will retry in 1 minute. 1 retries left.
 ERR_PNPM_FETCH_404  GET https://registry.npmjs.org/@nodelib/fs.scandir/-/@nodelib/fs.scandir-2.1.5.tgz: Not Found - 404

No authorization header was set for the request.
Progress: resolved 893, reused 891, downloaded 0, added 0

but it's normal if turn to yarn.

problem 2

scripts/client.tsline:12

const socketHost = `localhost:3303`

the socket port is hardcoded, so If the port is set by env variables, the follwoing error would occur:

client.ts:26 

       WebSocket connection to 'ws://localhost:3303/' failed: Error in connection establishment: net::ERR_CONNECTION_REFUSED

problem3

HMR is not stable, sometimes the changed reflected immediately, but sometimes, the changes can't be rendered in time. especially when continuously change the code. I haven't found what's the problem is. @tmkx

Reproduction

reproduction

System Info

System:
    OS: Linux 5.15 Ubuntu 20.04.4 LTS (Focal Fossa)
    CPU: (8) x64 Intel Core Processor (Haswell, no TSX, IBRS)
    Memory: 25.33 GB / 31.35 GB
    Container: Yes
    Shell: 5.8 - /usr/bin/zsh
  Binaries:
    Node: 18.12.0 - ~/.nvm/versions/node/v18.12.0/bin/node
    Yarn: 1.22.19 - ~/.nvm/versions/node/v18.12.0/bin/yarn
    npm: 8.19.2 - ~/.nvm/versions/node/v18.12.0/bin/npm
  Browsers:
    Chrome: 103.0.5060.53
    Firefox: 105.0

Used Package Manager

yarn

Validations

[X] Follow our Code of Conduct
[X] Read the Contributing Guide.
[X] Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.
[X] Check that this is a concrete bug. For Q&A, please open a GitHub Discussion instead.
[X] The provided reproduction is a minimal reproducible of the bug.

tmkx commented 2 years ago

Problem1. remove pnpm-lock.yaml and try install again

Problem2. can be solved

Problem3. HMR is unstable in MV3 as expected because the implementation is a little tricky, and that's why it's still not merged into main branch. needs more investigation.

heavenkiller2018 commented 2 years ago

@tmkx

problem4

error Uncaught SyntaxError: Cannot use import statement outside a module would occur when importing other modules in background.

for example, add such codes in background.ts

import axios from 'axios'

const axiosTest = () => {
  const axiosInstance = axios.create({})
  return axiosInstance
}
console.log(axiosTest)

show: I tried other modules such as 'ramda', it's the same error!

problem 5

how to dynamic inject content-script in mv3?

in mv3 template, dynamic inject by adding the following code in background.ts:

if (import.meta.hot) {
  // @ts-expect-error for background HMR
  import('/@vite/client')
  // load latest content script
  import('./contentScriptHMR')
}
```ts

but when adding this to mv3, such compile error occured:
```sh
✘ [ERROR] Could not resolve "/@vite/client"

    src/background/index.ts:32:9:
      32 │   import('/@vite/client')
         ╵          ~~~~~~~~~~~~~~~

IIFE Build failed
Error: Build failed with 1 error:
src/background/index.ts:32:9: ERROR: Could not resolve "/@vite/client"

and if I delete the import('/@vite/client'), another error Cannot use 'import.meta' outside a module would appear:

tmkx commented 2 years ago

Problem 4

The background script is bundled by tsup, it will external dependencies and peerDependencies, see the tsup doc.
To solve ①, you can install axios to devDependencies, or add noExternal: ['axios'] in tsup.config.ts.
There is another error: Adapter 'http' is not available in the build, try axios.create({ adapter: 'xhr' })
Still get Adapter xhr is not supported by the environment, because that XMLHttpRequest is not supported in background pages in MV3

heavenkiller2018 commented 2 years ago

@tmkx I followed your instruction of adding noExternal: ['axios'] in tsup.config.ts, but the error Uncaught SyntaxError: Cannot use import statement outside a module was still throwed out, insteadly, it occured in import part of background.ts.

besides, so it means that axios can't be used in MV3 background unless to find or write an adapter using fetch() ?
And what about the problem 5?

tmkx commented 2 years ago

Problem5

background script is not bundled by Vite, there is no import.meta.hot and HMR.

tmkx commented 2 years ago

It seems that axios is based on XHR, you can find a fetch-based alternative, or use fetch directly.

heavenkiller2018 commented 2 years ago

so how to use a background script to always inject the latest version of content-scripts in MV3, as as browsers would cache them for each reload

tmkx commented 2 years ago

Maybe you can create a WebSocket connection to Vite server, cleanup & remount content-scripts to the page when receiving updates? It's not implemented yet.

heavenkiller2018 commented 2 years ago

@tmkx, for problem 4, I have searched a similar issue which also got errors when bundling axios with tsup. https://github.com/axios/axios/discussions/4842. It explains it maybe caused by nodejs builtin modules when using for the browser. so I followed his instruction, firstly, add esbuild-plugins-node-modules-polyfill

 yarn add esbuild-plugins-node-modules-polyfill --dev

then , modify tsup.config.ts by adding:

import { nodeModulesPolyfillPlugin } from 'esbuild-plugins-node-modules-polyfill'
export default defineConfig(() => ({
...
  esbuildPlugins: [nodeModulesPolyfillPlugin()],
...
}))

this time, axios was bundled without any errors.

then, I tried to use axios to request something. Indeedly, the error "AxiosError: Adapter http is not supported by the environment you refered to occured.

{
    "message": "Adapter http is not supported by the environment",
    "name": "AxiosError",
    "stack": "AxiosError: Adapter http is not supported by the environment\n    at Object.getAdapter (chrome-extension://hbidegeigejmmpppbdmbbmobgcingopm/dist/background/index.mjs:69518:15)\n    at Axios.dispatchRequest (chrome-extension://hbidegeigejmmpppbdmbbmobgcingopm/dist/background/index.mjs:69554:36)\n    at Axios.request (chrome-extension://hbidegeigejmmpppbdmbbmobgcingopm/dist/background/index.mjs:69803:33)\n    at Axios.get (chrome-extension://hbidegeigejmmpppbdmbbmobgcingopm/dist/background/index.mjs:69822:17)\n    at Function.wrap [as get] (chrome-extension://hbidegeigejmmpppbdmbbmobgcingopm/dist/background/index.mjs:54941:15)\n    at chrome-extension://hbidegeigejmmpppbdmbbmobgcingopm/dist/background/index.mjs:70515:15",
    "code": "ERR_NOT_SUPPORT",
    "status": null
}

so I found an fetch adapter for axios to install:

yarn add @vespaiach/axios-fetch-adapter --dev

and use it in background.ts

import axios from 'axios'
import fetchAdapter from '@vespaiach/axios-fetch-adapter'
const axiosTest = () => {
  const axiosInstance = axios.create({ adapter: fetchAdapter })
  return axiosInstance
}
console.log(axiosTest)

but this time, new errors occured when bundling:

✘ [ERROR] Could not resolve "axios/lib/core/settle"

    node_modules/@vespaiach/axios-fetch-adapter/index.js:2:19:
      2 │ import settle from 'axios/lib/core/settle';
        ╵                    ~~~~~~~~~~~~~~~~~~~~~~~

  The path "./lib/core/settle" is not exported by package "axios":

    node_modules/axios/package.json:6:13:
      6 │   "exports": {
        ╵              ^

  You can mark the path "axios/lib/core/settle" as external to exclude it from the bundle, which
  will remove this error.

✘ [ERROR] Could not resolve "axios/lib/helpers/buildURL"

    node_modules/@vespaiach/axios-fetch-adapter/index.js:3:21:
      3 │ import buildURL from 'axios/lib/helpers/buildURL';
        ╵                      ~~~~~~~~~~~~~~~~~~~~~~~~~~~~

  The path "./lib/helpers/buildURL" is not exported by package "axios":

    node_modules/axios/package.json:6:13:
      6 │   "exports": {
        ╵              ^

  You can mark the path "axios/lib/helpers/buildURL" as external to exclude it from the bundle,
  which will remove this error.

✘ [ERROR] Could not resolve "axios/lib/core/buildFullPath"

    node_modules/@vespaiach/axios-fetch-adapter/index.js:4:26:
      4 │ import buildFullPath from 'axios/lib/core/buildFullPath';
        ╵                           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

  The path "./lib/core/buildFullPath" is not exported by package "axios":

    node_modules/axios/package.json:6:13:
      6 │   "exports": {
        ╵              ^

  You can mark the path "axios/lib/core/buildFullPath" as external to exclude it from the bundle,
  which will remove this error.

✘ [ERROR] Could not resolve "axios/lib/utils"

    node_modules/@vespaiach/axios-fetch-adapter/index.js:5:62:
      5 │ import { isUndefined, isStandardBrowserEnv, isFormData } from 'axios/lib/utils';
        ╵                                                               ~~~~~~~~~~~~~~~~~

  The path "./lib/utils" is not exported by package "axios":

    node_modules/axios/package.json:6:13:
      6 │   "exports": {
        ╵              ^

  You can mark the path "axios/lib/utils" as external to exclude it from the bundle, which will
  remove this error.

Oh,my god! Why can't tsup handle so common packages? It seems tsup can't resolve the path axios/lib/*. I don't know what's wrong with it and how to fix it now, can you help me?

tmkx commented 2 years ago

There is a solution:

It's no need to install esbuild-plugins-node-modules-polyfill, just set tsup.platform to browser(default node, I forgot to mention it yesterday)
@vespaiach/axios-fetch-adapter requires ^0.27.2(as devDeps), but it declared a ">=0.26.0" peer range, I guess that you've installed a 1.x version, it's not compatible.

pnpm add axios@^0.27.2 @vespaiach/axios-fetch-adapter -D

 // tsup.config.ts
 export default defineConfig(() => ({
+  platform: 'browser',
 }))

import axios from 'axios'
import fetchAdapter from '@vespaiach/axios-fetch-adapter'

const instance = axios.create({ adapter: fetchAdapter })
instance.get('https://httpbin.org/get').then(res => console.log(res.data))

heavenkiller2018 commented 2 years ago

@tmkx thanks for your suggestion, it's fixed. It's hardly to relate to the compatible problem by the poor prompt! But it's tedious and labourious to be bundled by vite and tsup separately, as some scripts can't be reused for them together (such as import.meta.env). Is there a better alternative that only uses one bundle tool? just like the mv2 template which only use vite.

tmkx commented 2 years ago

background can run as HTML script in MV2, but it should be a service worker in MV3, it's different. take a look at the first refactor commit

heavenkiller2018 commented 2 years ago

@tmkx Problem 6
when I imported an sass file, the error [sass] This file is already being loaded. occurred, but the same codes in MV2 template run well.

add in popup/main.ts

import 'quasar/src/css/index.sass'

errors when bundle:

/home/john/project/testscratch/typescript-demos/bex-demo-mv3/node_modules/.pnpm/sass@1.32.12/node_modules/sass/sass.dart.js:13172
      var error = new self.Error(message);
                  ^

Error: [sass] This file is already being loaded.
  ╷
1 │ @import '/home/john/project/testscratch/typescript-demos/bex-demo-mv3/src/quasar-variables.sass'
  │         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  ╵
  src/quasar-variables.sass 1:9  root stylesheet
    at Object._newRenderError (/home/john/project/testscratch/typescript-demos/bex-demo-mv3/node_modules/.pnpm/sass@1.32.12/node_modules/sass/sass.dart.js:13172:19)
    at Object._wrapException (/home/john/project/testscratch/typescript-demos/bex-demo-mv3/node_modules/.pnpm/sass@1.32.12/node_modules/sass/sass.dart.js:12998:16)
...
    at Object._rootRunBinary (/home/john/project/testscratch/typescript-demos/bex-demo-mv3/node_modules/.pnpm/sass@1.32.12/node_modules/sass/sass.dart.js:4749:18) {
  formatted: 'Error: This file is already being loaded.\n' +
    '  ╷\n' +
    "1 │ @import '/home/john/project/testscratch/typescript-demos/bex-demo-mv3/src/quasar-variables.sass'\n" +
    '  │         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n' +
    '  ╵\n' +
    '  src/quasar-variables.sass 1:9  root stylesheet',
  line: 1,
  column: 9,
  file: '/home/john/project/testscratch/typescript-demos/bex-demo-mv3/src/quasar-variables.sass',
  status: 1,
  id: '/home/john/project/testscratch/typescript-demos/bex-demo-mv3/src/quasar-variables.sass',
  frame: 'Error: This file is already being loaded.\n' +
    '  ╷\n' +
    "1 │ @import '/home/john/project/testscratch/typescript-demos/bex-demo-mv3/src/quasar-variables.sass'\n" +
    '  │         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n' +
    '  ╵\n' +
    '  src/quasar-variables.sass 1:9  root stylesheet',
  plugin: 'vite:css',
  pluginCode: "@import '/home/john/project/testscratch/typescript-demos/bex-demo-mv3/src/quasar-variables.sass'\n" +
    "@import 'quasar/src/css/variables.sass'\n" +
    '\n' +
    '\n' +
    '$primary   : #1976D2\n' +
    '$secondary : #26A69C\n' +
    '$accent    : #9C27B0\n' +
    '\n' +
    '$dark      : #1D1D1D\n' +
    '\n' +
    '$positive  : #21BA45\n' +
    '$negative  : #C10015\n' +
    '$info      : #31CCEC\n' +
    '$warning   : #F2C037',
  loc: {
    file: '/home/john/project/testscratch/typescript-demos/bex-demo-mv3/src/quasar-variables.sass',
    line: 1,
    column: 9
  }
}

Node.js v18.12.0
 ELIFECYCLE  Command failed with exit code 1.
ERROR: "dev:web" exited with 1.
 ELIFECYCLE  Command failed with exit code 1.

what's wrong with it?

tmkx commented 2 years ago

Have you tried this https://quasar.dev/start/vite-plugin?

heavenkiller2018 commented 2 years ago

@tmkx yes, I followed their instructions and modified following files.

pnpm add quasar @quasar/extras @quasar/vite-plugin sass@1.32.12 -D

vite.config.ts

import { quasar, transformAssetUrls } from '@quasar/vite-plugin'
...
export const sharedConfig: UserConfig = {
  root: r('src'),
  resolve: {
    alias: {
      '~/': `${r('src')}/`,
    },
  },
  plugins: [
    Vue({
      template: { transformAssetUrls },
    }),
    quasar({
      sassVariables: r('src/quasar-variables.sass'),
    }),]
...
}
...

src/popup/main.ts
```ts
import { createApp } from 'vue'
// quasar
import { Quasar } from 'quasar'
import iconSet from 'quasar/icon-set/svg-material-icons'
// Import icon libraries
import '@quasar/extras/material-icons/material-icons.css'
import 'quasar/src/css/index.sass'
import App from './Popup.vue'
import '../styles'

const app = createApp(App)
app.use(Quasar, {

  iconSet,
}).mount('#app')

src/quasar-vaiables.sass

$primary   : #1976D2
$secondary : #26A69C
$accent    : #9C27B0

$dark      : #1D1D1D

$positive  : #21BA45
$negative  : #C10015
$info      : #31CCEC
$warning   : #F2C037

then pnpm run dev, the aforementioned errors appear.

I'm confused that for the same codes, why mv2 boilerplate is ok, but mv3 not to bundle sass?

heavenkiller2018 commented 2 years ago

@tmkx , I searched two similar issues which haven't presented an answer yet. [vue.js - Vue CLI 5 with Vuetify SCSS variables and CSS imports - Stack Overflow](https://stackoverflow.com/questions/71709928/vue-cli-5-with-vuetify-scss-variables-and-css-imports)

[javascript - SassError: This file is already being loaded. @import '~src/css/quasar.variables.scss', 'quasar/src/css/variables.sass'; src\css\quasar.variables.scss - Stack Overflow](https://stackoverflow.com/questions/68496522/sasserror-this-file-is-already-being-loaded-import-src-css-quasar-variables)

tmkx commented 2 years ago

it seems that HMR does not work with quasar sass variables. I'm not familiar with quasar, you can use import 'quasar/dist/quasar.css' for dev while import 'quasar/src/css/index.sass' for build? I haven't had much time lately.

heavenkiller2018 commented 2 years ago

It's a good provisional way to resolve this problem ! Thanks,now I can go along it again! Until now, it's a nearly perfect boilerplate for mv3 supporting HMR

heavenkiller2018 commented 2 years ago

@tmkx Problem 7 new apis which are only supported in mv3 can't be typed well in this boilerplate as "@types/webextension-polyfill" only contains mv2 types, is there a better practice to improve it?

tmkx commented 2 years ago

you can declare missing definitions like this:

// shim.d.ts
declare module 'webextension-polyfill' {
  declare namespace Browser {
    const mv3: {
      hello: () => void;
    };
  }

  export = Browser;
}

heavenkiller2018 commented 2 years ago

That should be the last choice because I have to manually written all new-added mv3 type definitions one by one. It's very tedious. Since mv3 has been published from 2020, it's weird that there hasn't yet been an public mv3 types package in community for the past two years.

tmkx commented 2 years ago

There is an official type definitions package, but it's for Chrome only, have a try!

pnpm add @types/chrome -D

 // tsconfig.json
 "types": [
   "vite/client",
+  "chrome"
 ],

chrome.tabGroups.get()

enigmazack commented 1 year ago

Problem1. remove pnpm-lock.yaml and try install again

Problem2. can be solved

Problem3. HMR is unstable in MV3 as expected because the implementation is a little tricky, and that's why it's still not merged into main branch. needs more investigation.

Hi @tmkx, any advance on Prob. 3?

I recently update my project to the mv3 template, most functions works fine for me, but the dev model for vite. It won't hot reload modification of .ts file, like the vuex storage index.ts I used in my project. After changging the index.ts, the shell print

9:38:47 AM [vite] page reload store/index.ts
ESM Build start
ESM extension/dist/mv3client.mjs        38.74 KB
ESM extension/dist/background/index.mjs 100.08 KB
ESM ⚡️ Build success in 12ms

However, the file in the extention dir extention/store/index.ts.js won't chage accodingly, unless I re-run the npm run dev cmd.

I investigate the MV3Hmr vite plugin, but I don't understand how it works.

tmkx commented 1 year ago

HMR is supported in the main branch, closing this issue.

if there still have any problems, please reopen it :)

brauliobo commented 5 months ago

Problem5

background script is not bundled by Vite, there is no import.meta.hot and HMR.

I still have this issue in the main branch, import.meta.hot is undefined

antfu-collective / vitesse-webext