embroider-build / embroider

Compiling Ember apps into spec-compliant, modern Javascript.
MIT License
334 stars 136 forks source link

Multiple copies of the `typescript-memoize` package crashes builds #769

Open sandydoo opened 3 years ago

sandydoo commented 3 years ago

I was playing around with @embroider/macros and noticed that my addon’s dummy app wouldn’t build with Embroider anymore. Here’s the error it throws:

  - stack: TypeError: Cannot redefine property: __memoized_value_2
    at Function.defineProperty (<anonymous>)
    at DummyPackage.<anonymous> (/Users/Sander/Programming/ember-google-maps/node_modules/typescript-memoize/dist/memoize-decorator.js:68:28)
    at /Users/Sander/Programming/ember-google-maps/node_modules/@embroider/shared-internals/src/package.js:156:84
    at arrayMap (/Users/Sander/Programming/ember-google-maps/node_modules/lodash/_arrayMap.js:16:21)
    at map (/Users/Sander/Programming/ember-google-maps/node_modules/lodash/map.js:50:10)
    at Object.flatMap [as default] (/Users/Sander/Programming/ember-google-maps/node_modules/lodash/flatMap.js:26:22)
    at DummyPackage.get dependencies (/Users/Sander/Programming/ember-google-maps/node_modules/@embroider/shared-internals/src/package.js:156:38)
    at DummyPackage.<anonymous> (/Users/Sander/Programming/ember-google-maps/node_modules/typescript-memoize/dist/memoize-decorator.js:67:52)
    at MovedSet.check (/Users/Sander/Programming/ember-google-maps/node_modules/@embroider/compat/src/moved-package-cache.js:228:29)
    at new MovedSet (/Users/Sander/Programming/ember-google-maps/node_modules/@embroider/compat/src/moved-package-cache.js:199:14)

So I started looking into this…

In a nutshell, we’re trying to get packageJSON in DummyPackage. So, @Memoize is going to run the getter function and then cache the value to this.__memoized_value_2.

https://github.com/darrylhodgins/typescript-memoize/blob/6c215d5fe37e2149c0463cfee6a7b5ad908951bd/src/memoize-decorator.ts#L91-L97

The problem is that in the time that we’re “computing” the value, fetching nonResolvableDeps, internalPackageJSON and the like, the context this seems to change. @Memoize then tries to do an Object.define, using the key __memoized_value_2 as it initially intended, but that property already exists on this.

I’ve found two ways to make the bug stop:

  1. Remove at least one @Memoize from either DummyPackage or the parent Package. PackageJSON, internalPackageJSON, or nonResolvableDeps. Doesn’t matter which. This seems to make the cache counters in typescript-memoize happy in this particular case.

  2. Remove @embroider/macros from the addon’s dependencies. This I don’t understand. It happily builds without it. What? Why? How? Any ideas why this would work? Sometimes, you have to nuke node_modules to get it to build again.

This is where I’m at.

I tried recreating a similar scenario in the tests for typescript-memoize. Their tests don’t test class inheritance with a bunch of @Memoizes. I figured it’d be worth a shot, but it hasn’t panned out.

I also started peppering the code with some classic console.logs, so here’s a rough call stack of what happens right up until the crash:

getting internalPackage inside packageJSON
getting internalPackage inside packageJSON
getting nonResolvableDeps inside packageJSON with name `ember-source`
getting nonResolvableDeps inside packageJSON
returning packageJSON with name `ember-source`
getting dependencies
getting internalPackage inside packageJSON
getting internalPackage inside packageJSON
getting internalPackage inside packageJSON
getting nonResolvableDeps inside packageJSON with name `ember-google-maps`
getting nonResolvableDeps inside packageJSON
getting internalPackage inside packageJSON 
getting internalPackage inside packageJSON
getting nonResolvableDeps inside packageJSON with name `in-repo-pin-addon`
getting nonResolvableDeps inside packageJSON
returning packageJSON with name `in-repo-pin-addon`
returning packageJSON with name `ember-google-maps`
getting internalPackageJSON in DummyPackage
deepcloning owningAddon.packageJSON. owningAddon’s name: `ember-google-maps`
getting nonResolvableDeps from packageJSON with name `dummy`
getting nonResolvableDeps inside packageJSON
calling super.nonResolvableDeps in DummyPackage
returning from nonResolvableDeps with name `dummy`
returning packageJSON with name `dummy`
--------------------------------------------------------------------------------
CRASH
--------------------------------------------------------------------------------

I hope this helps somewhat, or at least sparks some ideas of where to look. 👍

Steps to reproduce

I couldn’t reliably reproduce this in a fresh repo, but I’ve made a repro branch for you on ember-google-maps.

  1. Clone https://github.com/sandydoo/ember-google-maps/tree/repro-embroider-memoize-bug
  2. git switch repro-embroider-memoize-bug
  3. ember b

Versions


Updates

1. The counters overlap

On DummyPackage: __memoize_value_1 <== internalPackageJSON __memoize_value_2 <== nonResolvableDeps and then we try to overwrite __memoize_value_2 with packageJSON.

So the counters match the order of the getters in each class:

Order DummyPackage Package
1 internalPackageJSON internalPackageJSON
2 nonResolvableDeps packageJSON

Is this a module thing? Is it because DummyPackage and Package are defined in different packages? Each has it’s own “instance” of typescript-memoize, with separate counters, and it so happens that they overlap between DummyPackage and Package?

2. It‘s a module issue

Using yarn link typescript-memoize to a local repo fixes the issue. So it really is a module issue!

3. It’s a module and YARN thing

Using npm install fixes the issue. npm seems to deduplicate and hoist typescript-memoize as a top-level module, while yarn hoists, but each package still gets its own version. So what do you do if you want to use yarn?

4. It’s a module + yarn + ember-auto-import thing

It seems ember-auto-import is also playing a role here. It builds if you remove it. Shenanigans! Adding typescript-memoize to dependencies on the root embroider workspace also works.

sandydoo commented 3 years ago

TL;DR

DummyPackage and Package do not share the same typescript-memoize dependency in addons that use yarn. This causes @Memoize to generate conflicting/overlapping caching keys, as it uses a top-level, module-scoped counter to increment these keys. This is an issue for DummyPackage and Package, since they both use @Memoize, are both defined in separate packages, and are related by class inheritance. They end up trying to overwrite each other’s caches.

Possible solutions:

  1. Make typescript-memoize a singleton dependency? Adding it as a top-level dependency to Embroider seems to work. Are there downsides to this I‘m not aware of?
  2. ~~Propose a change to typescript-memoize to use a different caching scheme? Perhaps one that uses the property key, instead of a “global” counter. Something like this should fix the issue: https://github.com/sandydoo/typescript-memoize/commit/31527dbafb7fb732219fa8bd88edd4d81549b042~~
simonihmig commented 3 years ago

I just ran into this as well, when adding embroider dependencies to a project with a yarn.lock. After that, you would end up with different versions of typescript-memoize (yarn why typescript-memoize).

However I was able to easily resolve this by running npx yarn-deduplicate!

sandydoo commented 3 years ago

However I was able to easily resolve this by running npx yarn-deduplicate!

Oh, haven’t seen that before.

I’ll try to upstream https://github.com/sandydoo/typescript-memoize/commit/31527dbafb7fb732219fa8bd88edd4d81549b042 on the weekend.

Update

I totally forgot that you can call super. 🤦 Well, that totally breaks my brilliant caching idea. 🤣

muziejus commented 3 years ago

I am getting the same error with the add-on I'm helping with (Cannot redefine property: __memoized_value_2), but this add-on uses npm, not yarn.

I get the error with ember try:one embroider-safe

    stack: TypeError: Cannot redefine property: __memoized_value_2
    at Function.defineProperty (<anonymous>)
    at DummyPackage.<anonymous> (/Users/moacir/Projects/ember-leaflet/node_modules/@embroider/shared-internals/node_modules/typescript-memoize/dist/memoize-decorator.js:101:28)
    at /Users/moacir/Projects/ember-leaflet/node_modules/@embroider/shared-internals/src/package.js:156:84
    at arrayMap (/Users/moacir/Projects/ember-leaflet/node_modules/lodash/_arrayMap.js:16:21)
    at map (/Users/moacir/Projects/ember-leaflet/node_modules/lodash/map.js:50:10)
    at Object.flatMap [as default] (/Users/moacir/Projects/ember-leaflet/node_modules/lodash/flatMap.js:26:22)
    at DummyPackage.get dependencies (/Users/moacir/Projects/ember-leaflet/node_modules/@embroider/shared-internals/src/package.js:156:38)
    at DummyPackage.<anonymous> (/Users/moacir/Projects/ember-leaflet/node_modules/@embroider/shared-internals/node_modules/typescript-memoize/dist/memoize-decorator.js:100:52)
    at MovedSet.check (/Users/moacir/Projects/ember-leaflet/node_modules/@embroider/compat/src/moved-package-cache.js:227:29)
    at new MovedSet (/Users/moacir/Projects/ember-leaflet/node_modules/@embroider/compat/src/moved-package-cache.js:198:14)

I'm new to add-ons, but I added the embroider test-setup utility using the instructions and @embroider/util (I'm trying to squash the component deprecation that ensure-safe-component fixes). If there's a step I'm missing, please let me know.

muziejus commented 3 years ago

running npm i -D typescript-memoize seems to have worked, this being the "add as a singleton" suggestion @sandydoo made. Of course, the ember try still fails all over the place, but at least it's failing tests, not crashing outright.