search

mastodon / mastodon

Your self-hosted, globally interconnected microblogging community
https://joinmastodon.org
GNU Affero General Public License v3.0
47.15k stars 6.99k forks source link

ActionView::Template::Error: Webpacker can't find media/images/preview.png in /home/mastodon/live/public/packs/manifest.json #19881

Closed aswinramakrish closed 2 years ago

aswinramakrish commented 2 years ago

Steps to reproduce the problem

I recently setup my Mastodon instance from Digital Ocean droplet. I originally had version 3.5.3 and it was working well. However, I decided to upgrade to 4.0.0rc1 and have been having this error ever since. Here are the steps I followed

Got the 4.0.0rc1 version

  1. su - mastodon
  2. cd /home/mastodon/live
  3. git fetch --tags
  4. git checkout v4.0.0rc1

Installed dependencies

  1. bundle install
  2. yarn install

Ran the migration and compilation scripts (based on the release notes)

  1. SKIP_POST_DEPLOYMENT_MIGRATIONS=true RAILS_ENV=production bundle exec rails db:migrate
  2. RAILS_ENV=production bundle exec rails assets:precompile
  3. RAILS_ENV=production bundle exec rails db:migrate

Restarted Mastodon services

  1. exit (to exit out of mastodon user)
  2. systemctl restart mastodon-sidekiq
  3. systemctl reload mastodon-web

Expected behaviour

I shouldn't be getting the template / webpacker error

Actual behaviour

I get the Template / Webpacker error when I hit the URL

Detailed description

Here is the actual error -

method=GET path=/ format=html controller=HomeController action=index status=500 

error='ActionView::Template::Error: Webpacker can't find media/images/preview.png in 
/home/mastodon/live/public/packs/manifest.json. 

Possible causes:
1. You want to set webpacker.yml value of compile to true for your environment
unless you are using the `webpack -w` or the webpack-dev-server.
2. webpack has not yet re-run to reflect updates.
3. You have misconfigured Webpacker's config/webpacker.yml file.
4. Your webpack configuration is not creating a manifest.

And then it lists the manifest.json

I looked at this issue - https://github.com/mastodon/mastodon/issues/10926 but the fix doesn't work for me.

This is the first time I'm setting up an instance and I used to be a Rails developer a decade ago, but not anymore. So it's very likely I'm missing something obvious. It would be great if anyone can tell me what I'm missing.

Thanks in advance! 🙏

Specifications

Browsers checked

skiman6010 commented 2 years ago

I had this same issue and I did a couple of things and not sure which one was the one that worked. I made sure that bundler was up to date with gem install bundler and I also checked out the new yarn.lock file with git checkout yarn.lock which I think is what really fixed my error. When I upgraded before I noticed that yarn.lock was having a merge conflict but ignored it. Worth a shot? So basically I double checked the yarn.lock was the same as in the repo and ran through the upgrade steps again (I'm not sure if you need to migrate the database again but I assume not) and make sure you pre-compile again at least!

aswinramakrish commented 2 years ago

Thank you @skiman6010!

yarn.lock was the culprit. However, git checkout yarn.lock didn't work me. I had to remove the yarn.lock file and re-run yarn install. After that I ran all the migration scripts. Unlike before, RAILS_ENV=production bundle exec rails assets:precompile showed the following (which made sense and why it was fixed) -

Compiling...
Compiled all packs in /home/mastodon/live/public/packs
SkaveRat commented 2 years ago

I had this error happen on a completely fresh install.

running RAILS_ENV=production bundle exec rails assets:precompile did fix it for me, although I'm not sure what step in the "Installing from source" documentation went wrong

ShaunGVos commented 2 years ago

I just want to comment that I got the same issue as the original post when upgrading from 3.5.3 to 4.0.2. I tried aswinramakrish suggestion(https://github.com/aswinramakrish) but it didn't work for me. (though I still subsequently deleted the yarn.lock file before running yarn.install following on from this)

I then checked out the code of 4.0.1 and run the migration and compilation scripts again. Mastodon was then running fine but strangely had version 3.5.3 running (I couldn't follow hashtags and it stated v3.5.3 at bottom of page).

I then checked out the code of 4.0.2 and run the migration and compilation scripts again. Mastodon is now running fine with the ability to follow hash tags and states version 4.0.2 is running.

I'm not entirely sure what happened but I thought mentioning what I did maybe useful to others.

kostajh commented 2 years ago

I had the same issue. What worked for me was:

Running yarn install after removing yarn.lock seemed to not trigger the compilation step, which is where I was getting stuck.

JetForMe commented 2 years ago

I'm running into this today trying to update v3.5.3 to v4.0.2, even running the precompile step, even first deleting yarn.lock.

Also, I get a lot of warnings about out-of-date dependencies. I'm not sure if this is to be expected or not:

$ yarn install
yarn install v1.22.19
info No lockfile found.
[1/6] Validating package.json...
[2/6] Resolving packages...
warning babel-plugin-react-intl@6.2.0: this package has been renamed to babel-plugin-formatjs
warning babel-plugin-react-intl > intl-messageformat-parser@4.1.4: We've written a new parser that's 6x faster and is backwards compatible. Please use @formatjs/icu-messageformat-parser
warning babel-plugin-react-intl > intl-messageformat-parser > @formatjs/intl-unified-numberformat@3.3.7: We have renamed the package to @formatjs/intl-numberformat
warning babel-plugin-react-intl > intl-messageformat-parser > @formatjs/intl-unified-numberformat > @formatjs/intl-utils@2.3.0: the package is rather renamed to @formatjs/ecma-abstract with some changes in functionality (primarily selectUnit is removed and we don't plan to make any further changes to this package
warning compression-webpack-plugin > cacache > @npmcli/move-file@1.1.2: This functionality has been moved to @npmcli/fs
warning cssnano > cssnano-preset-default > postcss-svgo > svgo@1.3.2: This SVGO version is no longer supported. Upgrade to v2.x.x.
warning cssnano > cssnano-preset-default > postcss-svgo > svgo > stable@0.1.8: Modern JS already guarantees Array#sort() is a stable sort, so this library is deprecated. See the compatibility table on MDN: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/sort#browser_compatibility
warning intl-messageformat > intl-messageformat-parser@1.4.0: We've written a new parser that's 6x faster and is backwards compatible. Please use @formatjs/icu-messageformat-parser
warning react-intl > intl-relativeformat@2.2.0: This package has been deprecated, please see migration guide at 'https://github.com/formatjs/formatjs/tree/master/packages/intl-relativeformat#migration-guide'
warning tesseract.js > resolve-url@0.2.1: https://github.com/lydell/resolve-url#deprecated
warning twitter-text > core-js@2.6.12: core-js@<3.23.3 is no longer maintained and not recommended for usage due to the number of issues. Because of the V8 engine whims, feature detection in old core-js versions could cause a slowdown up to 100x even if nothing is polyfilled. Some versions have web compatibility issues. Please, upgrade your dependencies to the actual version of core-js.
warning webpack > watchpack > watchpack-chokidar2 > chokidar@2.1.8: Chokidar 2 does not receive security updates since 2019. Upgrade to chokidar 3 with 15x fewer dependencies
warning webpack > watchpack > watchpack-chokidar2 > chokidar > fsevents@1.2.13: fsevents 1 will break on node v14+ and could be using insecure binaries. Upgrade to fsevents 2.
warning webpack > micromatch > snapdragon > source-map-resolve@0.5.3: See https://github.com/lydell/source-map-resolve#deprecated
warning webpack > micromatch > snapdragon > source-map-resolve > resolve-url@0.2.1: https://github.com/lydell/resolve-url#deprecated
warning webpack > node-libs-browser > url > querystring@0.2.0: The querystring API is considered Legacy. new code should use the URLSearchParams API instead.
warning webpack > micromatch > snapdragon > source-map-resolve > source-map-url@0.4.1: See https://github.com/lydell/source-map-url#deprecated
warning webpack > micromatch > snapdragon > source-map-resolve > urix@0.1.0: Please see https://github.com/lydell/urix#deprecated
warning webpack-dev-server > chokidar@2.1.8: Chokidar 2 does not receive security updates since 2019. Upgrade to chokidar 3 with 15x fewer dependencies
warning webpack-dev-server > webpack-log > uuid@3.4.0: Please upgrade  to version 7 or higher.  Older versions may use Math.random() in certain circumstances, which is known to be problematic.  See https://v8.dev/blog/math-random for details.
[3/6] Fetching packages...
[4/6] Linking dependencies...
warning Workspaces can only be enabled in private projects.
[5/6] Building fresh packages...
[6/6] Cleaning modules...
success Saved lockfile.
Done in 37.73s.
SkaveRat commented 2 years ago

I ran into this problem again when updating my 3.5 instance to 4.0.2. The solution in my case: not enough memory.

The compilation of the assets used up all of the 2GB memory of my small instance VPS.

Adding more swap helped in my case

JetForMe commented 2 years ago

Update: I saw @SkaveRat's comment about memory after I originally wrote this one. Adding 4G of swap to my 2GB instance and re-precompiling assets (as well as doing assets:clobber) fixed the problems for me.

--

Hmm, a few files reference preview.png, but the manifest has a preview-XXX.jpg:

$ grep "preview" public/packs/manifest.json
  "media/images/preview.jpg": {
    "src": "/packs/media/images/preview-5df98290371ead9a70bc3cd4733bbfa7.jpg",
$ ag preview.png
app/views/application/_sidebar.html.haml
3:    = image_tag @instance_presenter.thumbnail&.file&.url(:'@1x') || asset_pack_path('media/images/preview.png'), alt: @instance_presenter.title

app/views/shared/_og.html.haml
11:= opengraph 'og:image', full_asset_url(thumbnail&.file&.url(:'@1x') || asset_pack_path('media/images/preview.png', protocol: :request))

app/serializers/rest/v1/instance_serializer.rb
36:    instance_presenter.thumbnail ? full_asset_url(instance_presenter.thumbnail.file.url(:'@1x')) : full_pack_url('media/images/preview.png')

app/serializers/rest/instance_serializer.rb
31:        url: full_pack_url('media/images/preview.png'),
kostajh commented 2 years ago

I ran into this problem again when updating my 3.5 instance to 4.0.2. The solution in my case: not enough memory.

The compilation of the assets used up all of the 2GB memory of my small instance VPS.

Adding more swap helped in my case

That was part of my motivation for my pull request in mastodon/documentation/pull/1000, fwiw :)

masukomi commented 1 year ago

went through all the steps, hoped that @SkaveRat was wrong because needing that much ram seems just ridiculous.

Reader, they were not wrong. If you're doing this on Digital Ocean with a 2GB RAM droplet, like some of us, then the answer is as follows

su - mastodon
cd ~/live

# probably unnecessary but won't hurt...
git checkout yarn.lock

# clobber the precompiled assets because it probably thinks everything's fine with them
RAILS_ENV=production bundle exec rails assets:clobber
RAILS_ENV=production bundle exec rails assets:precompile

# switch back to the root user
exit

# restart all the mastodon services
systemctl restart mastodon-sidekiq
systemctl reload mastodon-web
# probably unnecessary but won't hurt...
systemctl restart mastodon-streaming

that should fix it

now, if you want to go back to saving money because you don't have many active users

posmoda commented 1 year ago

This works for me. Many thanks!!

risingsunomi commented 1 year ago

I would like to add I was upgrading from 3.5 instance to 4.0.2 and at the

RAILS_ENV=production bundle exec rails assets:precompile

step was getting this error

yarn install v1.22.19 [1/6] Validating package.json... [2/6] Resolving packages... [3/6] Fetching packages... [4/6] Linking dependencies... warning Workspaces can only be enabled in private projects. [5/6] Building fresh packages... [6/6] Cleaning modules... Done in 22.59s. I, [2022-12-12T06:18:51.386298 #340876] INFO -- : Writing /home/mastodon/live/public/assets/doorkeeper/admin/application-a644908e7bab54fb749be0f59fb64a7480bbf9c4c2b79d4a65791cb7ab4d8730.css I, [2022-12-12T06:18:51.386778 #340876] INFO -- : Writing /home/mastodon/live/public/assets/doorkeeper/admin/application-a644908e7bab54fb749be0f59fb64a7480bbf9c4c2b79d4a65791cb7ab4d8730.css.gz I, [2022-12-12T06:18:51.396726 #340876] INFO -- : Writing /home/mastodon/live/public/assets/doorkeeper/application-c93dac2ad9d65e3393e0e2c958481e86ef7a5e5b0f6ce406842a7b99b25a4850.css I, [2022-12-12T06:18:51.396992 #340876] INFO -- : Writing /home/mastodon/live/public/assets/doorkeeper/application-c93dac2ad9d65e3393e0e2c958481e86ef7a5e5b0f6ce406842a7b99b25a4850.css.gz I, [2022-12-12T06:18:51.421029 #340876] INFO -- : Writing /home/mastodon/live/public/assets/pghero/favicon-db10337a56c45eb43c22ff5019546b520fa22c7281d4d385f235cbca67ed26bb.png I, [2022-12-12T06:18:51.444655 #340876] INFO -- : Writing /home/mastodon/live/public/assets/pghero/application-a60bf0a452ed064fef3594cf52a4c998712da7c76150f890f4eaa644f59671e4.js I, [2022-12-12T06:18:51.444975 #340876] INFO -- : Writing /home/mastodon/live/public/assets/pghero/application-a60bf0a452ed064fef3594cf52a4c998712da7c76150f890f4eaa644f59671e4.js.gz I, [2022-12-12T06:18:51.469550 #340876] INFO -- : Writing /home/mastodon/live/public/assets/pghero/application-c31338f656687c1d733bb0f48d40acd076e24060f3dcff83b34870e4ccc2789d.css I, [2022-12-12T06:18:51.469788 #340876] INFO -- : Writing /home/mastodon/live/public/assets/pghero/application-c31338f656687c1d733bb0f48d40acd076e24060f3dcff83b34870e4ccc2789d.css.gz Compiling... Compilation failed: node:internal/crypto/hash:71 this[kHandle] = new _Hash(algorithm, xofLen); ^

Error: error:0308010C:digital envelope routines::unsupported at new Hash (node:internal/crypto/hash:71:19) at Object.createHash (node:crypto:133:10) at module.exports (/home/mastodon/live/node_modules/webpack/lib/util/createHash.js:135:53) at ConcatenatedModule._createIdentifier (/home/mastodon/live/node_modules/webpack/lib/optimize/ConcatenatedModule.js:563:16) at new ConcatenatedModule (/home/mastodon/live/node_modules/webpack/lib/optimize/ConcatenatedModule.js:445:27) at /home/mastodon/live/node_modules/webpack/lib/optimize/ModuleConcatenationPlugin.js:250:26 at SyncBailHook.eval [as call] (eval at create (/home/mastodon/live/node_modules/tapable/lib/HookCodeFactory.js:19:10), :7:16) at SyncBailHook.lazyCompileHook (/home/mastodon/live/node_modules/tapable/lib/Hook.js:154:20) at /home/mastodon/live/node_modules/webpack/lib/Compilation.js:1351:37 at AsyncSeriesHook.eval [as callAsync] (eval at create (/home/mastodon/live/node_modules/tapable/lib/HookCodeFactory.js:33:10), :6:1) at AsyncSeriesHook.lazyCompileHook (/home/mastodon/live/node_modules/tapable/lib/Hook.js:154:20) at Compilation.seal (/home/mastodon/live/node_modules/webpack/lib/Compilation.js:1342:27) at /home/mastodon/live/node_modules/webpack/lib/Compiler.js:675:18 at /home/mastodon/live/node_modules/webpack/lib/Compilation.js:1261:4 at AsyncSeriesHook.eval [as callAsync] (eval at create (/home/mastodon/live/node_modules/tapable/lib/HookCodeFactory.js:33:10), :24:1) at AsyncSeriesHook.lazyCompileHook (/home/mastodon/live/node_modules/tapable/lib/Hook.js:154:20) at Compilation.finish (/home/mastodon/live/node_modules/webpack/lib/Compilation.js:1253:28) at /home/mastodon/live/node_modules/webpack/lib/Compiler.js:672:17 at eval (eval at create (/home/mastodon/live/node_modules/tapable/lib/HookCodeFactory.js:33:10), :11:1) at /home/mastodon/live/node_modules/webpack/lib/Compilation.js:1185:12 at /home/mastodon/live/node_modules/webpack/lib/Compilation.js:1097:9 at process.processTicksAndRejections (node:internal/process/task_queues:77:11) { opensslErrorStack: [ 'error:03000086:digital envelope routines::initialization error' ], library: 'digital envelope routines', reason: 'unsupported', code: 'ERR_OSSL_EVP_UNSUPPORTED' }

Node.js v18.12.1

I found that using

export NODE_OPTIONS=--openssl-legacy-provider

before the precompile fixed this issues and allowed things to finish

yarn install v1.22.19 [1/6] Validating package.json... [2/6] Resolving packages... success Already up-to-date. Done in 0.88s. I, [2022-12-12T06:21:38.254936 #341487] INFO -- : Writing /home/mastodon/live/public/assets/doorkeeper/admin/application-a644908e7bab54fb749be0f59fb64a7480bbf9c4c2b79d4a65791cb7ab4d8730.css I, [2022-12-12T06:21:38.255447 #341487] INFO -- : Writing /home/mastodon/live/public/assets/doorkeeper/admin/application-a644908e7bab54fb749be0f59fb64a7480bbf9c4c2b79d4a65791cb7ab4d8730.css.gz I, [2022-12-12T06:21:38.259919 #341487] INFO -- : Writing /home/mastodon/live/public/assets/doorkeeper/application-c93dac2ad9d65e3393e0e2c958481e86ef7a5e5b0f6ce406842a7b99b25a4850.css I, [2022-12-12T06:21:38.260153 #341487] INFO -- : Writing /home/mastodon/live/public/assets/doorkeeper/application-c93dac2ad9d65e3393e0e2c958481e86ef7a5e5b0f6ce406842a7b99b25a4850.css.gz I, [2022-12-12T06:21:38.261462 #341487] INFO -- : Writing /home/mastodon/live/public/assets/pghero/favicon-db10337a56c45eb43c22ff5019546b520fa22c7281d4d385f235cbca67ed26bb.png I, [2022-12-12T06:21:38.271570 #341487] INFO -- : Writing /home/mastodon/live/public/assets/pghero/application-a60bf0a452ed064fef3594cf52a4c998712da7c76150f890f4eaa644f59671e4.js I, [2022-12-12T06:21:38.271910 #341487] INFO -- : Writing /home/mastodon/live/public/assets/pghero/application-a60bf0a452ed064fef3594cf52a4c998712da7c76150f890f4eaa644f59671e4.js.gz I, [2022-12-12T06:21:38.276100 #341487] INFO -- : Writing /home/mastodon/live/public/assets/pghero/application-c31338f656687c1d733bb0f48d40acd076e24060f3dcff83b34870e4ccc2789d.css I, [2022-12-12T06:21:38.276353 #341487] INFO -- : Writing /home/mastodon/live/public/assets/pghero/application-c31338f656687c1d733bb0f48d40acd076e24060f3dcff83b34870e4ccc2789d.css.gz Compiling... Compiled all packs in /home/mastodon/live/public/packs

digitalcircuit commented 1 year ago

@risingsunomi

I would like to add I was upgrading from 3.5 instance to 4.0.2 and at the

RAILS_ENV=production bundle exec rails assets:precompile

step was getting this error

[…trimmed quote…]

Thank you for sharing this! It resolved the issue I had with NodeJS 18 when upgrading to Mastodon v4.2.0-beta3.

Fortunately, as of 1 hour ago, Mastodon has updated webpack to v.4.47.0 in the latest commit. Note that this is not in beta3!

This is specifically noted as fixing the need to do this workaround: https://github.com/webpack/webpack/pull/17628#issuecomment-1707790411

vmstan commented 1 year ago

In testing, it has not successfully compiled yet with the updated Webpack on Node 18, yet.

The compression plugin might also need updating.

tnsasse commented 1 year ago

Deleting yarn.lock, running yarn install, and RAILS_ENV=production bundle exec rails assets:precompile worked for me upgrading to v4.2.0 on RHEL9

graham73may commented 1 year ago

Also just had this upgrading to 4.2.0 on a mini 1GB RAM (2GB swap) server.

Had to:

  1. RAILS_ENV=production bundle exec rails assets:clobber
  2. Follow these instructions: https://www.mattburkedev.com/adventures-in-mastodon-self-hosting-upgrade-to-4-1-0rc1/

image

  1. RAILS_ENV=production bundle exec rails assets:precompile

Error I was getting: 

Compilation failed:
warning ../../../package.json: No license field
`isModuleDeclaration` has been deprecated, please migrate to `isImportOrExportDeclaration`
    at isModuleDeclaration (/home/mastodon/live/node_modules/babel-plugin-lodash/node_modules/@babel/types/lib/validators/generated/index.js:2740:35)
    at PluginPass.Program (/home/mastodon/live/node_modules/babel-plugin-lodash/lib/index.js:102:44)

<--- Last few GCs --->

[20579:0x61d11d0]   108691 ms: Mark-sweep 452.9 (469.4) -> 448.9 (469.6) MB, 1065.8 / 0.0 ms  (average mu = 0.139, current mu = 0.065) allocation failure scavenge might not succeed
[20579:0x61d11d0]   109831 ms: Mark-sweep 453.1 (469.6) -> 449.4 (470.1) MB, 1073.3 / 0.0 ms  (average mu = 0.099, current mu = 0.059) allocation failure scavenge might not succeed

<--- JS stacktrace --->

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
 1: 0xb090e0 node::Abort() [node]
 2: 0xa1b70e  [node]
 3: 0xce1a20 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [node]
 4: 0xce1dc7 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [node]
 5: 0xe99435  [node]
 6: 0xe99f16  [node]
 7: 0xea843e  [node]
 8: 0xea8e80 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [node]
 9: 0xeabdfe v8::internal::Heap::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [node]
10: 0xe6d33a v8::internal::Factory::NewFillerObject(int, bool, v8::internal::AllocationType, v8::internal::AllocationOrigin) [node]
11: 0x11e61e6 v8::internal::Runtime_AllocateInYoungGeneration(int, unsigned long*, v8::internal::Isolate*) [node]
12: 0x15d9e59  [node]