Browser report fails when `node_modules` are read-only

singles commented 6 years ago

Description of a problem

In environments where node_modules are read-only browser report isn't created.

Examples of such environments: nix based builds, where node_modules land in /nix/store as read-only (555 or 444 rights). Another one might be some stateless build systems, where node_modules is passed between build steps as an artifact and also might be read-only.

How to reproduce:

mkdir test && cd test
npm init -y && npm install backstopjs
./node_modules/.bin/backstop init
chmod -w ./node_modules/.bin/backstop/compare/output # simulate read-only case
./node_modules/.bin/backstop test
...

report | 2 Failed
       report | Writing browser report
       report | Failed writing report with error: undefined # <-- this

Potential cause

I dug a little into the internals and it looks that it's caused by https://github.com/garris/BackstopJS/blob/ddfedcbd2d63fd6108922bf0bfabc31d672fd5c1/core/command/report.js#L26

where fs.copy copies output with directory rights. Assuming I'm not wrong about how *nix systems handle permissions, in presented case, there's no write for output directory - thus no new directories html_report directory on the project side can be created as part of copy.

Workarounds

One workaround is to create directory structure before running tests, something like:

rm -rf backstop_data/html_report && mkdir -p backstop_data/html_report/fonts/assets

Summary

Not sure whether it should be considered as a bug, I just wanted to point out that current implementation might be causing some issues.

lo1tuma commented 6 years ago

I’ve stumbled upon the same problem. Additionally when creating the report fails the process exits with 0, which made our CI pipeline silently pass, even though the tests were actually failing.

lo1tuma commented 6 years ago

This might be an issue of fs-extra which preservers the permissions of copied files.

XjSv commented 2 years ago

I am also just running into this problem when packaging in using Electron. The files get packaged in \release\build\win-ia32-unpacked\resources\app.asar\node_modules\backstopjs\compare\output which is a read-only archive.

XjSv commented 2 years ago

I am using the code from #1400 for my Electron project and I am passing the comparePath in my config like so:

"comparePath": ${backstopDataDir}/compare/output,

It seems to be working at the moment.

garris / BackstopJS