Closed tpraxl closed 4 years ago
grgur
commented
4 years ago Thanks for creating this issue @tpraxl
What's the path to your application ~sources~ build directory? e.g. build/ or something else?
If you have a gimbalrc file, can you post it here?
tpraxl
commented
4 years ago In the use case above, it was build/. Please try to reproduce it as described. Maybe your system yields another result and that would lead us on the right track.
In the other project, I had it all:
.gimbalrc.yml.gimbalrc.yml and gimbal --config=.gimbalrc.yml auditpublic and configured using
gimbal audit --build-dir publicbuildI tested both .gimbalrc.yml files:
Your example:
configs:
heap-snapshot:
threshold:
Documents: 5
Frames: 2
JSHeapTotalSize: 23356000
JSHeapUsedSize: 15068000
Nodes: 800
RecalcStyleCount: 9
lighthouse:
threshold:
accessibility: 93
'best-practices': 90
performance: 50
pwa: 75
seo: 100
size:
- path: ./build/precache-*.js
maxSize: 500 B
- path: ./build/static/js/*.chunk.js
maxSize: 1 MB
- path: ./build/static/js/runtime*.js
maxSize: 10 KB
- path: ./build/
maxSize: 1 Band a custom file (the exact configuration changed massively during testing):
configs:
puppeteer:
headless: true
lighthouse:
outputHtml: reports/lighthouse.html
treshold:
accessibility: 100
'best-practices': 100
performance: 100
pwa: 100
sweo: 100
size:
treshold:
- path: ./public/css/
maxSize: 1 B
- path: ./public/js/
maxSize: 1 B
- path: ./public/
maxSize: 2 B
unused-source:
treshold:
- path: public/css/
type: css
maxSize: 25%
- path: public/js/
type: js
maxSize: 25%
outputs:
html: ./reports/gimbal.html
json: ./reports/gimbal.json
markdown: ./reports/gimbal.md
jobs:
- audit
mitchellsimoens
commented
4 years ago @tpraxl thank you for opening a bug and thank you for trying Gimbal by Modus Labs!
Trying your steps in the OP, I get such results:
➜ gimbal-problem gimbal audit
___ ___ ___
/\__\ /\ \ _____ /\ \
/:/ _/_ ___ |::\ \ /::\ \ /::\ \
/:/ /\ \ /\__\ |:|:\ \ /:/\:\ \ /:/\:\ \
/:/ /::\ \ /:/__/ __|:|\:\ \ /:/ /::\__\ /:/ /::\ \ ___ ___
/:/__\/\:\__\ /::\ \ /::::|_\:\__\ /:/_/:/\:|__| /:/_/:/\:\__\ /\ \ /\__\
\:\ \ /:/ / \/\:\ \__ \:\~~\ \/__/ \:\/:/ /:/ / \:\/:/ \/__/ \:\ \ /:/ /
\:\ /:/ / \:\/\__\ \:\ \ \::/_/:/ / \::/__/ \:\ /:/ /
\:\/:/ / \::/ / \:\ \ \:\/:/ / \:\ \ \:\/:/ /
\::/ / /:/ / \:\__\ \::/ / \:\__\ \::/ /
\/__/ \/__/ \/__/ \/__/ \/__/ \/__/
_ __ __ _ ____ _
| |__ _ _ | \/ | ___ __| | _ _ ___ / ___| _ __ ___ __ _ | |_ ___
| '_ \ | | | | | |\/| | / _ \ / _` | | | | | / __| | | | '__| / _ \ / _` | | __| / _ \
| |_) | | |_| | | | | | | (_) | | (_| | | |_| | \__ \ | |___ | | | __/ | (_| | | |_ | __/
|_.__/ \__, | |_| |_| \___/ \__,_| \__,_| |___/ \____| |_| \___| \__,_| \__| \___|
|___/
─────────────────────────────────────────────────────────────────────────────────────────────────────
┌────────────────────────────────────────────────────────┬──────────┬───────────┬─────────┐
│ Label │ Value │ Threshold │ Success │
├────────────────────────────────────────────────────────┴──────────┴───────────┴─────────┤
│ Size Checks [ success: x ] │
├────────────────────────────────────────────────────────┬──────────┬───────────┬─────────┤
│ build/index.html │ 14.54 KB │ 3 KB │ x │
├────────────────────────────────────────────────────────┼──────────┼───────────┼─────────┤
│ build │ 69.19 KB │ 500 KB │ ✓ │
├────────────────────────────────────────────────────────┴──────────┴───────────┴─────────┤
│ │
├─────────────────────────────────────────────────────────────────────────────────────────┤
│ Heap Snapshot Checks [ success: x ] │
├────────────────────────────────────────────────────────┬──────────┬───────────┬─────────┤
│ Documents │ 6 │ 5 │ x │
├────────────────────────────────────────────────────────┼──────────┼───────────┼─────────┤
│ Frames │ 2 │ 2 │ ✓ │
├────────────────────────────────────────────────────────┼──────────┼───────────┼─────────┤
│ JSHeapTotalSize │ 10092544 │ │ ✓ │
├────────────────────────────────────────────────────────┼──────────┼───────────┼─────────┤
│ JSHeapUsedSize │ 5140800 │ │ ✓ │
├────────────────────────────────────────────────────────┼──────────┼───────────┼─────────┤
│ LayoutCount │ 5 │ 5 │ ✓ │
├────────────────────────────────────────────────────────┼──────────┼───────────┼─────────┤
│ Nodes │ 1700 │ 75 │ x │
├────────────────────────────────────────────────────────┼──────────┼───────────┼─────────┤
│ RecalcStyleCount │ 7 │ 5 │ x │
├────────────────────────────────────────────────────────┴──────────┴───────────┴─────────┤
│ │
├─────────────────────────────────────────────────────────────────────────────────────────┤
│ Unused Source Checks [ success: x ] │
├────────────────────────────────────────────────────────┬──────────┬───────────┬─────────┤
│ http://localhost:3000/ │ 90.07% │ │ ✓ │
├────────────────────────────────────────────────────────┼──────────┼───────────┼─────────┤
│ ... ts/frameworks-2e9090135c22aad5f56c2f72dcba7880.css │ 92.21% │ 35% │ x │
├────────────────────────────────────────────────────────┼──────────┼───────────┼─────────┤
│ ... m/assets/site-789f905d50a214e0c8606578148aa830.css │ 100.00% │ 35% │ x │
├────────────────────────────────────────────────────────┼──────────┼───────────┼─────────┤
│ ... assets/github-7265230e39dc9980c60a5c0a9785286a.css │ 98.15% │ 35% │ x │
├────────────────────────────────────────────────────────┼──────────┼───────────┼─────────┤
│ ... thubassets.com/assets/compat-bootstrap-90c0ace0.js │ 86.29% │ 70% │ x │
├────────────────────────────────────────────────────────┼──────────┼───────────┼─────────┤
│ ... hub.githubassets.com/assets/frameworks-86315523.js │ 74.35% │ 70% │ x │
├────────────────────────────────────────────────────────┼──────────┼───────────┼─────────┤
│ ... thubassets.com/assets/github-bootstrap-1d407193.js │ 85.99% │ 70% │ x │
├────────────────────────────────────────────────────────┴──────────┴───────────┴─────────┤
│ │
├─────────────────────────────────────────────────────────────────────────────────────────┤
│ Lighthouse Audits [ success: x ] │
├────────────────────────────────────────────────────────┬──────────┬───────────┬─────────┤
│ Performance │ 91 │ 50 │ ✓ │
├────────────────────────────────────────────────────────┼──────────┼───────────┼─────────┤
│ Accessibility │ 96 │ 75 │ ✓ │
├────────────────────────────────────────────────────────┼──────────┼───────────┼─────────┤
│ Best Practices │ 85 │ 95 │ x │
├────────────────────────────────────────────────────────┼──────────┼───────────┼─────────┤
│ SEO │ 90 │ 90 │ ✓ │
├────────────────────────────────────────────────────────┼──────────┼───────────┼─────────┤
│ Progressive Web App │ 40 │ 50 │ x │
└────────────────────────────────────────────────────────┴──────────┴───────────┴─────────┘
➜ gimbal-problem gimbal --version
___ ___ ___
/\__\ /\ \ _____ /\ \
/:/ _/_ ___ |::\ \ /::\ \ /::\ \
/:/ /\ \ /\__\ |:|:\ \ /:/\:\ \ /:/\:\ \
/:/ /::\ \ /:/__/ __|:|\:\ \ /:/ /::\__\ /:/ /::\ \ ___ ___
/:/__\/\:\__\ /::\ \ /::::|_\:\__\ /:/_/:/\:|__| /:/_/:/\:\__\ /\ \ /\__\
\:\ \ /:/ / \/\:\ \__ \:\~~\ \/__/ \:\/:/ /:/ / \:\/:/ \/__/ \:\ \ /:/ /
\:\ /:/ / \:\/\__\ \:\ \ \::/_/:/ / \::/__/ \:\ /:/ /
\:\/:/ / \::/ / \:\ \ \:\/:/ / \:\ \ \:\/:/ /
\::/ / /:/ / \:\__\ \::/ / \:\__\ \::/ /
\/__/ \/__/ \/__/ \/__/ \/__/ \/__/
_ __ __ _ ____ _
| |__ _ _ | \/ | ___ __| | _ _ ___ / ___| _ __ ___ __ _ | |_ ___
| '_ \ | | | | | |\/| | / _ \ / _` | | | | | / __| | | | '__| / _ \ / _` | | __| / _ \
| |_) | | |_| | | | | | | (_) | | (_| | | |_| | \__ \ | |___ | | | __/ | (_| | | |_ | __/
|_.__/ \__, | |_| |_| \___/ \__,_| \__,_| |___/ \____| |_| \___| \__,_| \__| \___|
|___/
─────────────────────────────────────────────────────────────────────────────────────────────────────
1.1.4However, I'm using mac so I'm hoping today I can start a Mint VM and test from there as maybe it's either a Mint/Linux issue or localized to something on your machine. Thank you for trying with a gimbal config file to see if that helps and any continued debugging!
tpraxl
commented
4 years ago Where does http://localhost:3000/ in unused source checks come from? Is anything special expected from the system, like a running server or something?
Thanks for testing it inside a Mint VM! I appreciate that. Looking forward to your results.
mitchellsimoens
commented
4 years ago The file protocol could prevent certain things from happening so we launch a simple static server (using serve-handler, link to gimbal code here if interested) to host the build directory. So that line in the result is reported by puppeteer on the index.html page.
tpraxl
commented
4 years ago FYI: I cloned this project and ran the tests. Tests seem to pass:
$ yarn test
yarn run v1.17.3
$ lerna run test --stream
lerna notice cli v3.16.4
lerna info Executing command in 6 packages: "yarn run test"
[… shortened …]
lerna success run Ran npm script 'test' in 6 packages in 17.1s:
lerna success - @modus/gimbal-core
lerna success - @modus/gimbal-plugin-axe
lerna success - @modus/gimbal-plugin-last-value
lerna success - @modus/gimbal-plugin-mysql
lerna success - @modus/gimbal-plugin-source-map-explorer
lerna success - @modus/gimbal-plugin-sqlite
Done in 17.52s.Any progress? Does it work on your Mint VM?
mitchellsimoens
commented
4 years ago Hey sorry, been a little bogged down with a client release coming up soon. I did spin up a Linux Mint 18.3 and was able to reproduce so I'll be working in that VM to use dev version to track down the missing link.
tpraxl
commented
4 years ago Oh great. I'm looking forward to the fix! Thanks, good luck and happy debugging :)
mitchellsimoens
commented
4 years ago @tpraxl Good news, we actually had another internal project that was hitting this error so I had help from @theneverstill at least figuring out how to get something to work.
First, I got this to work on Linux Mint 18.3. I created a .gimbalrc.yml with:
audits:
- heap-snapshot
- lighthouse
- size
- unused-sourceAnd then just ran gimbal (not gimbal audit) and I got the tables of each audit and failures and such.
So that leads to the error that was found, for some reason executing gimbal audit directly isn't working as it is supposed. This is interesting to me since when the config file has that audits array and you execute gimbal, internally it actually executes the gimbal audit command (direct code here).
Can you at least confirm the work around? I only did the original steps (where you wget the index.html) and then added the above .gimbalrc.yml and ran gimbal instead. Even if you do confirm the workaround at least works I'll keep this issue open to investigate why running gimbal audit directly doesn't "just work".
tpraxl
commented
4 years ago Great. I can confirm that the workaround works, given your build dir is called build. Just running gimbal with the above .gimbalrc.yml produces reports.Thanks!
Is there any way to configure the build dir in .gimbalrc.yml? gimbal --build-dir public won't work, it needs to be gimbal audit --build-dir public but that doesn't work currently.
mitchellsimoens
commented
4 years ago Thank you for verifying that!
Yes, you can pass the build directory as a config using the buildDir config as seen in this angular example. In fact, it was a relatively recent release that allowed any cli option to be used as a config (removing the dash and capitalizing the next letter hence --build-dir works with buildDir config).
tpraxl
commented
4 years ago Great! Thanks. This seems to work.
Now, there's another weird effect (maybe this should go into another issue?):
If you have a build dir and you don't specify a buildDir, then the output is as expected:
┌─────────────────────────────────┬──────────┬───────────┬─────────┐
│ Label │ Value │ Threshold │ Success │
├─────────────────────────────────┴──────────┴───────────┴─────────┤
│ Size Checks [ success: ✓ ] │
├─────────────────────────────────┬──────────┬───────────┬─────────┤
│ build/index.html │ 267 B │ 3 KB │ ✓ │
├─────────────────────────────────┼──────────┼───────────┼─────────┤
│ build │ 27.29 KB │ 500 KB │ ✓ │If you specify a buildDir: public instead of the default buildDir: build, as in:
configs:
buildDir: publicand there is still a directory called build, then the output is unexpected
┌─────────────────────────────────┬──────────┬───────────┬─────────┐
│ Label │ Value │ Threshold │ Success │
├─────────────────────────────────┴──────────┴───────────┴─────────┤
│ Size Checks [ success: ✓ ] │
├─────────────────────────────────┬──────────┬───────────┬─────────┤
│ build/index.html │ 267 B │ 3 KB │ ✓ │
├─────────────────────────────────┼──────────┼───────────┼─────────┤
│ build │ 27.29 KB │ 500 KB │ ✓ │
├─────────────────────────────────┴──────────┴───────────┴─────────┤If there's no build directory and you specify buildDir: public, then parts of the output are missing:
┌─────────────────────────────────┬─────────┬───────────┬─────────┐
│ Label │ Value │ Threshold │ Success │
├─────────────────────────────────┴─────────┴───────────┴─────────┤
│ Size Checks [ success: ✓ ] │
├─────────────────────────────────────────────────────────────────┤
│ │If you specify buildDir and that directory does exist (it should throw an error if it doesn't exist) then it will run the audits on it. If there are no files matching for an audit (like the size audit as pictured), there currently is no logic to error out so I'd expect an empty table and a success. If you feel otherwise, I'd open as a new GitHub issue as an enhancement and we can discuss separate of this gimbal audit error.
tpraxl
commented
4 years ago I have opened a new issue for it: #145
grgur
commented
4 years ago Thanks for resolving this @mitchellsimoens
mitchellsimoens
commented
4 years ago @tpraxl I had a couple minutes to debug this (sorry it's been a month). Looks like it's because gimbal is very async but we are using commander.js to parse cli and commander.js itself does not support async at all. So when commander does the executing, the process will continue regardless of the async/await in gimbal and therefore does not have time to execute anything. This is due to when we added ability to execute with config file, that being the preferred method tho I liked the idea of keeping the individual commands around.
So couple ways to solve this:
gimbal audit usage (altho if you visit --help, commander.js will show it but you can take over --help). I don't like this as it's only half solved IMO.mitchellsimoens
commented
4 years ago Been a loooong road but got gimbal audit to work. Commander is really just a cli arg parser now and I run the audit outside of commander. In fact, running gimbal now will run an audit just as gimbal audit would regardless if you have a config file. This is deployed as v1.2.6.
Describe the bug When running
gimbal auditfor a web project, nothing is reported.To Reproduce Steps to reproduce the behavior:
npm install -g @modus/gimbalmkdir gimbal-problemcd gimbal-problemmkdir buildwget https://github.com/ModusCreateOrg/gimbal/tree/master/packages/gimbal/docs/module/size -O build/index.html(should be around 70KB)gimbal auditExpected behavior Some kind of report. Especially since index.html busts the default size treshold of
build/index.html.Screenshots
The same applies in an actual web project, regardless of several different configurations.
Desktop (please complete the following information):
Additional context