Possibly inaccurate reporting on latest imports

[nickautomatic]

Hello. I've been playing with Project Wallace, and I think it looks like a really excellent tool, but I've run into a puzzling issue.

For a while imports weren't working at all on this project. That issue seems to be fixed now - I was able to import today - but the stats now seem to be off: it thinks that the CSS has increased in size by ~500% since the last time I was able to import successfully (on 11th Feb). It reports a file size of 783kB, but I can't see where it's getting this figure from: if I visit the page directly it looks like the actual CSS size is nowhere near this size. If you compare other stats (selectors, rules, etc), they've also increased by similarly large amounts since the previous import (11th Feb).

At first I thought this increase in size might indicate a real problem, but after investigating I'm now wondering if the same file is being analysed multiple times or something?

[bartveneman]

Hey, thanks for opening this issue. I'll give a couple of things that I think are at play:

• An embedded font-file was found in the CSS, increasing the filesize quite a bit: https://www.projectwallace.com/~nickf/futurelearn/imports/20190326110813500#atrules.fontfaces.unique
• Project Wallace uses extract-css.now.sh under the hood, and I've been updating that quite a lot lately, so it has become better at finding CSS because it now uses an actual Chromium browser to render the page and get all the CSS. Pretty much all projects on PW have seen significant increases in amounts of CSS found, because the scraping mechanism is so much better now. Check out the results of extract-css for FutureLearn
• The network panel in Firefox shows me that there is 917.21 KB (raw bytes) / 328.57 KB (transferred, compressed) in CSS. The css-analyzer module reports the amount of bytes (uncompressed), so actually it looks like it's still missing some CSS 😄

Let me know if there's anything else that's weird. But most of the issues you describe are related to updates in https://github.com/bartveneman/extract-css and not to css-analyzer.

[nickautomatic]

Thanks for the quick and helpful response @bartveneman (and sorry if I've opened this issue in the wrong place!) Now I look at it again I think you're right: it looks like the current measure probably is accurate (which is a shame, because it means everything is worse than I thought! 😄)

What I'm still not clear about is why the stats have all jumped by such a large amount - ie. whether this is an artefact of changes to how the analysis (or extraction) works (ie. the new numbers are accurate, but the old ones were overlooking something?), or whether it reflects a genuine change in our code? I don't think we could have done anything that would cause such a huge change (the embedded font file, for instance, is not a new thing), but it would be good to understand where that jump has come from!

Incidentally, in case it's of interest, I think the discrepancy between the network panel in Firefox and the value that extract-css is getting might be due to a stylesheet that is being loaded with rel="prefetch" (ie. it's pre-emptively loaded and cached for subsequent pages, but not used on the current page). This shows up in the network panel (since the file is loaded) but I'm guessing maybe isn't analysed by extract-css, since the browser doesn't actually parse the stylesheet? (Fwiw, I think this is probably the desirable behaviour)

[bartveneman]

What I'm still not clear about is why the stats have all jumped by such a large amount - ie. whether this is an artefact of changes to how the analysis (or extraction) works (ie. the new numbers are accurate, but the old ones were overlooking something?), or whether it reflects a genuine change in our code?

99% sure this is an effect of https://extract-css.now.sh/ becoming more accurate. Before I created extract-css, I used https://github.com/cssstats/get-css, but that was even more limiting and less accurate. Extract-css was introduced into projectwallace.com at Feb. 16 2019.

My recommendation is to use the webhook feature in projectwallace.com, so you can push the exact CSS that you generate every time you make a change. I have not documented it yet (will do soon), but you can see the basic example on the New Import page in Project Wallace.

This shows up in the network panel (since the file is loaded) but I'm guessing maybe isn't analysed by extract-css, since the browser doesn't actually parse the stylesheet?

That sounds reasonable 😄 . Extract-css uses the Chrome coverage API and a little bit of JS to fetch the contents of all <style> elements. I guess Chrome is smart enough not to count the prefetched CSS in the coverage report, which is quite cool 😎

Closing this issue, let me know if there's anything else related to this issue or feel free to open a new one.