jsdelivr / bot

DEPRECATED The jerk of a bot that checks PRs and responds in comments
22 stars 14 forks source link

detect minified files #9

Closed tomByrer closed 8 years ago

tomByrer commented 10 years ago

ref Phase 1 Check sourceMappingURL /(\/\*+[\s\S]*?sourceMappingURL\s*=[\s\S]*?\*\/)|(\/\/.*?sourceMappingURL\s*=.*?[\r\n])/g If there is a .map file pointed to, then likely this is a minified file. Other tests should be skipped if true.

Phase 2: remove comments (where possible without trashing code) This RegEx test should work: /(^(\/\*+[\s\S]*?\*\/)|(\/\*+.*\*\/)|^\/\/.*?[\r\n])[\r\n]*/gm on this test code:

Match full line & blank newlines after: 
//! https://github.com/tomByrer/web-engeneering-notes/blob/master/banner-tips.md#mini-banner

hunk
/**
* usage: https://github.com/jsdelivr/bot/issues/1#issuecomment-42943418
**/
junk
/*! script.node.js github.com/owner/scriptjs 0.7.0 (c)2014 License GPL */
funk
End line comment=! /*** I like stars ***/

left/*middle*/right
trunk
// project-legacy.js github.com/owner/project 0.1.0 (c)2014 @licence MIT
stunk
/* @preserve this comment please? */
sunk
// @License to @Preserve: none
bunk
/* important info but not enough to keep in minified code */
punk
// comment
edge-case (works if have space between **)
/**/
space
/*
*/

Unsure what to do:
//# sourceMappingURL=jquery.mobile-1.4.2.min.map
var x=y;// Great coding!  Have to keep this comment because of above example...
var twitterHandle = '@preserve';  //oops

Should Fail:
https://user:pass@license.not-a-comment.it/is-url.js
/!escape
<!-- remove this comment -->
"/*"

True; EOF test:
/* test */

Bonus: grabs newlines after comment.

Phase 3

If ( comments-removed.match(/\n\r/g).length > ( comments-removed.length / 80 ) {
  not mified 
}
megawac commented 10 years ago

I think we can assume if sourceMappingURL\s*= is present in a comment in a file we can consider it minified. The only file I can think of off the top of my head that will break this assumption is bootstrap.css which has broken sourcemapping iirc

tomByrer commented 10 years ago

sourceMappingURL\s*= is a great idea! What should happen if there is: var sourceMappingURL = somefile.js? I'm thinking looking for a .map file is better.

Updated original post with test URL & Phase 2.

megawac commented 10 years ago

thats why I said in a comment :)

tomByrer commented 10 years ago

oops! Fixed top issue

megawac commented 10 years ago

Lookin like a good start. @tomByrer I can give you pointers if you want to learn a bit of python and make a pr

megawac commented 10 years ago

A branch of this issue is validating .map files: validate .map points to valid files; ensure x.map is referenced by some file; Ensure that file is minified... etc

tomByrer commented 10 years ago

Yep, removing comments before checking if minified is helpful

megawac commented 8 years ago

I think https://github.com/jsdelivr/bot/commit/547c10b8e984654140e354630b89e791e8fe21b6 is a reasonable fix based on average line lengths. Essentially I assume if a file has an average line width of 200 characters we classify the file as minified