search

phetsims / chipper

Tools for developing and building PhET interactive simulations.
http://scenerystack.org/
MIT License
12 stars 14 forks source link

Update all of our package.json dependencies in chipper #1084

Closed samreid closed 3 years ago

samreid commented 3 years ago

From https://github.com/phetsims/perennial/issues/236, we discussed upgrading all our dependencies in package.json. @zepumph volunteered to take a look.

zepumph commented 3 years ago

Here is what master looks like to me right now:

 "devDependencies": {
    "@babel/core": "~7.8.6",
    "@babel/preset-env": "~7.8.6",
    "@types/lodash": "^4.14.172",
    "archiver": "~3.1.1",
    "babel-eslint": "^10.0.3",
    "eslint": "~7.22.0",
    "eslint-plugin-react": "^7.18.0",
    "grunt": "~1.4.0",
    "html-webpack-plugin": "^5.3.1",
    "jimp": "^0.2.0",
    "jpeg-js": "~0.4.3",
    "jsdoc": "~3.6.3",
    "jsondiffpatch": "^0.4.1",
    "lodash": "^4.17.10",
    "marked": "^0.3.13",
    "md5": "~2",
    "memfs": "^3.0.3",
    "node-html-encoder": "~0.0.2",
    "pngjs": "~0.4.0",
    "puppeteer": "^10.1.0",
    "qunit": "~2.10.0",
    "request": "^2.87.0",
    "request-promise-native": "^1.0.7",
    "taffydb": "^2.7.3",
    "terser": "~4.6.4",
    "typescript": "4.4.2",
    "webpack": "^5.47.1",
    "webpack-cli": "^4.7.0",
    "webpack-dev-server": "^3.11.2"
  },
zepumph commented 3 years ago

The only dependencies with major version differences are:

archiver
pngjs
terser
webpack-dev-server

I created https://github.com/phetsims/chipper/issues/1086 because it seemed good to try to remove all packages that aren't really needed.

Please unassign after comment.

Next I'll test updating all of them to latest and see what breaks!

zepumph commented 3 years ago

terser v5 had breaking changes, one of which I was unable to remedy, and so I gave up on it.

```diff Index: js/grunt/buildRunnable.js IDEA additional info: Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP <+>UTF-8 =================================================================== diff --git a/js/grunt/buildRunnable.js b/js/grunt/buildRunnable.js --- a/js/grunt/buildRunnable.js (revision ca64493ba1810161b9a6c3a3881af4eb6cf4e1e6) +++ b/js/grunt/buildRunnable.js (date 1630616525795) @@ -201,7 +201,7 @@ const productionScripts = await recordTime( async () => { return [ ...startupScripts, - ...minifiableScripts.map( js => minify( js, minifyOptions ) ) + ...minifiableScripts.map( async js => minify( js, minifyOptions ) ) ]; }, ( time, scripts ) => { grunt.log.ok( `Production minification complete: ${time}ms (${_.sum( scripts.map( js => js.length ) )} bytes)` ); @@ -209,7 +209,7 @@ const debugScripts = await recordTime( async () => { return [ ...startupScripts, - ...minifiableScripts.map( js => minify( js, debugMinifyOptions ) ) + ...minifiableScripts.map( async js => minify( js, debugMinifyOptions ) ) ]; }, ( time, scripts ) => { grunt.log.ok( `Debug minification complete: ${time}ms (${_.sum( scripts.map( js => js.length ) )} bytes)` ); Index: js/grunt/copyDirectory.js IDEA additional info: Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP <+>UTF-8 =================================================================== diff --git a/js/grunt/copyDirectory.js b/js/grunt/copyDirectory.js --- a/js/grunt/copyDirectory.js (revision ca64493ba1810161b9a6c3a3881af4eb6cf4e1e6) +++ b/js/grunt/copyDirectory.js (date 1630616747409) @@ -17,7 +17,7 @@ * @param {function} [filter] - rules for filtering files. If returns falsy, then the file will be copied directly (helps with images) * @param {Object} [options] */ -module.exports = function( src, dst, filter, options ) { +module.exports = async function( src, dst, filter, options ) { options = _.assignIn( { failOnExistingFiles: false, @@ -28,7 +28,7 @@ }, options ); // Copy built sim files (assuming they exist from a prior grunt command) - grunt.file.recurse( src, ( abspath, rootdir, subdir, filename ) => { + grunt.file.recurse( src, async ( abspath, rootdir, subdir, filename ) => { let isInBlacklistedDir = false; @@ -57,7 +57,7 @@ // Minify the file if it is javascript code if ( options.minifyJS && filename.endsWith( '.js' ) && abspath.indexOf( 'chipper/templates/' ) < 0 ) { const toBeMinified = filteredContents ? filteredContents : contents; - filteredContents = minify( toBeMinified, options.minifyOptions ); + filteredContents = await minify( toBeMinified, options.minifyOptions ); // Only add the license to the javascript code filteredContents = options.licenseToPrepend + filteredContents; Index: js/grunt/buildStandalone.js IDEA additional info: Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP <+>UTF-8 =================================================================== diff --git a/js/grunt/buildStandalone.js b/js/grunt/buildStandalone.js --- a/js/grunt/buildStandalone.js (revision ca64493ba1810161b9a6c3a3881af4eb6cf4e1e6) +++ b/js/grunt/buildStandalone.js (date 1630616525811) @@ -69,7 +69,7 @@ // Wrap with an IIFE fullSource = `(function() {\n${fullSource}\n}());`; - fullSource = minify( fullSource, minifyOptions ); + fullSource = await minify( fullSource, minifyOptions ); return fullSource; }; Index: js/grunt/Gruntfile.js IDEA additional info: Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP <+>UTF-8 =================================================================== diff --git a/js/grunt/Gruntfile.js b/js/grunt/Gruntfile.js --- a/js/grunt/Gruntfile.js (revision ca64493ba1810161b9a6c3a3881af4eb6cf4e1e6) +++ b/js/grunt/Gruntfile.js (date 1630616525811) @@ -243,7 +243,7 @@ if ( repoPackageObject.phet.standaloneTranspiles ) { for ( const file of repoPackageObject.phet.standaloneTranspiles ) { - fs.writeFileSync( `../${repo}/build/${path.basename( file )}`, minify( grunt.file.read( file ) ) ); + fs.writeFileSync( `../${repo}/build/${path.basename( file )}`, await minify( grunt.file.read( file ) ) ); } } } Index: js/grunt/minify.js IDEA additional info: Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP <+>UTF-8 =================================================================== diff --git a/js/grunt/minify.js b/js/grunt/minify.js --- a/js/grunt/minify.js (revision ca64493ba1810161b9a6c3a3881af4eb6cf4e1e6) +++ b/js/grunt/minify.js (date 1630616525811) @@ -26,7 +26,7 @@ beautify: false }; -const minify = function( js, options ) { +const minify = async function( js, options ) { options = _.assignIn( {}, MINIFY_DEFAULTS, options ); // Promote to top level variables @@ -73,7 +73,7 @@ } if ( uglify ) { - const result = terser.minify( js, uglifyOptions ); + const result = await terser.minify( js, uglifyOptions ); if ( result.error ) { console.log( result.error ); Index: js/grunt/copySupplementalPhetioFiles.js IDEA additional info: Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP <+>UTF-8 =================================================================== diff --git a/js/grunt/copySupplementalPhetioFiles.js b/js/grunt/copySupplementalPhetioFiles.js --- a/js/grunt/copySupplementalPhetioFiles.js (revision ca64493ba1810161b9a6c3a3881af4eb6cf4e1e6) +++ b/js/grunt/copySupplementalPhetioFiles.js (date 1630616649201) @@ -246,7 +246,7 @@ const fullBlacklist = wrappersBlacklist.concat( libFileNames ); // wrapping function for copying the wrappers to the build dir - const copyWrapper = ( src, dest, wrapper, wrapperName ) => { + const copyWrapper = async ( src, dest, wrapper, wrapperName ) => { const wrapperFilterWithNameFilter = ( abspath, contents ) => { const result = filterWrapper( abspath, contents ); @@ -260,7 +260,7 @@ } return result; }; - copyDirectory( src, dest, wrapperFilterWithNameFilter, { + await copyDirectory( src, dest, wrapperFilterWithNameFilter, { blacklist: fullBlacklist, minifyJS: true, minifyOptions: { @@ -278,7 +278,7 @@ packageObject.phet[ 'phet-io' ].wrappers ? wrappers.concat( packageObject.phet[ 'phet-io' ].wrappers ) : wrappers; - wrappers.forEach( wrapper => { + for ( const wrapper of wrappers ) { const wrapperParts = wrapper.split( '/' ); @@ -286,14 +286,14 @@ const wrapperName = wrapperParts.length > 1 ? wrapperParts[ wrapperParts.length - 1 ] : wrapperParts[ 0 ].replace( DEDICATED_REPO_WRAPPER_PREFIX, '' ); // Copy the wrapper into the build dir /wrappers/, exclude the blacklist - copyWrapper( `../${wrapper}`, `${wrappersLocation}${wrapperName}`, wrapper, wrapperName ); - } ); + await copyWrapper( `../${wrapper}`, `${wrappersLocation}${wrapperName}`, wrapper, wrapperName ); + } // Copy the wrapper index into the top level of the build dir, exclude the blacklist - copyWrapper( '../phet-io-wrappers/index', `${buildDir}`, null, null ); + await copyWrapper( '../phet-io-wrappers/index', `${buildDir}`, null, null ); // Create the lib file that is minified and publicly available under the /lib folder of the build - handleLib( buildDir, filterWrapper ); + await handleLib( buildDir, filterWrapper ); // Create the zipped file that holds all needed items to run PhET-iO offline. NOTE: this must happen after copying wrapper await handleOfflineArtifact( buildDir, repo, version ); @@ -305,7 +305,7 @@ await handleJSDOC( buildDir ); // create the client guides - handleClientGuides( repo, buildDir ); + await handleClientGuides( repo, buildDir ); await handleStudio( wrappersLocation ); @@ -329,7 +329,7 @@ * @param {Function} filter - the filter function used when copying over wrapper files to fix relative paths and such. * Has arguments like "function(abspath, contents)" */ -const handleLib = ( buildDir, filter ) => { +const handleLib = async ( buildDir, filter ) => { grunt.log.debug( 'Creating phet-io lib file from: ', LIB_FILES ); grunt.file.mkdir( `${buildDir}lib` ); @@ -344,7 +344,7 @@ consolidated += filteredContents ? filteredContents : contents; } ); - const minified = minify( consolidated, { + const minified = await minify( consolidated, { stripAssertions: false } ); @@ -466,7 +466,7 @@ * @param {string} repoName * @param {string} buildDir */ -const handleClientGuides = ( repoName, buildDir ) => { +const handleClientGuides = async ( repoName, buildDir ) => { const builtClientGuidesOutputDir = `${buildDir}doc/guides/`; const clientGuidesSourceRoot = `${CLIENT_GUIDES_DIR}${repoName}/`; @@ -477,12 +477,12 @@ } // copy over common images and styles - copyDirectory( `${CLIENT_GUIDES_DIR}${COMMON_DIR}`, `${builtClientGuidesOutputDir}${COMMON_DIR}` ); + await copyDirectory( `${CLIENT_GUIDES_DIR}${COMMON_DIR}`, `${builtClientGuidesOutputDir}${COMMON_DIR}` ); // copy over the sim-specific phet-io guide images const simSpecificGuideImagesDir = `${CLIENT_GUIDES_DIR}${repoName}/images/`; if ( fs.existsSync( simSpecificGuideImagesDir ) ) { - copyDirectory( simSpecificGuideImagesDir, `${builtClientGuidesOutputDir}images/` ); + await copyDirectory( simSpecificGuideImagesDir, `${builtClientGuidesOutputDir}images/` ); } // handle generating and writing the html file for each client guide ```
zepumph commented 3 years ago

png looks safe, but it also if going from major version 0 to 6. I tested by modulifying bending-light, which uses mipmaps.

For webpack-dev-server, there were breaking changes, but I was able to parse through them and test it. Honestly I didn't spend too much time on this because I don't think that anyone ever uses it.

I tested many grunt commands, but probably not all of them. I am feeling good about things enough to commit.

Over to other assignees for comment about the above comment.

zepumph commented 3 years ago

@jonathanolson gave me the go-ahead, and I removed taffydb also while I was at it. @samreid please review this issue. Anything else you would like done?

samreid commented 3 years ago

This has been a bold and brave issue! I think the best gauge of the status is that we have had sims branched with these shas, and I'm not aware of any problems caused by this. I'm not worried about the dependencies that stayed behind. Thanks for doing this! Closing.