WebKit / Speedometer

An open source repository for the Speedometer benchmark
Other
592 stars 70 forks source link

Npm audit fix #360

Closed issackjohn closed 7 months ago

issackjohn commented 7 months ago

This PR updates the package-lock.json files for the affected packages by running npm audit fix to address CVE-2023-26159.

Hosted at: https://issackjohn.github.io/Speedometer3

closes #355

julienw commented 7 months ago

GIven all the changes, I'm wondering that wemay need to regenerate all dists as well. What do you think?

flashdesignory commented 7 months ago

GIven all the changes, I'm wondering that wemay need to regenerate all dists as well. What do you think?

I had that as a todo item ... not sure if @issackjohn wants to run the builds for these anyways and I can do the same with the few that weren't touched afterwards?

julienw commented 7 months ago

I think it would be good to do that here to make sure that they still build after the changes.

issackjohn commented 7 months ago

I think it would be good to do that here to make sure that they still build after the changes.

Rebuild all the dists which had package-lock.json updated in this PR right?

issackjohn commented 7 months ago

GIven all the changes, I'm wondering that wemay need to regenerate all dists as well. What do you think?

I had that as a todo item ... not sure if @issackjohn wants to run the builds for these anyways and I can do the same with the few that weren't touched afterwards?

Looks like news-next doesn't work on github pages. @flashdesignory Would you be able to help me collect the before and after numbers for that? I can do the others.

issackjohn commented 7 months ago

5 runs

Screenshot 2024-01-25 at 2 15 26 PM

Other view: Before:

Screenshot 2024-01-25 at 2 23 42 PM

After:

Screenshot 2024-01-25 at 2 24 25 PM
julienw commented 7 months ago

I notice that jQuery (both simple and complex versions) shows quite a big increase on all browsers, do we know why?

julienw commented 7 months ago

It could come from the handlebars update. We were previously using a very old version from 2017.

smaug---- commented 7 months ago

This affects performance numbers a lot, so we should know more why that is happening. I don't think we should land this before there has been some more investigation on that.

issackjohn commented 7 months ago

I would like to scope this. The intentions of my change were to update the follow-redirects package version to conform with internal compliance. Perhaps we could upgrade the other packages of jQuery in a separate PR? That way we don't block this one.

bgrins commented 7 months ago

I would like to scope this. The intentions of my change were to update the follow-redirects package version to conform with internal compliance. Perhaps we could upgrade the other packages of jQuery in a separate PR? That way we don't block this one.

That seems fine to me - let's get this one closed with the development environment specific updates

bgrins commented 7 months ago

It could come from the handlebars update. We were previously using a very old version from 2017.

This affects performance numbers a lot, so we should know more why that is happening. I don't think we should land this before there has been some more investigation on that.

I agree. We intentionally decided on framework versions for 3.0, and changing them should be deliberately considered - not based on the output of the audit command. We can consider whether updating this (along with other frameworks) is a good idea in a future version, there's plenty else to do now.

bgrins commented 7 months ago

Thanks for the update Issack. Looking at the new diff, the following dist/ directories appear to have nontrivial changes:

Do we know if these are latent differences on main between what's in dist/ and the output of npm run build, or are these changes as a result of something in the audit fix? Because presumably updating follow-redirects should result in no changes to any dist.

issackjohn commented 7 months ago

Before

Screenshot 2024-01-26 at 1 51 33 PM

After:

Screenshot 2024-01-26 at 1 51 52 PM
issackjohn commented 7 months ago

Thanks for the update Issack. Looking at the new diff, the following dist/ directories appear to have nontrivial changes:

  • newssite-next
  • react-complex
  • react-redux-complex
  • react-redux
  • react
  • vue-complex
  • vue

Do we know if these are latent differences on main between what's in dist/ and the output of npm run build, or are these changes as a result of something in the audit fix? Because presumably updating follow-redirects should result in no changes to any dist.

The result of npm audit and then rebuilding the dists. You're saying that we should only run npm update follow-redirects in each of the affected packages of #351 right?

julienw commented 7 months ago

Besides jQuery with handlebars, I think that the other changes are fairly trivial. The large amount of upgraded packages comes mostly from babel. Have you seen different results in the packages you mentioned @bgrins ?

bgrins commented 7 months ago

The result of npm audit and then rebuilding the dists. You're saying that we should only run npm update follow-redirects in each of the affected packages of https://github.com/WebKit/Speedometer/issues/351 right?

My suggestion is that we do two separate steps: first we just go through the existing directories and do npm run build to start with a baseline of the current tree with no package.json changes - this is what #346 was opened for.

Then separately, we would do some kind of package.json updates (either the approach here or the approach in #351 - I don't have a strong opinion about this and haven't closely been following the discussion, but presumably the current approach in this PR is good), and do npm run build again. That way we'll know exactly what is causing changes to the dist directory.

bgrins commented 7 months ago

Besides jQuery with handlebars, I think that the other changes are fairly trivial. The large amount of upgraded packages comes mostly from babel. Have you seen different results in the packages you mentioned @bgrins ?

It's hard for me to tell what the substance of the changes are (it's hard to tell from the PR diff, and why I think it would be helpful to "clear out" any latent differences with main dist/ directories and the result of the build). But I'll defer to you on the review here: if you and others are satisfied with the current diff it's fine with me

julienw commented 7 months ago

GitHub has a nice viewer for the package-lock.json, you can click the button with the "document" icon. I believe we can revert the update of handlebars in jQuery. I'll look closely at the others on Monday, see if there's anything other than devDependencies.

julienw commented 7 months ago

Thanks for changing the jquery workload so that just "follow-redirects" is updated there.

I looked at all other updates, they're all dev dependencies: mostly babel, webpack, and related packages such as postcss. The angular package got some updates for angular-related packages too, but only build-related, not runtime.

Can you please provide new numbers in the various browsers so that we can see if the regression is still present? Thanks!

issackjohn commented 7 months ago

Before

Screenshot 2024-01-26 at 1 51 33 PM

After:

Screenshot 2024-01-26 at 1 51 52 PM

@julienw these are the new numbers after the commit that only changed jQuery

issackjohn commented 7 months ago

Before:

Screenshot 2024-01-29 at 8 33 21 AM

After:

Screenshot 2024-01-29 at 8 32 58 AM
julienw commented 7 months ago

OK, this looks good to me! Thanks for all the updates

issackjohn commented 7 months ago

OK, this looks good to me! Thanks for all the updates

Thanks!

issackjohn commented 7 months ago

@rniwa PTAL

issackjohn commented 7 months ago

Thank you all for your reviews and help!