Review and update dependencies in package.json

[edwinjue]

Overview

Many of the dependencies in package.json are either outdated or may not being used.

We'll need to take a closer look to see which packages are still being used for the new tech stack and whether we can upgrade to the latest version of each required dependency without breaking the app

Action Items

Review all dependencies listed in the following sections of package.json:

• [ ] dependencies
• [ ] devDependencies

Resources/Instructions

[edwinjue]

Here are some resources that may help you get started.

1) To see what's outdated: npm outdated

2) To check and pick packages to update: npx npm-check -u

https://stackoverflow.com/a/52272165

Some gists from another project I am working on but might be relevant to getting our node dependencies updated to the latest versions:

• Example of some updated dependencies in a package.json that work well with one another using Webpack 5: https://gist.github.com/edwinjue/e2e9f816f3bd92c2895a406cdd36ea0a

• Example of a working webpack.config.js using Webpack 5: https://gist.github.com/edwinjue/e788eaf61648df16833c5ebc26de312f

Some packages might require significant refactoring when upgrading, for example MUI from 4 to 5.

With that said, and so we can focus on quick wins, we can start small and upgrade only those packages that require little to no refactoring and remove any that are no longer being used.

Then for those that might require more effort to upgrade like MUI, let's add them to a list to consider later and break them out into separate issues. What do you think?

Thanks for looking into this Ryan. Much appreciated!

[ryanfchase]

Having an initial look. Running npm outdated yields the following list. Going to proceed and investigate which packages that require minor effort level to update / remove.

Edit: Forgot to run npm i since pulling latest.

Package                                         Current          Wanted          Latest  Location
@babel/core                                      7.11.1         7.22.10         7.22.10  311-data
@babel/plugin-proposal-class-properties          7.10.4          7.18.6          7.18.6  311-data
@babel/preset-env                                7.11.0         7.22.10         7.22.10  311-data
@babel/preset-react                              7.10.4          7.22.5          7.22.5  311-data
@duckdb/duckdb-wasm                              1.26.0          1.27.0          1.27.0  311-data
@mapbox/mapbox-gl-geocoder                        4.7.0           4.7.4           5.0.1  311-data
@material-ui/core                                4.11.0          4.12.4          4.12.4  311-data
@material-ui/icons                                4.9.1          4.11.3          4.11.3  311-data
@material-ui/lab                         4.0.0-alpha.58  4.0.0-alpha.61  4.0.0-alpha.61  311-data
@react-pdf/renderer                              1.6.10          1.6.17          3.1.12  311-data
@turf/turf                                        5.1.6           5.1.6           6.5.0  311-data
apache-arrow                                     12.0.0          12.0.1          12.0.1  311-data
are-you-es5                                       2.1.1           2.1.2           2.1.2  311-data
axios                                            0.21.4          0.21.4           1.4.0  311-data
babel-jest                                       24.9.0          24.9.0          29.6.2  311-data
babel-loader                                      8.1.0           8.3.0           9.1.3  311-data
chart.js                                          2.9.4           2.9.4           4.3.3  311-data
chartjs-chart-box-and-violin-plot                 2.3.0           2.4.0           4.0.0  311-data
chartjs-plugin-datalabels                         0.7.0           0.7.0           2.2.0  311-data
classnames                                        2.2.6           2.3.2           2.3.2  311-data
clsx                                              1.2.1           1.2.1           2.0.0  311-data
copy-webpack-plugin                               6.4.1           6.4.1          11.0.0  311-data
core-js                                           3.6.5          3.32.0          3.32.0  311-data
css-loader                                        3.6.0           3.6.0           6.8.1  311-data
dataframe-js                                      1.4.3           1.4.4           1.4.4  311-data
dfa                                               1.1.0           1.1.0           1.2.0  311-data
dotenv                                            8.2.0           8.6.0          16.3.1  311-data
dotenv-webpack                                    1.8.0           1.8.0           8.0.1  311-data
enzyme-adapter-react-16                          1.15.3          1.15.7          1.15.7  311-data
eslint                                            6.8.0           6.8.0          8.47.0  311-data
eslint-config-airbnb                             18.2.0          18.2.1          19.0.4  311-data
eslint-import-resolver-node                       0.3.4           0.3.9           0.3.9  311-data
eslint-import-resolver-webpack                   0.12.2          0.12.2          0.13.4  311-data
eslint-plugin-jsx-a11y                            6.3.1           6.7.1           6.7.1  311-data
eslint-plugin-react                              7.22.0          7.33.1          7.33.1  311-data
eslint-plugin-react-hooks                         1.7.0           1.7.0           4.6.0  311-data
extract-text-webpack-plugin                4.0.0-beta.0    4.0.0-beta.0           3.0.2  311-data
file-loader                                       4.3.0           4.3.0           6.2.0  311-data
focus-visible                                     5.1.0           5.2.0           5.2.0  311-data
gh-pages                                          5.0.0           5.0.0           6.0.0  311-data
html-webpack-plugin                               3.2.0           3.2.0           5.5.3  311-data
html2canvas                                  1.0.0-rc.5           1.4.1           1.4.1  311-data
husky                                             4.3.8           4.3.8           8.0.3  311-data
jest                                             24.9.0          24.9.0          29.6.2  311-data
mapbox-gl                                        1.12.0          1.13.3          2.15.0  311-data
mini-css-extract-plugin                           0.9.0           0.9.0           2.7.6  311-data
mixpanel-browser                                 2.38.0          2.47.0          2.47.0  311-data
node-sass                                        4.14.1          4.14.1           9.0.0  311-data
npm                                             6.14.18         6.14.18           9.8.1  311-data
prop-types                                       15.7.2          15.8.1          15.8.1  311-data
query-string                                      5.1.1           5.1.1           8.1.0  311-data
react                                           16.13.1         16.14.0          18.2.0  311-data
react-day-picker                                  7.4.8          7.4.10           8.8.0  311-data
react-dom                                       16.13.1         16.14.0          18.2.0  311-data
react-markdown                                    5.0.3           5.0.3           8.0.7  311-data
react-redux                                       7.2.1           7.2.9           8.1.2  311-data
react-router                                      5.2.0           5.3.4          6.15.0  311-data
react-router-dom                                  5.2.0           5.3.4          6.15.0  311-data
react-swipeable                                   6.0.1           6.2.2           7.0.1  311-data
react-test-renderer                             16.13.1         16.14.0          18.2.0  311-data
react-toastify                                    9.0.8           9.1.3           9.1.3  311-data
redux                                             4.0.5           4.2.1           4.2.1  311-data
redux-devtools-extension                         2.13.8          2.13.9          2.13.9  311-data
redux-saga                                        1.1.3           1.2.3           1.2.3  311-data
regenerator-runtime                              0.13.7         0.13.11          0.14.0  311-data
sass-loader                                       8.0.2           8.0.2          13.3.2  311-data
style-loader                                      1.2.1           1.3.0           3.3.3  311-data
webpack                                          4.46.0          4.46.0          5.88.2  311-data
webpack-cli                                      3.3.12          3.3.12           5.1.4  311-data
webpack-dev-server                               3.11.0          3.11.3          4.15.1  311-data
webpack-merge                                     4.2.2           4.2.2           5.9.0  311-data

[edwinjue]

Thanks for the list Ryan.

I just stumbled upon a project that is used to find unused files, dependencies and exports in Javascript and Typescript. I haven't used it yet but it looks pretty interesting:

https://github.com/webpro/knip

Let me know what you think

[ryanfchase]

Update for weekly meeting on 8/16. Currently trying out if any LTS releases of Node are viable for installing our dependencies, starting with v18.17.1 and working backwards. If none work, I'll try to adapt the linked gists to use in our repo.

[ryanfchase]

Per my convo with @edwinjue in https://github.com/hackforla/311-data/issues/1545#issuecomment-1681383682, we're going to box the actionable work of this issue to one task:

• getting a working version of our app using Node v18.17.1

Moving forward I'll be working on this issue and preparing it for next week.

[ryanfchase]

Update for weekly meeting. Still testing out LTS Hydrogen (v18.17.1) to see if we can build our application. I think that updating packages as much as possible without introducing breaking API changes is going to be tricky, since several packages are dependent on each other -- and I'm not so skilled in migrations to know about an easy way to go about this. Instead, I'm currently undertaking the following steps:

1. NPM uninstall all packages (start with blank package.json)
2. go through each package 1 by 1 and check to see if we are indeed using it (reference NPM docs + check usage in codebase)
3. add latest major release of each necessary package, downgrade versions if there is a conflict
4. attempt to build project, fixing breaking API changes until successful

I'm on step 4. I've fixed a few API changes, currently looking at one in webpack.dev.js, (see below). Suggestions welcome, I'm just referring to documentation and trying things out.

[webpack-cli] Invalid options object. Dev Middleware has been initialized using an options object that does not match the API schema.
 - options has an unknown property 'historyApiFallback'. These properties are valid:
   object { mimeTypes?, writeToDisk?, methods?, headers?, publicPath?, stats?, serverSideRender?, outputFileSystem?, index? }

---

I also have some notes about which packages I'm keeping vs ones I'm discarding:

Notes for work 8/19 (WIP)
- Material UI v4 can be upgraded to v5, but this will likely be a larger effort
- Noticing we are using clx (from classnames package) and clsx for conditional classname construction. We should just pick one?
- DataFrame is interesting. It's found in utils/DataService.js, which it feels like this was useful for jupyter notebooks
- dfa is a tool for compiling deterministic finite automata. I'm not sure we're making use of this anywhere.
- Enzyme is not being used, so I won't be including it
- html2canvas was for exporting the charts to image format (I assume). I think we can skip it
- I'm keeping proptypes and prop-types, but we should probably assimilate to using one over the other
- we are getting locked into react-router-dom@5.3.4 rather than 6.15.0 because we're using react 17 (not react 18). Material-ui 4 only allows us to use React 17, upgrading to React 18  would allow us to use material-ui 5, and thus use latest of react-router-dom

More discussion to come during our breakout rooms. Please ping me if there are additional questions.

[edwinjue]

@ryanfchase Thanks for the update! Yes, this kind of issue can be difficult because of how closely coupled dependencies can get. I will take a look when I have time and will communicate what I find. We'll discuss more then.

[ryanfchase]

Unassigning myself and moving to Prioritized Backlog.