Technical Debt: Update crawler dependencies to current versions

[elrayle]

Description

Many of the dependencies for this repo are out of date. A significant set of dependencies have reached EOL and are no longer supported. Update dependencies to current maintained versions.

NOTE: If code changes are required to update a dependency, consider whether a refactor from JavaScript to TypeScript would help with long term maintenance.

Related Work

• https://github.com/clearlydefined/crawler/issues/527
• https://github.com/clearlydefined/crawler/issues/502
• https://github.com/clearlydefined/service/issues/1007
• https://github.com/clearlydefined/website/issues/1024

[qtomlinson]

@clearlydefined/spdx has been upgraded and released. need to upgrade in crawler's package.json.