ipfs / helia

An implementation of IPFS in JavaScript
https://helia.io
Other
814 stars 81 forks source link

feat: also check content-type with @sgtpooki/file-type #416

Closed SgtPooki closed 4 months ago

SgtPooki commented 5 months ago

This PR adds an additional check for content-type via a fork of the file-type library, similar to how it's used in helia-http-gateway.

I forked https://github.com/sindresorhus/file-type because it doesn't currently support browsers due to direct or dependency consumption of node:stream, node:buffer, node:fs, or node:process.

The fork could still use some updates to support streams and confirm that memory usage is reasonable, but it's a start and confirmed working in helia-service-worker-gateway.

Using this fork, we would be able to remove the file-type dependency from helia-http-gateway and de-dupe the content-type checking code.

Related issues in the original repo:

achingbrain commented 5 months ago

file-type really looks like it needs a new module that strips out all the old node stuff.

We now have mime-types and the file-type fork as deps - how much do these add to the bundle size?

achingbrain commented 5 months ago

how much do these add to the bundle size?

You can find this out with npm run build -- -b in packages/verified-fetch

Somewhat eye-opening:

image

Do we really need to be able to detect all of these mime types?

SgtPooki commented 4 months ago

Do we really need to be able to detect all of these mime types?

@achingbrain honestly, no. and mime-types only does extension recognition, where file-type does magic byte recognition (which we need more) so I think removing mime-type would be ideal, and replacing with file-type

file-type really looks like it needs a new module that strips out all the old node stuff.

We now have mime-types and the file-type fork as deps

yea, if we're open to migrating to a file-type fork I can work on re-adding stream support using browser compatible streams, and also splitting up the node stuff so file reading works as well

SgtPooki commented 4 months ago

removing mime-types reduces bundle size by 147kb (707.2kb to 560.2kb)

image

SgtPooki commented 4 months ago

Closing this in favor of #422