sindresorhus / file-type

Detect the file type of a file, stream, or data
MIT License
3.71k stars 351 forks source link

Support for reading from Blob in Node.js #588

Closed Borewit closed 1 year ago

Borewit commented 1 year ago

Support for reading from Blob in Node.js context.

Resolves: #581

First merge:

sindresorhus commented 1 year ago

@Borewit Thanks. Maybe we can get rid of browser.js completely by moving the ReadableStream method into core and exposing it in Node.js if Node.js 18 (that's when it's a global) or later?

Borewit commented 1 year ago

The current implementation fileTypeFromStream() converts a Web-API readable stream into a Node.js readable stream, so essentially an adapter.

The fact that it implemented as an adapter makes it unsuitable at the moment to be moved into core.

Only when we are able utilize the Node.js ReadableStream it does make sense to move it into core. And actually only when this could be used to process both the

  1. Web-API readable stream and the
  2. Node.js ReadableStream

which should be in theory the same streams WHATWG streams, but are in reality, not necessary interchangeable:

  1. There are issues with different TypeScript definitions
  2. If I remember well, also the implementation was done differently hence the typings could not be blended.

Caveat on the file-type Blob function

The way the implementation is, and was, inconsistent with the other function, as it does not utilize the lazy reading we use with streams. As it converts the Blob to a buffer, use cases where a Blob (File) is still stream (like when you upload a file), the current mechanism will read the entire Blob (File) in memory.

Related to the previous discussion, utilizing the Blob.stream() this could be prevented. At the other hand, the user can do this as well, and provide that to the WHATWG stream read function we just discussed.

jimmywarting commented 1 year ago

if the hole file dose not need to be read in order to get magic numbers (head and footer) then i suggest that you also slice the file

head = blob.slice(0, 1024)
footer = blob.slice(-1024)

you could also make a really small blob out of it: new Blob([head, footer]).stream()

Also worth mentioning that there now also is a fs.openAsBlob(path) method that could make reading files (partially) easier and more cross platform ergonomic. just using openAsBlob alone won't allocate the file into memory. it's read lazzily when you use any of the read methods...