search

KristofferStrube / Blazor.FileSystemAccess

A Blazor wrapper for the File System Access browser API.
https://kristofferstrube.github.io/Blazor.FileSystemAccess/
MIT License
327 stars 39 forks source link

question: random access (seeking?) #37

Closed aloneguid closed 1 year ago

aloneguid commented 1 year ago

Thanks for the great library by the way. I have an issue where I'm trying to get a small amount of data from a very large file on disk (wasm application). Is it possible to seek inside the file without re-reading all the data from start?

KristofferStrube commented 1 year ago

Hey @aloneguid, Glad that you appreciate the library.

It is an interesting question. This is actually more related to my Blazor File API library which this library depends on.

When we read a file we use Blazor.FileAPI. I can't remember if getting the File loads the entire file into memory, but my guess would be that it is only a reference. If it is only a reference then we can use the Slice method to get a specific section as we see in this sample.

I will try to check this myself to see if the solution is performant for files on the disc (check whether or not we can slice files from the disc without reading it all).

aloneguid commented 1 year ago

Thanks @KristofferStrube. I've made an early prototype using slice and it seems to work. I think this can be optimised somehow but works much faster than IBrowserFile with read skipping. I can raise a PR for this if you like.

link to prototype: https://github.com/aloneguid/parquet-online/blob/master/ParquetViewer/WebAdapters/FileApiRandomAccessReadableStream.cs

aloneguid commented 1 year ago

btw i have to call slice every time Read is called, otherwise read will try to read more than i need and throw an exception, not sure it's by design.

KristofferStrube commented 1 year ago

Oh are you using it to wrap ReadableStream? You can already get a wrapper for ReadableStream by calling StreamAsymc on the File if that is what you are looking for.

aloneguid commented 1 year ago

Yes, but ReadableStream stream is forward only (not seekable) which is what I need ;)

KristofferStrube commented 1 year ago

Ah, cool I got you. What would the PR entail?

KristofferStrube commented 1 year ago

Side comment: I saw you were using a file drop library in your project. This library will support that in the near future as well. ;)

aloneguid commented 1 year ago

Thank you :) Closing.

lvde0 commented 6 months ago

Hey, are there still plans to merge a seekable Stream any time soon?

KristofferStrube commented 6 months ago

I would be very interested in implementing this in Blazor.FileAPI. I will create an issue there soon.

lvde0 commented 6 months ago

@KristofferStrube Awesome! I am trying to read large (>1Gb) zip files, but ZipArchive can only deal with seekable and synchronous streams. So at least the seekable bit would be solved that way :)