Storing files in an Uint8List is not possible with large files, please consider providing a stream.

[Wissperwind]

Hi,

The related plugin file_picker provides some options when picking files: FilePicker.platform.pickFiles(allowMultiple: true, withReadStream: true, withData: false); If you set withReadStream: true, withData: false the files were not read into int lists by the plugin. Instead a stream is provided. This stream can be provided to a request:

request.files.add(http.MultipartFile(
  "Your parameter name on server side",
  **file.readStream!**,
  file.size,
  filename: file.name));
}

In this way large files can be handeled becaus they don't need to be read completely into memory. Could you please provide a similar solution for your plugin?

[deakjahn]

That's a nice idea but we have a problem, Houston. :-)) Please, look into the underlying code that returns the actual file contents:

https://github.com/deakjahn/flutter_dropzone/blob/0e334e4254de1f92cb4cca0a055d3eb03c1140b8/flutter_dropzone_web/lib/flutter_dropzone_web.dart#L94-L100

FileReader has three functions: readAsArrayBuffer(), readAsDataUrl() and readAsText() to return the file contents. I can't yet see any way on the JS side to get it in chunks so that we can build a chunked stream out of it.

[Wissperwind]

I think it is not necessary to use FileReader in all cases. A file object has file.openRead(). What about:

 Stream<List<int>> getFileStream(File file) { 

   return file.openRead();
 }

You can pass this stream to the request like in my example above:

request.files.add(http.MultipartFile(
  "Your parameter name on server side",
  file.readStream!,
  file.size,
  filename: file.name));
}

This can handle larger files compared to the getFileData() method. This is how they do it in the file_picker library. Of course xou can leave all your methods like getFileData(). But would it be possible to add one to get a stream?

[deakjahn]

I think you're looking in the wrong place. This isn't a mobile Flutter app or package. This is web. The File you mention is from dart:io, something completely different. It doesn't make us any happier that on Android and iOS, yes, you can read files in a multitude of ways. In a web browser, you cannot. And whatever you can still do in a web browser, if the web engine of Flutter doesn't expose it, we can't call it from the package. Check out dart:html instead.

[Wissperwind]

You were right. I looked at the wrong place. But it is possible to provide a stream for a file in flutter web. See this implementation: https://github.com/miguelpruivo/flutter_file_picker/blob/master/lib/_internal/file_picker_web.dart

  Stream<List<int>> _openFileReadStream(File file) async* {
    final reader = FileReader();

    int start = 0;
    while (start < file.size) {
      final end = start + _readStreamChunkSize > file.size
          ? file.size
          : start + _readStreamChunkSize;
      final blob = file.slice(start, end);
      reader.readAsArrayBuffer(blob);
      await reader.onLoad.first;
      yield reader.result as List<int>;
      start += _readStreamChunkSize;
    }
  }

[deakjahn]

A bit suspicious but let's see. The browser might cheat and read the whole file, anyway, just returning it in chunks. :-)

I'll try to bump up the interface version on Pub.dev but keep the actual implementation here in GitHub. Try it and I'll only publish the whole if it really works...

[deakjahn]

@Wissperwind I guess you could just refer to the GitHub version temporarily and see if it works. The interface is OK and the rest seems to be without error for me now.

[Wissperwind]

Hi, Thanks for adding it. I am looking forward to test it! At first I tried to reference the library from github:

Warning: You are using these overridden dependencies:
! flutter_dropzone 3.0.2 from git https://github.com/deakjahn/flutter_dropzone at c48c24 in flutter_dropzone

Looks good so far. After that I tried to get the stream: Stream<List<int>>? stream = fileUploadView.controller?.getFileStream(file); Unfortunately it throws an unimplemented exception.

I debugged into it and walked along dropzone_view.dart line 160 to flutter_dropzone_plattform_interface.dart line 94 where the exception is thrown. Is there maybe a tiny bit of code missing to enable it for the web platform?

[deakjahn]

Because it's darn difficult to handle version differences even from GitHub. I really think the best way is to publish and hope for the best.

[deakjahn]

Published. The example project seems to return the stream for me. There's a warning about a stray async left there, already removed from GitHub, it will be replaced in the next publication phase. But it's a benign warning, doesn't make any difference.

[Wissperwind]

Thanks! The github version works perfeklty now. This library is now on the same level with filer_picker in terms of sending large files. Sending large files is now possible up to 537MB. After that errors occour. I think this is flutters fault. See my issue for more Information: https://github.com/flutter/flutter/issues/94104

[deakjahn]

On a completely different note, did you experiment with FTP upload instead of HTTP in your particular case? Not to do with picking files, of course, but it might be more fault-tolerant (continuation of failed transfers, for instance).