Very inefficient RAM usage

[Andreigr0]

I'm not sure whether it's right to compare these methods directly as they work completely different, but I'm sure that the library is very unoptimized and consumes a lot of RAM: opening an 4 Mb excel file (15k rows) uses 1 GB of RAM and there is linear dependency – the more file size is the more RAM is consumed (i.e. 8 Mb file would consume 2 GB of RAM) while originial Microsoft's Excel consumes ~250 Mb of RAM with the same 8 mb file

Method I used (use only one logTimeAndRam otherwise they may interfere each other):

import 'dart:io';

import 'package:archive/archive_io.dart';
import 'package:excel/excel.dart';
import 'package:test/test.dart';

void main() {
  final path = 'test/some_excel_file.xlsx';

  print('File:');
  final file = File(path);
  final bytesSync = file.readAsBytesSync();
  // logTimeAndRam(() => ZipDecoder().decodeBytes(bytesSync)); // Memory used: 0.56 MB (576.0 KB) in 4 ms
  logTimeAndRam(() => Excel.decodeBytes(bytesSync)); // Memory used: 1879.53 MB (1924640.0 KB) in 10344 ms

  // print('\nInputFileStream:');
  // final input = InputFileStream(path);
  // logTimeAndRam(() => ZipDecoder().decodeBuffer(input)); // Memory used: 0.64 MB (656.0 KB) in 6 ms
  // logTimeAndRam(() => Excel.decodeBuffer(input)); // Memory used: 1814.75 MB (1858304.0 KB) in 10344 ms
}

void logTimeAndRam(void Function() fn) {
  final startRss = ProcessInfo.currentRss;
  final start = DateTime.now();

  fn();

  final bytes = ProcessInfo.currentRss - startRss;
  final time = DateTime.now().difference(start).inMilliseconds;
  final kilobytes = bytes / 1024;
  final megabytes = (kilobytes / 1024).toStringAsFixed(2);

  print('Memory used: $megabytes MB ($kilobytes KB) in $time ms');
}

Though I also don't provide any solution for this, probably it's more like a reminder to do something about this

[danieledas]

Out of curiosity, which platform did you test this on?

[Andreigr0]

Out of curiosity, which platform did you test this on?

macOS 14.4.1