dart-lang / sdk

The Dart SDK, including the VM, JS and Wasm compilers, analysis, core libraries, and more.
https://dart.dev
BSD 3-Clause "New" or "Revised" License
10.12k stars 1.57k forks source link

The dart server was terminated when uploading files with `/snap/flutter/130/dart.sh: line 20: 130841 Killed $DART "$@"` #51814

Open SittiphanSittisak opened 1 year ago

SittiphanSittisak commented 1 year ago

I am using Ubuntu 20.04 (LTS) x64, NGINX, memory 1 GB, Dart SDK version: 2.19.4 (stable), and Dart server. I run this dart server in the console with:

pm2 start dart --name dart_server -- run dart_server/bin/dart_server.dart
pm2 logs

The issue happened when the server tried to save a file(image) to the folder in the server(it happened in the host but it will be fine if run on my PC). This issue is the dart server terminated and gives an error:

/snap/flutter/130/dart.sh: line 20: 130841 Killed                  $DART "$@"

The example processing of my code is shown in the image below. image

I tried --old_gen_heap_size=2048 and this did not help anything. After I tried to check this issue, I found the issue happened when the server is uploading files. This error will happen in the save file to a directory step. The server can receive files of more than 2MB and 100MB but it will error in this step with:

I tried to find this issue on the internet but not found anything. For using the chat GPT, it said this error indicates about of memory. but my host memory is 1GB and this step just uploads files lower than 3 MB.

I don't know why this files size sums (size <= 2 MB) more than 2-3 MB issue is happening. In my opinion When the first file(2 MB) is saved the memory used for the first file should be free then the second file should save with success like this

Isn't it? How to fix this? I want to make the server can save files with a file size maximum lower than 10MB and a sum of files size lower than 1000 MB. (my server can receive but can't save these files)

I am not sure if this error seems like not happened to other people, but I can't find anyone that faces this issue.

julemand101 commented 1 year ago

Are there any information about the reason for killing the application in your dmesg log? I can see you are running Dart from a snap so I wonder if there are some additional process limitations being applied to Dart. Could you try download Dart from https://dart.dev/get-dart/archive and use that version instead?

Would also be interesting if you try compile the application with dart compile exe (https://dart.dev/tools/dart-compile#exe) to see if that makes a difference.

About the code itself, would it be possible for you to make a small example project which you can share and which does reproduce the issue? I doubt your code looks like your screenshots since uploadFile() does not take any parameters.

SittiphanSittisak commented 1 year ago

Are there any information about the reason for killing the application in your dmesg log?

I read it in /var/log/dmesg I compare this file before making the app terminate and after the app terminated, and I have not found a difference between them. So, I think there is no information about the reason for killing. Do I understand right?

would it be possible for you to make a small example project which you can share and which does reproduce the issue?

An example in the dart server: After I created the example code, I tried it and it work successfully, I will check my code and I will update it.

mit-mit commented 1 year ago

Sorry @SittiphanSittisak this issue is not actionable as-is. We need a clear set of repro steps.

SittiphanSittisak commented 1 year ago

Sorry for replying late. After I tried all day, I "think" this issue happened when I import the mongo_dart package.

I tested by uploading 2MB x10 images on the flutter app. If I don't import the mongo_dart in the example, the dart server can work fine but if I import it, the dart server will terminate during resizing an image. But sometimes it works fine although I import this package and the dart server can be terminated again if I spam send requests (It works fine if I don't import this package).

I tried to add a delay for the uploading process and it can solve it(at least 1s)!?

example code:

import 'dart:io';
import 'dart:typed_data';
import 'package:shelf/shelf.dart';
import 'package:shelf/shelf_io.dart' as shelf_io;
import 'package:shelf_multipart/form_data.dart';
import 'package:shelf_router/shelf_router.dart' as shelf_router;
import 'package:image/image.dart';
import 'package:mongo_dart/mongo_dart.dart';

Future main(List<String> arguments) async {
  final cascade = Cascade().add(_router);
  final server = await shelf_io.serve(logRequests().addHandler(cascade.handler), InternetAddress.anyIPv4, 8050); //production localhost
  print('\nServer is running\naddress:${server.address.address}\nhost:${server.address.host}\nport: ${server.port}\n');
}

final _router = shelf_router.Router()
  ..post('/test', (Request request) async {
    try {
      print('\nstart get file from the request');
      Map<String, dynamic> bodyFile = {}; //Similar to $_FILES in the PHP
      final List<FormData> formDataList = await request.multipartFormData.toList();
      for (FormData formData in formDataList) {
        if (formData.filename is String) bodyFile[formData.name] = await formData.part.readBytes();
      }
      print('end  get file from the request\n');

      int time = DateTime.now().microsecondsSinceEpoch;
      for (MapEntry e in bodyFile.entries) {
        //await Future.delayed(Duration(seconds: 1)); can fix this!?
        if (e.value is! Uint8List) continue;
        final newPath = 'public_storage/test/${time++}.jpg';
        final File file = File(newPath);
        file.writeAsBytesSync(resizeImage(e.value)!, mode: FileMode.append);
        print('$newPath was uploaded');
      }

      return Response(HttpStatus.ok);
    } catch (e) {
      print('e: $e');
      return Response(HttpStatus.badRequest);
    }
  })
  ..all('/<ignored|.*>', (Request request) {
    return Response(HttpStatus.notFound);
  });

Uint8List? resizeImage(byte) {
  try {
    print('\nstart _resizeImage');
    if (byte is! Uint8List && byte is! List<int>) return null;
    Image image = decodeImage(byte)!;
    final bool isWidthBigger = image.width > image.height;
    late int widthRatio, heightRatio;
    widthRatio = isWidthBigger ? 1280 : 720;
    heightRatio = isWidthBigger ? 720 : 1280;
    image = isWidthBigger ? copyResize(image, width: widthRatio, height: (image.height * widthRatio) ~/ image.width) : copyResize(image, width: (image.width * heightRatio) ~/ image.height, height: heightRatio);
    return Uint8List.fromList(encodeJpg(image, quality: 90));
  } catch (e) {
    return null;
  } finally {
    print('end _resizeImage\n');
  }
}
dependencies:
  image: ^4.0.15
  shelf: ^1.4.0
  shelf_router: ^1.1.3
  shelf_multipart: ^1.0.0
  mongo_dart: ^0.9.1

At the moment, I fix it by await Future.delayed(Duration(seconds: 1));. I hope for a good solution instead of this. This solution takes too much time because every step has to use a delay after the process (rotate image -> delay 1 sec -> resize image -> delay 1 sec -> upload image -> delay 1 sec -> next image...).

SittiphanSittisak commented 1 year ago

This issue is similar to this. The html_universal package makes it an error too. but these packages make the server terminate after running the server(the server can't run). But for this topic, the server terminates with large or too many images.