Can't read file in integration test android

[ductranit]

I had the same problem with this issue So when I run integration test, the Directory.current is / - this is the root dir on my mac device and it's readonly. I was trying to read json file & custom font in the test, so I have to pass the real path on my mac like:

setUpAll(() async {
    var currentDir = Directory.current.path;  // return "/"
    rootPath ='/Users/admin/Documents/flutter/my_app';
   final fontPath = '$rootPath/assets/fonts/custom_font.ttf';
   loader.addFont(File(fontPath)
          .readAsBytes()
          .then((bytes) => ByteData.view(Uint8List.fromList(bytes).buffer)));
   loader.load()
  // load json...
  var json = await File('$rootPath/assets/data.json'). readAsString();
  });

If I run command flutter test integration_test on ios, it works fine, I can load custom font, json & can also take screenshot with this path. But when I run on android device, it throw file not found error with the same path when I load font, json

FileSystemException: Cannot open file, path = '/Users/admin/Documents/flutter/my_app/assets/fonts/custom_font.ttf' (OS Error: No such file or directory, errno = 2)
  dart:io                                                                                                                _File.readAsBytes
  Users/admin/Documents/flutter/my_app/integration_test/app_test.dart 49:11                                        main.<fn>
  Users/admin/Documents/flutter/my_app/integration_test/app_test.dart 42:12                                        main.<fn>
  ===== asynchronous gap ===========================
  package:stream_channel                                                                                                 _GuaranteeSink.add
  var/folders/11/wq1vn8yd2739xsmmzbq3m06w0000gn/T/flutter_tools.6Thidy/flutter_test_listener.DYB5ji/listener.dart 53:22  main.<fn>
  var/folders/11/wq1vn8yd2739xsmmzbq3m06w0000gn/T/flutter_tools.6Thidy/flutter_test_listener.DYB5ji/listener.dart 52:20  main.<fn>

So my questions are:

• Why do integration test has different directory path with unit test according to this issue? The unit flutter has the current project dir by default, but integration test has / as root
• What is the official way to read/write data in integration test? why I can't read file when run on android?

flutter doctor -v

``` [✓] Flutter (Channel master, 2.6.0-12.0.pre.826, on Mac OS X 10.15.7 19H1519 darwin-x64, locale en-US) • Flutter version 2.6.0-12.0.pre.826 at /Users/admin/Documents/Soft/flutter • Upstream repository https://github.com/flutter/flutter.git • Framework revision 6c1455a23a (25 hours ago), 2021-11-23 21:13:05 -0500 • Engine revision 539239e256 • Dart version 2.16.0 (build 2.16.0-42.0.dev) • DevTools version 2.8.0 [✓] Android toolchain - develop for Android devices (Android SDK version 31.0.0) • Android SDK at /Users/admin/Library/Android/sdk • Platform android-31, build-tools 31.0.0 • ANDROID_HOME = /Users/admin/Library/Android/sdk • ANDROID_SDK_ROOT = /Users/admin/Library/Android/sdk • Java binary at: /Applications/Android Studio 4.2 Preview.app/Contents/jre/Contents/Home/bin/java • Java version OpenJDK Runtime Environment (build 11.0.10+0-b96-7281165) • All Android licenses accepted. [!] Xcode - develop for iOS and macOS (Xcode 12.4) • Xcode at /Applications/Xcode.app/Contents/Developer ! Flutter recommends a minimum Xcode version of 13. Download the latest version or update via the Mac App Store. • CocoaPods version 1.11.2 [✓] Chrome - develop for the web • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome [✓] Android Studio (version 2020.3) • Android Studio at /Applications/Android Studio 4.2 Preview.app/Contents • Flutter plugin can be installed from: 🔨 https://plugins.jetbrains.com/plugin/9212-flutter • Dart plugin can be installed from: 🔨 https://plugins.jetbrains.com/plugin/6351-dart • Java version OpenJDK Runtime Environment (build 11.0.10+0-b96-7281165) [✓] VS Code (version 1.58.2) • VS Code at /Applications/Visual Studio Code.app/Contents • Flutter extension version 3.24.0 [✓] VS Code (version 1.63.0-insider) • VS Code at /Applications/Visual Studio Code - Insiders.app/Contents • Flutter extension version 3.28.0 [✓] Connected device (2 available) • macOS (desktop) • macos • darwin-x64 • Mac OS X 10.15.7 19H1519 darwin-x64 • Chrome (web) • chrome • web-javascript • Google Chrome 96.0.4664.55 ```

[darshankawar]

@ductranit Since you are running the integration test on a device, can you try to bundle them in as an asset and then try to access it using rootBundle ?

[ductranit]

@ductranit Since you are running the integration test on a device, can you try to bundle them in as an asset and then try to access it using rootBundle ?

I didn't try it yet, but I need to access the system path to load custom fonts (for golden test), load some test configs, take some screenshots & save to files. How can bundle support for that? I noticed that my test code works when I run integration test on ios, but not for android.

[darshankawar]

How can bundle support for that?

Since you are saving to the files, and the same file if you add it to pubspec.yaml under assets and refer it to something like await rootBundle.loadString('assets/my_text.txt');

[ductranit]

@darshankawar So I tried to read by rootBundle & it works

final bytes = await rootBundle.load('assets/data.json');

But as I said, the problem is that I need to write screenshot to file which bundle can't help. I tried to save screenshot with the path like: screen.png /Users/admin/Documents/flutter/my_app/assets/screen.png assets/screen.png and none of them can work

await binding.convertFlutterSurfaceToImage();
await tester.pumpAndSettle();
final data = await binding.takeScreenshot(name);
 final path = 'screen.png';  // or `/Users/admin/Documents/flutter/my_app/screen.png` or `assets/screen.png`
final imageFile = File(path);
await imageFile.create(recursive: true);
await imageFile.writeAsBytesSync(data);

It throws exception when write file:

══╡ EXCEPTION CAUGHT BY FLUTTER TEST FRAMEWORK ╞════════════════════════════════════════════════════
The following FileSystemException was thrown running a test:
Cannot create file, path = 'screen.png' (OS Error: Read-only file system, errno = 30)

When the exception was thrown, this was the stack:
#0      _File.create.<anonymous closure> (dart:io/file_impl.dart:255:9)
<asynchronous suspension>
<asynchronous suspension>
(elided 5 frames from dart:async and package:stack_trace)

The test description was:
  Test app flow
════════════════════════════════════════════════════════════════════════════════════════════════════
══╡ EXCEPTION CAUGHT BY FLUTTER TEST FRAMEWORK ╞════════════════════════════════════════════════════
The following message was thrown:
Multiple exceptions (5) were detected during the running of the current test, and at least one was
unexpected.

P/s: I had to change FlutterFragmentActivity to FlutterActivity to take screenshot because of this issue, but it will have a trouble because other plugins require fragment activity

[darshankawar]

@ductranit Thanks for the update. Since it's throwing The following FileSystemException was thrown running a test: Cannot create file, path = 'screen.png' (OS Error: Read-only file system, errno = 30), can you try to pass full path ? ie instead of relative path, try to pass absolute path.

[ductranit]

As I said above, all kinds of path have exception. This is the full path /Users/admin/Documents/flutter/my_app/assets/screen.png when I create file await imageFile.create();

══╡ EXCEPTION CAUGHT BY FLUTTER TEST FRAMEWORK ╞════════════════════════════════════════════════════
The following FileSystemException was thrown running a test:
Cannot create file, path =
'/Users/admin/Documents/flutter/my_app/assets/screen.png' (OS Error: No such
file or directory, errno = 2)

When the exception was thrown, this was the stack:
#0      _File.create.<anonymous closure> (dart:io/file_impl.dart:255:9)
<asynchronous suspension>
<asynchronous suspension>
(elided 5 frames from dart:async and package:stack_trace)

The test description was:
  Test app flow
════════════════════════════════════════════════════════════════════════════════════════════════════
07:12 +0 -1: Test app flow [E]                                                                                                      
  Test failed. See exception logs above.
  The test description was: Test app flow

If I tried with await imageFile.create(recursive: true);, then it throws this exception

══╡ EXCEPTION CAUGHT BY FLUTTER TEST FRAMEWORK ╞════════════════════════════════════════════════════
The following FileSystemException was thrown running a test:
Creation failed, path = '/Users' (OS Error: Read-only file system, errno = 30)

When the exception was thrown, this was the stack:
#0      _Directory.create.<anonymous closure> (dart:io/directory_impl.dart:117:11)
<asynchronous suspension>
<asynchronous suspension>
(elided 5 frames from dart:async and package:stack_trace)

The test description was:
  Test app flow
════════════════════════════════════════════════════════════════════════════════════════════════════
06:18 +0 -1: Test app flow [E]                                                                                                      
  Test failed. See exception logs above.
  The test description was: Test app flow

The problem is that the root path of integration test is / - this is the root dir on my mac device and it's readonly

[darshankawar]

Thanks for the update. I see what you are saying. Can you provide us a minimal code sample, ie, main.dart and integration_test -> app_test.dart file that we can directly use and verify it further ?

[ductranit]

@darshankawar Here is it ui_test.zip This sample is just created by flutter create on latest master I run the test by flutter test integration_test

[darshankawar]

Thanks for the info. I verified this on Android and get same behavior / exception on latest stable and master. Keeping it open for further insights from the team.

app_test.dart

``` import 'dart:io'; import 'package:flutter_test/flutter_test.dart'; import 'package:integration_test/integration_test.dart'; import 'package:ui_test/main.dart' as app; void main() { setUpAll(() async {}); final binding = IntegrationTestWidgetsFlutterBinding.ensureInitialized() as IntegrationTestWidgetsFlutterBinding; testWidgets('Test app flow', (WidgetTester tester) async { app.main(); await tester.pumpAndSettle(); expect(find.byType(app.MyHomePage), findsOneWidget); await _takeScreenshot(tester, binding, 'Home', app.MyHomePage); }); } Future _takeScreenshot( WidgetTester tester, IntegrationTestWidgetsFlutterBinding binding, String name, Type type) async { await binding.convertFlutterSurfaceToImage(); await tester.pumpAndSettle(); final data = await binding.takeScreenshot(name); // root dir const rootDir = '/Users/admin/Documents/flutter/ui_test'; final imageFile = File('$rootDir/assets/$name.png'); print('take screenshot ${imageFile.path} with size ${data.length}'); if (!imageFile.existsSync()) { await imageFile.create(recursive: true); } await imageFile.writeAsBytes(data); } ```

main.dart

``` import 'package:flutter/material.dart'; void main() { runApp(const MyApp()); } class MyApp extends StatelessWidget { const MyApp({Key? key}) : super(key: key); // This widget is the root of your application. @override Widget build(BuildContext context) { return MaterialApp( title: 'Flutter Demo', theme: ThemeData( // This is the theme of your application. // // Try running your application with "flutter run". You'll see the // application has a blue toolbar. Then, without quitting the app, try // changing the primarySwatch below to Colors.green and then invoke // "hot reload" (press "r" in the console where you ran "flutter run", // or simply save your changes to "hot reload" in a Flutter IDE). // Notice that the counter didn't reset back to zero; the application // is not restarted. primarySwatch: Colors.blue, ), home: const MyHomePage(title: 'Flutter Demo Home Page'), ); } } class MyHomePage extends StatefulWidget { const MyHomePage({Key? key, required this.title}) : super(key: key); // This widget is the home page of your application. It is stateful, meaning // that it has a State object (defined below) that contains fields that affect // how it looks. // This class is the configuration for the state. It holds the values (in this // case the title) provided by the parent (in this case the App widget) and // used by the build method of the State. Fields in a Widget subclass are // always marked "final". final String title; @override State createState() => _MyHomePageState(); } class _MyHomePageState extends State { int _counter = 0; void _incrementCounter() { setState(() { // This call to setState tells the Flutter framework that something has // changed in this State, which causes it to rerun the build method below // so that the display can reflect the updated values. If we changed // _counter without calling setState(), then the build method would not be // called again, and so nothing would appear to happen. _counter++; }); } @override Widget build(BuildContext context) { // This method is rerun every time setState is called, for instance as done // by the _incrementCounter method above. // // The Flutter framework has been optimized to make rerunning build methods // fast, so that you can just rebuild anything that needs updating rather // than having to individually change instances of widgets. return Scaffold( appBar: AppBar( // Here we take the value from the MyHomePage object that was created by // the App.build method, and use it to set our appbar title. title: Text(widget.title), ), body: Center( // Center is a layout widget. It takes a single child and positions it // in the middle of the parent. child: Column( // Column is also a layout widget. It takes a list of children and // arranges them vertically. By default, it sizes itself to fit its // children horizontally, and tries to be as tall as its parent. // // Invoke "debug painting" (press "p" in the console, choose the // "Toggle Debug Paint" action from the Flutter Inspector in Android // Studio, or the "Toggle Debug Paint" command in Visual Studio Code) // to see the wireframe for each widget. // // Column has various properties to control how it sizes itself and // how it positions its children. Here we use mainAxisAlignment to // center the children vertically; the main axis here is the vertical // axis because Columns are vertical (the cross axis would be // horizontal). mainAxisAlignment: MainAxisAlignment.center, children: [ const Text( 'You have pushed the button this many times:', ), Text( '$_counter', style: Theme.of(context).textTheme.headline4, ), ], ), ), floatingActionButton: FloatingActionButton( onPressed: _incrementCounter, tooltip: 'Increment', child: const Icon(Icons.add), ), // This trailing comma makes auto-formatting nicer for build methods. ); } } ```

flutter doctor -v

``` [√] Flutter (Channel master, 2.6.0-12.0.pre.975, on Microsoft Windows [Version 10.0.19042.1348], locale en-US) • Flutter version 2.6.0-12.0.pre.975 at D:\FlutterSDK\flutter • Upstream repository https://github.com/flutter/flutter.git • Framework revision 9b1bbdbd16 (2 hours ago), 2021-12-09 00:04:07 -0500 • Engine revision 30b0105c72 • Dart version 2.16.0 (build 2.16.0-85.0.dev) • DevTools version 2.9.1 [√] Android toolchain - develop for Android devices (Android SDK version 31.0.0) • Android SDK at C:\Users\Win\AppData\Local\Android\sdk • Platform android-31, build-tools 31.0.0 • Java binary at: C:\Program Files\Android\Android Studio\jre\bin\java • Java version OpenJDK Runtime Environment (build 11.0.10+0-b96-7249189) • All Android licenses accepted. [√] Chrome - develop for the web • Chrome at C:\Program Files (x86)\Google\Chrome\Application\chrome.exe [√] Visual Studio - develop for Windows (Visual Studio Community 2019 16.11.2) • Visual Studio at C:\Program Files (x86)\Microsoft Visual Studio\2019\Community • Visual Studio Community 2019 version 16.11.31624.102 • Windows 10 SDK version 10.0.19041.0 [√] Android Studio (version 2020.3) • Android Studio at C:\Program Files\Android\Android Studio • Flutter plugin can be installed from: https://plugins.jetbrains.com/plugin/9212-flutter • Dart plugin can be installed from: https://plugins.jetbrains.com/plugin/6351-dart • Java version OpenJDK Runtime Environment (build 11.0.10+0-b96-7249189) [√] VS Code (version 1.60.1) • VS Code at C:\Users\Win\AppData\Local\Programs\Microsoft VS Code • Flutter extension can be installed from: https://marketplace.visualstudio.com/items?itemName=Dart-Code.flutter [√] Connected device (4 available) • SM A260G (mobile) • 5200763ebcfa861f • android-arm • Android 8.1.0 (API 27) • Windows (desktop) • windows • windows-x64 • Microsoft Windows [Version 10.0.19042.1348] • Chrome (web) • chrome • web-javascript • Google Chrome 96.0.4664.45 • Edge (web) • edge • web-javascript • Microsoft Edge 96.0.1054.43 • No issues found! [√] Flutter (Channel stable, 2.8.0, on Microsoft Windows [Version 10.0.19042.1348], locale en-US) • Flutter version 2.8.0 at D:\FlutterSDK\flutter • Upstream repository https://github.com/flutter/flutter.git • Framework revision cf44000065 (8 hours ago), 2021-12-08 14:06:50 -0800 • Engine revision 40a99c5951 • Dart version 2.15.0 [√] Android toolchain - develop for Android devices (Android SDK version 31.0.0) • Android SDK at C:\Users\Win\AppData\Local\Android\sdk • Platform android-31, build-tools 31.0.0 • Java binary at: C:\Program Files\Android\Android Studio\jre\bin\java • Java version OpenJDK Runtime Environment (build 11.0.10+0-b96-7249189) • All Android licenses accepted. [√] Chrome - develop for the web • Chrome at C:\Program Files (x86)\Google\Chrome\Application\chrome.exe [√] Visual Studio - develop for Windows (Visual Studio Community 2019 16.11.2) • Visual Studio at C:\Program Files (x86)\Microsoft Visual Studio\2019\Community • Visual Studio Community 2019 version 16.11.31624.102 • Windows 10 SDK version 10.0.19041.0 [√] Android Studio (version 2020.3) • Android Studio at C:\Program Files\Android\Android Studio • Flutter plugin can be installed from: https://plugins.jetbrains.com/plugin/9212-flutter • Dart plugin can be installed from: https://plugins.jetbrains.com/plugin/6351-dart • Java version OpenJDK Runtime Environment (build 11.0.10+0-b96-7249189) [√] VS Code (version 1.61.0) • VS Code at C:\Users\Win\AppData\Local\Programs\Microsoft VS Code • Flutter extension version 3.27.0 [√] Connected device (4 available) • SM A260G (mobile) • 5200763ebcfa861f • android-arm • Android 8.1.0 (API 27) • Windows (desktop) • windows • windows-x64 • Microsoft Windows [Version 10.0.19042.1348] • Chrome (web) • chrome • web-javascript • Google Chrome 96.0.4664.45 • Edge (web) • edge • web-javascript • Microsoft Edge 96.0.1054.43 • No issues found! ```

/cc @jiahaog

[jiahaog]

Are you running the iOS integration test on a simulator? I'm not sure if that will work on a physical device.

This is probably happening because of the different filesystems used by the device. On iOS simulators, the filesystem of the simulator is the same as the host. Similarly for unit tests, the test run within the flutter tester device which runs on the host and has access to the same host filesystem. This is not the case on iOS physical devices or Android emulators or Android physical devices, which do not share the filesystem of the host.

[ductranit]

@jiahaog yes, I run it on ios simulator. I just try on ios device and got the same android issue. But if the device file system is different with the host, then how can I write file into it during the test? As you can see above, I was trying to use relative path like screen.png or assets/screen.png but none of them can work.

[00SunnyDay00]

any update? same problem~

[00SunnyDay00]

actually, it seems like golden test doesn't work in integration test

[lexxxel]

actually, it seems like golden test doesn't work in integration test

I stumbled across the same issue. Any solutions?

[ductranit]

@lexxxel @00SunnyDay00 I have to put the golden images into flutter assets (define in pubspec.yaml), then copy those golden files into internal storage of simulator/device when starting app and compare goldens

[lexxxel]

@ductranit so you can't use flutter test integration_test --update-golden to automatically create/update those screenshots AND you have to have a separate pubspec.yaml for integration tests (because nobody wants to deploy this screenshots to production)? If so, that makes golden integration tests unpractical.

[ductranit]

@lexxxel actually I did a trick for --update-golden : I upload the new golden into our server, and I have a script to download latest golden before running integration tests. And yes for a separate pubspec.yaml. I run tests & deploy build from CI system, and this issue stops me to do normally. I use separate script for deploy & test app to generate pubspec.yaml & goldens. I know this is an ugly hack,but there is no better way unless flutter dev supports for it.

[lexxxel]

@ductranit I might found a 'better' solution. Requirement: a emulator with root access. I do use (minimal example):

Directory tempDir = await getTemporaryDirectory();
String tempPath = tempDir.path;
await expectLater(introText, matchesGoldenFile(tempPath + "/QrScannerPageModel.verify.png"));

to push the .png into a folder that is predictable and adb pull/push within the ci pipeline to move the files onto the emulator