Open bummsa opened 3 months ago
flutter doctor -v [✓] Flutter (Channel stable, 3.19.3, on macOS 14.2.1 23C71 darwin-arm64, locale en-CH) • Flutter version 3.19.3 on channel stable at /Users/jenkins/software/flutter/3.19.3 • Upstream repository https://github.com/flutter/flutter.git • Framework revision ba39319843 (12 days ago), 2024-03-07 15:22:21 -0600 • Engine revision 2e4ba9c6fb • Dart version 3.3.1 • DevTools version 2.31.1
[✗] Android toolchain - develop for Android devices
✗ Unable to locate Android SDK.
Install Android Studio from: https://developer.android.com/studio/index.html
On first launch it will assist you in installing the Android SDK components.
(or visit https://flutter.dev/docs/get-started/install/macos#android-setup for detailed instructions).
If the Android SDK has been installed to a custom location, please use
flutter config --android-sdk
to update to that location.
[✓] Xcode - develop for iOS and macOS (Xcode 15.2) • Xcode at /Applications/Xcode.app/Contents/Developer • Build 15C500b • CocoaPods version 1.15.2
[✓] Chrome - develop for the web • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome
[✓] Android Studio (version 2023.1) • Android Studio at /Applications/Android Studio.app/Contents • Flutter plugin can be installed from: 🔨 https://plugins.jetbrains.com/plugin/9212-flutter • Dart plugin can be installed from: 🔨 https://plugins.jetbrains.com/plugin/6351-dart • Java version OpenJDK Runtime Environment (build 17.0.7+0-17.0.7b1000.6-10550314)
[✓] IntelliJ IDEA Community Edition (version 2023.3.3) • IntelliJ at /Applications/IntelliJ IDEA CE.app • Flutter plugin can be installed from: 🔨 https://plugins.jetbrains.com/plugin/9212-flutter • Dart plugin can be installed from: 🔨 https://plugins.jetbrains.com/plugin/6351-dart
[✓] Connected device (2 available) • macOS (desktop) • macos • darwin-arm64 • macOS 14.2.1 23C71 darwin-arm64 • Chrome (web) • chrome • web-javascript • Google Chrome 120.0.6099.234
[✓] Network resources • All expected network resources are available.
Running build_runner simultaneously in multiple projects that depend on each other is not supported, they don't know about each other and will end up deleting files out from under other instances in the middle of their builds.
Running build_runner simultaneously in multiple projects that depend on each other is not supported, they don't know about each other and will end up deleting files out from under other instances in the middle of their builds.
Hi, but in my case the build_runner do not run simultaneously, they run one after the other
@jakemac53 can you please review the case again? Thank you in advance!
Ah, you are manually doing a build_runner build
in each project individually? Or how are you enforcing that they run one after the other?
Also if you can run with --verbose
that would help.
Ah, you are manually doing a
build_runner build
in each project individually? Or how are you enforcing that they run one after the other?
Yes - The build runner is executed by Jenkins. Basically, I just check all folders within "my_app" and whether the file "pubspec.yaml" exists in the subfolder (shared_project, the app project). And then it checks if the pattern "build_runner" was found in the pubspec.yaml. If build_runner is found, then I'll run the cmd "dart run build_runner build -d"
Could it be a problem with the order in which we call the build_runner?
Here's a small piece of code from the jenkins pipeline:
Also if you can run with
--verbose
that would help.
[FINE] json_serializable on lib/src/workorder_search/ui/workorder_search_list_tile_full.dart:Running JsonSerializableGenerator - 1 of 2 [FINE] json_serializable on lib/src/workorder_search/ui/workorder_search_list_tile_full.dart:Running JsonLiteralGenerator - 2 of 2 [FINE] json_serializable on lib/src/workorder_search/data/web_wo_search_filter_dto.dart:Running JsonSerializableGenerator - 1 of 2 [FINE] json_serializable on lib/src/workorder_search/ui/workorder_preview_view.dart:Running JsonSerializableGenerator - 1 of 2 [FINE] json_serializable on lib/src/workorder_search/ui/workorder_preview_view.dart:Running JsonLiteralGenerator - 2 of 2 [SEVERE] json_serializable on lib/src/workorder_search/data/web_wo_search_filter_dto.dart:
Invalid argument(s): Missing library: package:shared_project_1/enum/filter_enums.dart Libraries: [dart:async, dart:collection, dart:convert, dart:core, dart:developer, dart:ffi, dart:html, dart:html_common, ...]
here it is. And I'm using json_serializable 6.7.1.
The order could matter yes - especially if you are not checking generated files into git. Maybe it is not deterministic which order packages are seen?
The order could matter yes - especially if you are not checking generated files into git. Maybe it is not deterministic which order packages are seen?
Good morning,
My project is pretty big, but here is an advanced example:
My app --> shared_1 --> shared_2 --> shared_3 --> the app
and shared_3 refer to shared_1 and shared_2, which means I have to build 1 and 2 first. So I have ordered the packages, but it still fails from time to time.
Do you have any other idea?
@jakemac53 Can you please review it again? Many thanks in advance!
I don't have any particular idea here, other than the symptom is what you would see when parallel builds happen. Are you sure Jenkins isn't running those steps in parallel? I don't know anything about Jenkins.
I've also been seeing intermittent failure with the same "Invalid argument(s): Missing library" error. For the last week or two after flutter/dart upgrade.
Verbose build for review
I've also been seeing intermittent failure with the same "Invalid argument(s): Missing library" error. For the last week or two after flutter/dart upgrade.
May I ask which OS you use?
Here's a list what I've already tried:
The best way might be to commit the generated files? I develop the app under Windows and have no problems here. The problem only occurs when developing/building under MacOS.
Mac OS Sonoma 14.4
Can we please open this back up? I think the issue is wider than originally reported.
@matt-hall-zory in your scenario is there just a single package that you are building?
Yeah, I'm just building the Zory App, but I'm regularly seeing the same "Missing Library" error as listed above. I don't use build runner watch. I manually run it from time to time, so I often see these errors. For others they might go unnoticed.
Ok, I will re-open given it can manifest in a single package scenario. Not sure if @scheglov has any ideas here. Possibly there has been some change where some behavior is now asynchronous which wasn't before, when adding new files to an analysis context?
I don't remember any recent changes here.
I see that package:zory/providers.dart
is removed because of pending file changes, probably because a file in its transitive dependencies was changed. My understanding is that this happens when you have a LibraryElement
named A
that requires some library L
and so the bundle to load L
is put into LinkedElementFactory
- if it is asked, it will be loaded. But then you apply changes to AnalysisDriver
, which causes it to remove the bundle to L
from LinkedElementFactory
. However you still hold A
, and when you attempt to access L
you find that it is not in LinkedElementFactory
anymore / yet.
To avoid this issue you need to do it sequentially:
Forgive my ignorance but I'm completely baffled by your reply. I'm using a number of packages (riverpod, dart_json_mapper and auto_route) which generate code. Whenever I make changes (add/remove/modify models, files, routes) I run dart run build_runner build --delete-conflicting-outputs. Why, as a developer of an app and not a package, should I have to understand any further than that? I don't even know what an element as you've described it. Is there a build runner clean or some procedure I can do here?
I think his response was for the build_runner authors than for users :).
- If you add new files, or update existing files, accumulate them, don't make these changes during (2).
Unfortunately due to the nature of how build_runner works, I just don't think this at all possible... we have any number of "parallel" build steps going at once, this is a pull-based process and so we can't just delay sending the file updates to the analyzer until some later time, even though we do have "phases" which would be natural split points, we don't run those phases in their entirety before running the next phase, they are only used to establish an ordering between individual build steps.
Essentially, lets consider a 2 layer build, where builders A/B run in "phases" A/B respectively (where phase B is after phase A), on files 1.dart and 2.dart. Lets also say that each of those phases is generating a part file which is included in the library the builder is applied to, and that file 1.dart imports 2.dart.
We start by building the final output, so from the latest phase (phase B). We start running that builder (B) on all the files in that phase (1.dart and 2.dart) at the same time.
These both request a LibraryElement, which will wait for any transitive inputs from previous phases to be built before it can return. So, on 2.dart that waits for the part file (2.a.dart), which we immediately start building. And for 1.dart it waits for both 1.a.dart and 2.a.dart since it depends on both.
So phase A is now also building, running also on 1.dart and 2.dart. Those each resolve their libraries as well, but don't depend on generated files so they are given a library element (note that some of the included parts, 1.b.dart and 2.b.dart are not yet present though).
If 2.a.dart finishes first, we call addFile
immediately, so that we can continue running phase B on 2.dart. But, the builder running in phase A on 1.dart (to produce 1.a.dart), is still running, a builder can reasonably still be using a LibraryElement that it had received, except it is no longer technically valid (one of its transitive deps, 2.a.dart was just added).
I see why this situation is problematic, but it used to work out fine, as long as you didn't reach out and try to use the analysis driver directly, old LibraryElement(s) were still valid?
old LibraryElement(s) were still valid?
Not quite. We load libraries from summaries incrementally, so that we don't spend time and heap loading elements that are never used. But we make sure that when we need something, it is available in form of LibraryReader
. But if there is a file change, we discard all potentially affected readers.
Here is the code that demonstrates the scenario.
solo_test_changeFile_XXX() async {
final a = newFile('$testPackageLibPath/a.dart', r'''
class A {}
''');
final b = newFile('$testPackageLibPath/b.dart', r'''
import 'a.dart';
A foo() {}
''');
// Build summaries.
{
final driver = driverFor(b);
await driver.getLibraryByUri('package:test/b.dart');
}
await disposeAnalysisContextCollection();
// Get the driver that reads libraries from summaries.
final driver = driverFor(b);
// This will make sure that `a` and `b` bundles are loaded for reading.
// And then read `b` bundle into the element.
final bResult = await driver.getLibraryByUri('package:test/b.dart');
bResult as LibraryElementResult;
final bUnit = bResult.element.definingCompilationUnit;
final foo = bUnit.functions.single;
// Notify that `a` changed.
driver.changeFile2(a);
// Give the driver enough time to process all scheduled events.
// This will unload `a` and `b`.
// Now we hold an invalidated `b` element.
await pumpEventQueue(times: 5000);
// This will need to instantiate `A` class.
// It will fail because `a` is not loaded anymore.
print(foo.returnType);
}
Yeah, I do understand the issue at play here. It seems to be new though? Or somehow only manifests now when it didn't used to. I haven't ever seen this error reported until this issue.
And, I don't have any proposal for a fix. It seems fundamentally at odds with how build_runner does builds.
It's probably not useful for you all but probably useful for any developers who stumble on this in the meantime. I found a workaround. Each time I get a "missing library" error, I touch that file with a comment. Then re-run build_runner. After running a few times, touching each file with an error I'm able to complete the build_runner process. I then remove the comments and push.
Hi there,
I'm having an issue building my Flutter app for iOS for about every 2-3 builds during code generation and it occurs randomly and I can't recognize a pattern.
My project structure looks like this:
my_app --> shared_project_1 --> shared_project_2 --> application_project
I run the build_runner in all projects and it fails randomly in any of the 3 subprojects and any dto's I have in the projects with the following error:
Invalid argument(s): Missing library: package:shared_project_1/shared/dto/fruit_dto.dart Libraries: [dart:async, dart:collection, dart:convert, dart:core, ...]
and the fruit_dto looks like this:
`import ...;
part 'fruit_dto.g.dart';
@JsonSerializable() class FruitDto extends Dto { final String name; final String shape; final String color;
FruitDto({ int? id, int? noi, super.entityTag, required this.name, required this.shape, this.color = '', }) : super(containerId: id, noI: noi);
static FruitDto fromJson(Map<String, dynamic> json) => $FruitDtoFromJson(json); @override Map<String, dynamic> toJson() => $FruitDtoToJson(this); }`
It lists like all the classes I have in my project, including some Flutter packages, such as "package:flutter/src/foundation/print.dart".
flutter sdk: 3.19.3 (reproducible with 3.16.x as well) build_runner: 2.4.8 json_serializable: 6.7.1
Mac Mini with M2 Pro: 14.2.1
Does anyone have any ideas on how to fix the issue?
Thanks and best regards bummsa