Open nielsenko opened 2 years ago
I removed an import from list_test.dart
:
import 'package:realm_dart/src/list.dart';
that contains a class RealmList
that is also imported via
import '../lib/realm.dart';
That contains an export of a file that itself export the same list.dart
showing RealmList
.
Pheww..
Anyway, if I remove the faulty import that was added by the vscode tooling without me being aware, everything works again.
The repo is available on github should you care to investigate. Please reach out if you need any help.
I have created a branch that includes the issue: https://github.com/realm/realm-dart/tree/kn/dart-sdk-49968. Only the last commit differs this branch from a successful build.
I have also created a draft PR for it, to trigger the CI https://github.com/realm/realm-dart/pull/912.
As I said, I don't know if it does you any good, but there be dragons here.
/cc @liamappelbe
@nielsenko I'm having trouble running these tests. From the test errors, it seems like I have to build librealm_dart.so, but I'm getting a cmake error (Error: could not load cache
) when I try to follow the instructions. When I look at the tests readme, it has a lot of instructions about running integration tests. Are test/list_test.dart and test/user_test.dart integration tests, or can I ignore those instructions?
You can actually hit the issue in a clean repo without building the shared lib, by running:
dart --version test -r expanded -j 1 --coverage ./coverage/ test/list_test.dart test/user_test.dart
You will see a lot of failed test like:
Invalid argument(s): Failed to load dynamic library 'librealm_dart.dylib'
But you will still get the crash at the end:
===== CRASH =====
si_signo=Segmentation fault: 11(11), si_code=2, si_addr=0xd8
version=2.18.1 (stable) (Tue Sep 13 11:42:55 2022 +0200) on "macos_arm64"
pid=30759, thread=11787, isolate_group=main(0x1268b8200), isolate=test/user_test.dart(0x1268b2a00)
isolate_instructions=102ab7ea0, vm_instructions=102ab7ea0
pc 0x0000000102c1b868 fp 0x000000016ddc5c50 dart::LongJumpScope::Jump(int)+0x18
pc 0x0000000102c1b850 fp 0x000000016ddc5c80 dart::LongJumpScope::Jump(int)+0x0
pc 0x0000000102cfee58 fp 0x000000016ddc5ca0 dart::Report::LongJumpF(dart::Error const&, dart::Script const&, dart::TokenPosition, char const*, ...)+0x0
pc 0x00000001030f5674 fp 0x000000016ddc5cd0 dart::kernel::TranslationHelper::LookupClassByKernelClass(dart::kernel::NameIndex)+0x0
pc 0x00000001030f573c fp 0x000000016ddc5d10 dart::kernel::TranslationHelper::LookupClassByKernelClass(dart::kernel::NameIndex)+0xc8
pc 0x0000000102c0d398 fp 0x000000016ddc5e50 dart::kernel::CollectConstConstructorCoverageFrom(dart::Script const&)+0x128
pc 0x0000000102d29314 fp 0x000000016ddc5f20 dart::SourceReport::CollectConstConstructorCoverageFromScripts(dart::GrowableArray<dart::SourceReport::ScriptTableEntry*>*, dart::JSONArray*)+0x94
pc 0x0000000102d28b34 fp 0x000000016ddc6020 dart::SourceReport::PrintJSON(dart::JSONStream*, dart::Script const&, dart::TokenPosition, dart::TokenPosition)+0x6ec
pc 0x0000000102d18874 fp 0x000000016ddc61b0 dart::Service::PrintJSONForVM(dart::JSONStream*, bool)+0x74b4
pc 0x0000000102d0feec fp 0x000000016ddc67d0 dart::Service::InvokeMethod(dart::Isolate*, dart::Array const&, bool)+0x2b8
pc 0x0000000102d10500 fp 0x000000016ddc67f0 dart::Service::HandleIsolateMessage(dart::Isolate*, dart::Array const&)+0x18
pc 0x0000000102bf49f0 fp 0x000000016ddc6da0 dart::IsolateMessageHandler::HandleMessage(std::__2::unique_ptr<dart::Message, std::__2::default_delete<dart::Message> >)+0x378
pc 0x0000000102c1ca14 fp 0x000000016ddc6e10 dart::MessageHandler::HandleMessages(dart::MonitorLocker*, bool, bool)+0x154
pc 0x0000000102c1d150 fp 0x000000016ddc6e70 dart::MessageHandler::TaskCallback()+0x204
pc 0x0000000102d3c1f8 fp 0x000000016ddc6f30 dart::ThreadPool::WorkerLoop(dart::ThreadPool::Worker*)+0x14c
pc 0x0000000102d3c5d8 fp 0x000000016ddc6f60 dart::ThreadPool::Worker::Main(unsigned long)+0x78
pc 0x0000000102cb3e2c fp 0x000000016ddc6fc0 dart::OSThread::GetMaxStackSize()+0xac
pc 0x0000000180b9026c fp 0x000000016ddc6fe0 _pthread_start+0x94
-- End of DumpStackTrace
But to build the lib you should be able to run ./scripts/build-macos.sh
or whatever platform you are on.
Better look in https://github.com/realm/realm-dart/blob/master/CONTRIBUTING.md for build instructions, as mentioned in the README.md
.. sorry for the old cruft in build.md
.
Anyway, as I said above, you don't really need to build the lib or setup integration tests to hit the issue.
@liamappelbe Did you get any further?
I was able to repro. I'm still investigating. Nothing to report yet
Traced the crash to an assert (kernel_translation_helper.cc::CheckStaticLookup) that has a TODO pointing to this bug: https://github.com/dart-lang/sdk/issues/37517
@rmacnak-google The conditions you mentioned for triggering #37517 have to do with reloading and mirrors. In this case the crash is happening when gathering coverage for const constructors. Is this another (less obscure) way of triggering the same bug, or is it something different?
00:17 +32 ~10: test/user_test.dart: (tearDownAll)
../../runtime/vm/compiler/frontend/kernel_translation_helper.cc: 533: error: expected: IsolateGroup::Current()->HasAttemptedReload()
version=2.19.0-edge.afae6541cb36b20b3d8e1bfab3964b5ad50f9654 (be) (Tue Sep 20 13:15:49 2022 -0700) on "linux_x64"
pid=3920592, thread=3920836, isolate_group=main(0x556641b69000), isolate=test/user_test.dart(0x556641f8c000)
os=linux, arch=x64, comp=no, sim=no
isolate_instructions=55663e906fe0, vm_instructions=55663e906fe0
pc 0x000055663ed2eb3c fp 0x00007fc9e2ffd8f0 dart::Profiler::DumpStackTrace(void*)+0x7c
pc 0x000055663e9071d4 fp 0x00007fc9e2ffd9d0 dart::Assert::Fail(char const*, ...) const+0x84
pc 0x000055663f2304cb fp 0x00007fc9e2ffd9f0 /usr/local/google/home/liama/dart-sdk/sdk/out/DebugX64/dart-sdk/bin/dart+0x29f84cb
pc 0x000055663f23037b fp 0x00007fc9e2ffda50 dart::kernel::TranslationHelper::LookupLibraryByKernelLibrary(dart::kernel::NameIndex)+0x1cb
pc 0x000055663f23061b fp 0x00007fc9e2ffdab0 dart::kernel::TranslationHelper::LookupClassByKernelClass(dart::kernel::NameIndex)+0x11b
pc 0x000055663ebce1d6 fp 0x00007fc9e2ffdbf0 dart::kernel::CollectConstConstructorCoverageFrom(dart::Script const&)+0x186
pc 0x000055663edd230b fp 0x00007fc9e2ffdc90 dart::SourceReport::CollectConstConstructorCoverageFromScripts(dart::GrowableArray<dart::SourceReport::ScriptTableEntry*>*, dart::JSONArray*)+0x7b
pc 0x000055663edd1cf6 fp 0x00007fc9e2ffdd40 dart::SourceReport::PrintJSON(dart::JSONStream*, dart::Script const&, dart::TokenPosition, dart::TokenPosition)+0x366
pc 0x000055663edbd2b0 fp 0x00007fc9e2ffdea0 /usr/local/google/home/liama/dart-sdk/sdk/out/DebugX64/dart-sdk/bin/dart+0x25852b0
pc 0x000055663edb2910 fp 0x00007fc9e2ffe4b0 dart::Service::InvokeMethod(dart::Isolate*, dart::Array const&, bool)+0x530
pc 0x000055663edb340e fp 0x00007fc9e2ffe4e0 dart::Service::HandleIsolateMessage(dart::Isolate*, dart::Array const&)+0x1e
pc 0x000055663eba71fe fp 0x00007fc9e2ffea80 dart::IsolateMessageHandler::HandleMessage(std::__2::unique_ptr<dart::Message, std::__2::default_delete<dart::Message> >)+0x29e
pc 0x000055663ebe9885 fp 0x00007fc9e2ffeb10 dart::MessageHandler::HandleMessages(dart::MonitorLocker*, bool, bool)+0x215
pc 0x000055663ebea669 fp 0x00007fc9e2ffeb90 dart::MessageHandler::TaskCallback()+0x369
pc 0x000055663edf282b fp 0x00007fc9e2ffec30 dart::ThreadPool::WorkerLoop(dart::ThreadPool::Worker*)+0x1fb
pc 0x000055663edf31d7 fp 0x00007fc9e2ffec80 dart::ThreadPool::Worker::Main(unsigned long)+0x157
pc 0x000055663ed27b14 fp 0x00007fc9e2ffedd0 /usr/local/google/home/liama/dart-sdk/sdk/out/DebugX64/dart-sdk/bin/dart+0x24efb14
-- End of DumpStackTrace
There might be a bug in implementation of const coverage. I think the assertion just indicates that we tried to find some class using its canonical name and that did not work. Have you tried printing what we are trying to lookup and where?
I think it's also worth dumping kernel binary and looking if it is somehow malformed (e.g. does not actually contain the class it refers to by a canonical name).
/cc @jensjoha
The class it's trying to look up is MetadataPersistenceMode
in package:realm_dart/src/app.dart
:
/// Specify if and how to persists user objects.
/// {@category Application}
enum MetadataPersistenceMode {
/// Persist [User] objects, but do not encrypt them.
plaintext,
/// Persist [User] objects in an encrypted store.
encrypted,
/// Do not persist [User] objects.
disabled,
}
Not sure how to get a dill to dump, because the crash only happens inside dart test
, and only if you include both test/list_test.dart and test/user_test.dart. When I compile and dump the kernel of either of them individually, they do contain class MetadataPersistenceMode extends core::_Enum /*isEnum*/ {
.
Actually, list_test.dill contains 2 copies of that class, because it has 2 app libaries: app and app2:
library from "file:///usr/local/google/home/liama/dev/temp/realm-dart/lib/src/app.dart" as app {
library from "package:realm_dart/src/app.dart" as app2 {
Seems weird, but I'm not sure if it's significant or a red herring. I'll continue investigating tomorrow.
library from "file:///usr/local/google/home/liama/dev/temp/realm-dart/lib/src/app.dart" as app { library from "package:realm_dart/src/app.dart" as app2 {
I can't say if this is why it crashes, but it's certainly not great. This has probably happened because something was importatet by file uri (and something correctly by package). This means that there are two of everything defined in that file. Normally this just means that you'll get errors like "Can't assign X to X" etc, but it could potentially lead to a crash as well.
From the above, probably
import '../lib/realm.dart';
is to blame. From a test file (which is not a package uri) it imports it relatively, i.e. still via file uri.
import 'package:realm_dart/realm.dart';
would probably be the right import.
I'm looking into this. I'm pretty sure this is an incremental compiler bug of sorts. I don't think @liamappelbe should currently spent more time on it.
Okay, so this is, simplified, what happens:
test/list_test.dart
which imports a file both via a file uri and a package uri.
uriToSource
--- a map indexed in file uris. As they both have the same file uri (with different import uris) we will have a mapping like {"/path/to/file.dart": {Reference via fileuri, reference via import uri}}
.test/user_test.dart
and return a full component (i.e. we're in an incremental mode but really it shouldn't be noticeable, though here it obviously is).
uriToSource
entry indexed under the file uri which from previously contains both the file and package uri references.(as a fun little something extra, if the second file had imported via package uri instead it wouldn't have crashed because we'd delete the file uri builder and cleaned up the uriToSource
(something we don't do for packages), but now we wouldn't have had any uriToSource
index for the library)
For now the solution is to use package imports in tests (e.g. import 'package:realm_dart/realm.dart';
).
Really, this should be done even if it didn't crash because we won't get duplicate libraries.
We obviously shouldn't crash, though. I'm thinking we should probably also warn the user about this. Even if it didn't crash the user would probably run into something weird (e.g. "X does not assign to X" because one X is from the file import library and one is from the package import library). Possibly the solution is for the compiler to check every file uri import to see if it could be written as a package import, issue a warning and rewrite to actually use the package uri instead to avoid problems. I'm not sure how expensive that is though. I'll look into that tomorrow.
Come to think of it, I'm not sure if the language specs allows us to do that either. Do you have any input @lrhn ?
/cc @johnniwinther for visibility and possible input.
The one general rule about imports is to never have any relative path from a file inside lib
to a file outside of lib
, or vice versa. (Never have /lib/
in the path, never use ..
to exit the lib/
directory.)
Obviously, things shouldn't crash if you do it anyway, you should just get two different libraries based on the same source file. Dart libraries are identified by their URIs, not their file paths.
I do think we should make the failure modes better. That will require a language specification change, but it might be worth it. In this case, and almost any other case, it'd be better to get a compile-time error than to be allowed to import the same library in two different ways. At least you'd know about the issue sooner, and be in a position to fix it.
I do think we should make the failure modes better. That will require a language specification change, but it might be worth it. In this case, and almost any other case, it'd be better to get a compile-time error than to be allowed to import the same library in two different ways. At least you'd know about the issue sooner, and be in a position to fix it.
What about the common pattern where a package has multiple dart files that are imported (using a relative path) by one main file for the package that the user is expected to import? If the user instead imports one of those internal dart files by a package path, would they hit this bug? If so, they wouldn't be able to fix that compile error, unless they own that package.
// Third party code. package:foo/foo.dart
import 'subfoo.dart'; // Relative path import.
// Third party code. package:foo/subfoo.dart
// User code. main.dart
import 'package:foo/foo.dart'; // Package import.
import 'bar.dart'; // Relative path import.
// User code. bar.dart
import 'package:foo/subfoo.dart'; // Package import.
// Does ^this^ conflict with the path import in package:foo/foo.dart?
No. Relative imports are resolved relative to the importers import uri.
In this example we import the package as package:foo/foo.dart
so that becomes the import uri of that library. Then when that file imports subfoo.dart
it's resolved relative to package:foo/foo.dart
and becomes package:foo/subfoo.dart
.
More discussion happening here (including proposal from @lrhn): https://github.com/dart-lang/language/issues/939
I've just started encountering this issue on a large project that has been in production for 2 years. I'm on macos but never encountered this problem before today. Did anyone find a solution?
Issue #56601 suggested using a newer test package fixed the issue for them but I'm running the latest version of dart and have latest version of the test
package.
jimmyff@jimmyff-mbp14 os_core % dart test test/models/profile_test.dart
Resolving dependencies in `/Users/jimmyff/dev/osp/os_core`...
Downloading packages...
Got dependencies in `/Users/jimmyff/dev/osp/os_core`.
Building package executable... (3.0s)
Built test:test.
00:00 +0: loading test/models/profile_test.dart Could not find a command named "/Users/jimmyff/sdks/flutter/bin/cache/dart-sdk/bin/snapshots/frontend_server.dart.snapshot".
Usage: dart <command|dart-file> [arguments]
Global options:
-v, --verbose Show additional command output.
--version Print the Dart SDK version.
--enable-analytics Enable analytics.
--disable-analytics Disable analytics.
--suppress-analytics Disallow analytics for this `dart *` run without changing the analytics configuration.
-h, --help Print this usage information.
Available commands:
analyze Analyze Dart code in a directory.
compile Compile Dart to various formats.
create Create a new Dart project.
devtools Open DevTools (optionally connecting to an existing application).
doc Generate API documentation for Dart projects.
fix Apply automated fixes to Dart source code.
format Idiomatically format Dart source code.
info Show diagnostic information about the installed tooling.
pub Work with packages.
run Run a Dart program.
test Run tests for a project.
Run "dart help <command>" for more information about a command.
See https://dart.dev/tools/dart-tool for detailed documentation.
00:00 +0 -1: loading test/models/profile_test.dart [E]
Failed to load "test/models/profile_test.dart":
SocketException: Write failed (OS Error: Broken pipe, errno = 32), port = 0
dart:io-patch/socket_patch.dart 1252:34 _NativeSocket.write
dart:io-patch/socket_patch.dart 2010:15 _RawSocket.write
dart:io-patch/socket_patch.dart 2487:18 _Socket._write
dart:io-patch/socket_patch.dart 2222:28 _SocketStreamConsumer.write
dart:io-patch/socket_patch.dart 2174:11 _SocketStreamConsumer.addStream.<fn>
dart:async/zone.dart 1422:47 _rootRunUnary
dart:async/zone.dart 1324:19 _CustomZone.runUnary
dart:async/zone.dart 1233:7 _CustomZone.runUnaryGuarded
dart:async/stream_impl.dart 366:11 _BufferingStreamSubscription._sendData
dart:async/stream_impl.dart 297:7 _BufferingStreamSubscription._add
dart:async/stream_controller.dart 777:19 _SyncStreamControllerDispatch._sendData
dart:async/stream_controller.dart 651:7 _StreamController._add
dart:async/stream_controller.dart 606:5 _StreamController.add
dart:io/io_sink.dart 154:17 _StreamSinkImpl.add
dart:io/io_sink.dart 287:5 _IOSinkImpl.write
dart:io-patch/socket_patch.dart 2320:36 _Socket.write
dart:io/stdio.dart 426:13 _StdSink.writeln
package:frontend_server_client/src/frontend_server_client.dart 328:21 FrontendServerClient._sendCommand
package:frontend_server_client/src/frontend_server_client.dart 245:5 FrontendServerClient.accept
package:test_core/src/runner/vm/test_compiler.dart 128:30 _TestCompilerForLanguageVersion._compile
===== asynchronous gap ===========================
package:pool/pool.dart 127:14 Pool.withResource
===== asynchronous gap ===========================
package:test_core/src/runner/vm/platform.dart 241:9 VMPlatform._compileToKernel
===== asynchronous gap ===========================
package:test_core/src/runner/vm/platform.dart 222:13 VMPlatform._spawnIsolate
===== asynchronous gap ===========================
package:test_core/src/runner/vm/platform.dart 75:19 VMPlatform.load
===== asynchronous gap ===========================
package:test_core/src/runner/loader.dart 219:27 Loader.loadFile.<fn>
===== asynchronous gap ===========================
package:test_core/src/runner/load_suite.dart 98:19 new LoadSuite.<fn>.<fn>
To run this test again: /Users/jimmyff/sdks/flutter/bin/cache/dart-sdk/bin/dart test test/models/profile_test.dart -p vm --plain-name 'loading test/models/profile_test.dart'
00:00 +0 -1: Some tests failed.
jimmyff@jimmyff-mbp14 os_core %
Dart SDK version: 3.6.0-216.1.beta (beta) (Wed Sep 4 14:34:46 2024 -0700) on "macos_arm64"
test:
dependency: "direct dev"
description:
name: test
sha256: "7ee446762c2c50b3bd4ea96fe13ffac69919352bd3b4b17bac3f3465edc58073"
url: "https://pub.dev"
source: hosted
version: "1.25.2"
Any advice would be really apprecicated. Thanks
@jimmyff start by triple checking that both test
and its transitive dependencies (like frontend_server_client
) are at their newest versions (e.g. by running dart pub outdated
). If that does not help - file bug against test
package itself: https://github.com/dart-lang/test
Hey @mraleph I restared mac and not sure why but then pub upgrade gave me a slightly newer version... 👀
> test 1.25.8 (was 1.25.2)
Issue no longer present for me, -sorry for the noise!
If I run:
I get a Dart VM crash. It only happens, if I enable coverage. Without the test pass green.
Happens on MacOS for dart version: 2.17.5 and forward, only for a particular branch, and only if I enable coverage. Notice it mentions
test/user_test.dart
, but if I don't includetest/list_test.dart
it don't happen either. Also, the order is important. Anyway the crash looks like this:If I downgrade to 2.17.3, I don't crash, but still get an error from the Dart VM:
I'm not sure if this does you any good, but here you have it.