dart-lang / co19

A Dart language and library conformance test suite
BSD 3-Clause "New" or "Revised" License
37 stars 28 forks source link

Failures on [co19] Roll co19 to 9934d8540d097366287267519413e888a7f6a482 #596

Closed athomas closed 4 years ago

athomas commented 4 years ago

There are new test failures on [co19] Roll co19 to 9934d8540d097366287267519413e888a7f6a482.

The tests

co19/LanguageFeatures/nnbd/expression_typing_A01_t01 MissingCompileTimeError (expected Pass)
co19/LanguageFeatures/nnbd/expression_typing_A02_t01 CompileTimeError (expected Pass)
co19/LanguageFeatures/nnbd/expression_typing_A03_t01 CompileTimeError (expected Pass)
co19/LanguageFeatures/nnbd/expression_typing_A06_t01 MissingCompileTimeError (expected Pass)
co19/LanguageFeatures/nnbd/static_errors_A05_t04 MissingCompileTimeError (expected Pass)
co19/LanguageFeatures/nnbd/static_errors_A05_t09 MissingCompileTimeError (expected Pass)
co19/LanguageFeatures/nnbd/static_errors_A05_t10 MissingCompileTimeError (expected Pass)
co19/LanguageFeatures/nnbd/static_errors_A05_t13 MissingCompileTimeError (expected Pass)
co19/LanguageFeatures/nnbd/static_errors_A26_t03 MissingCompileTimeError (expected Pass)

are failing on configurations

analyzer-asserts-strong-linux
cfe-strong-linux
dartdevk-strong-linux-release-chrome
sgrekhov commented 4 years ago

@athomas co19/LanguageFeatures/nnbd/static_errors_A05_t* failures on analyzer are very strange. For example, according to the log

/=======================================================================================================================================\
| analyzer-asserts-strong-linux:co19/LanguageFeatures/nnbd/static_errors_A05_t04 broke (Pass -> MissingCompileTimeError, expected Pass) |
\=======================================================================================================================================/

--- Command "dart2analyzer" (took 26ms):
DART_CONFIGURATION=ReleaseX64NNBD out/ReleaseX64NNBD/dart-sdk/bin/dartanalyzer --enable-experiment=non-nullable --ignore-unrecognized-flags --packages=/b/s/w/ir/cache/builder/sdk/.packages --format=machine --no-hints /b/s/w/ir/cache/builder/sdk/tests/co19/src/LanguageFeatures/nnbd/static_errors_A05_t04.dart

static error failures:
Unexpected static error at line 32, column 3, length 2:
- Had error code COMPILE_TIME_ERROR.NOT_INITIALIZED_NON_NULLABLE_INSTANCE_FIELD.

Unexpected static error at line 45, column 3, length 2:
- Had error code COMPILE_TIME_ERROR.NOT_INITIALIZED_NON_NULLABLE_INSTANCE_FIELD.

--- Re-run this test:
python tools/test.py -n analyzer-asserts-strong-linux co19/LanguageFeatures/nnbd/static_errors_A05_t04

But these errors are expected on lines 32 and 45. See the test

/*
 * Copyright (c) 2020, the Dart project authors.  Please see the AUTHORS file
 * for details. All rights reserved. Use of this source code is governed by a
 * BSD-style license that can be found in the LICENSE file.
 */
/**
 * @assertion It is an error if a class declaration declares an instance
 * variable with a potentially non-nullable type and no initializer expression,
 * and the class has a generative constructor where the variable is not
 * initialized via an initializing formal or an initializer list entry, unless
 * the variable is marked with the late modifier.
 *
 * @description Check that it is an error if a class declaration declares an
 * instance variable with a potentially non-nullable type and no initializer
 * expression, and the class has a generative constructor where the variable is
 * not initialized via an initializing formal or an initializer list entry,
 * unless the variable is marked with the late modifier. Test some class A
 * @author sgrekhov@unipro.ru
 * @issue 40951
 */
// SharedOptions=--enable-experiment=non-nullable
// Requirements=nnbd-strong
class A {}

class C1 {
  A a;
//  ^
// [cfe] unspecified
  String s;
//       ^
// [cfe] unspecified
  C1() {}
//^^
// [analyzer] COMPILE_TIME_ERROR.NOT_INITIALIZED_NON_NULLABLE_INSTANCE_FIELD

}

abstract class C2 {
  A a;
//  ^
// [cfe] unspecified
  String s;
//       ^
// [cfe] unspecified
  C2() {}
//^^
// [analyzer] COMPILE_TIME_ERROR.NOT_INITIALIZED_NON_NULLABLE_INSTANCE_FIELD
}

class C3 extends C2 {}

main() {
  new C1();
  new C3();
}

Do you have any ideas why these tests fail on bots? There are no failures if I run these tests localy

athomas commented 4 years ago

I don't see what's wrong here either.

@munificent Any ideas what's going on here?

sgrekhov commented 4 years ago

Triage results

co19/LanguageFeatures/nnbd/expression_typing_A01_t01 MissingCompileTimeError (expected Pass) 41282
co19/LanguageFeatures/nnbd/expression_typing_A02_t01 CompileTimeError (expected Pass) 41273
co19/LanguageFeatures/nnbd/expression_typing_A03_t01 CompileTimeError (expected Pass) 41273
co19/LanguageFeatures/nnbd/expression_typing_A06_t01 MissingCompileTimeError (expected Pass) 41283
co19/LanguageFeatures/nnbd/static_errors_A05_t04 MissingCompileTimeError (expected Pass) Infrastructure issue? There must be no this failure
co19/LanguageFeatures/nnbd/static_errors_A05_t09 MissingCompileTimeError (expected Pass) Infrastructure issue? There must be no this failure
co19/LanguageFeatures/nnbd/static_errors_A05_t10 MissingCompileTimeError (expected Pass) Infrastructure issue? There must be no this failure
co19/LanguageFeatures/nnbd/static_errors_A05_t13 MissingCompileTimeError (expected Pass) Infrastructure issue? There must be no this failure
co19/LanguageFeatures/nnbd/static_errors_A26_t03 MissingCompileTimeError (expected Pass) Infrastructure issue? There must be no this failure
sgrekhov commented 4 years ago

598 and #599 explained staticerrors* tests failures. Tests fixed. Closing this issue