search

scala / bug

Scala 2 bug reports only. Please, no questions — proper bug reports only.
https://scala-lang.org
232 stars 21 forks source link

investigate 2.13 community build breakage for 2.13.0-RC1 #11453

Closed SethTisue closed 5 years ago

SethTisue commented 5 years ago

current nightly: 2.13.0-pre-59975bb 2.13.0-pre-af24410 2.13.0-pre-b4926ef

I did a sweep through the latest run log at https://scala-ci.typesafe.com/job/scala-2.13.x-integrate-community-build/ , findings as follows (numbers are "number of downstream projects blocked")

advice for those who want to help is now up at https://github.com/scala/community-builds/wiki/Troubleshooting-a-failure . I can expand and update it in response to questions. you can also help simply by eyeballing a failure and investigating it by reading code, and posting your findings here

might possibly have hit as-yet-unknown 2.13 regressions:

I also found some projects that are only failing for trivial-looking reasons. we could submit upstream fixes, or just put fixes or workarounds in our forks, and see if that uncovers any real issues (or, in some cases, unblocks downstream stuff):

and then this is a dilemma:

fixed

not green, but investigated, and no unreported Scala regression seems present

joroKr21 commented 5 years ago

I would like to check the kittens issue if no one beats me to it. How can I reproduce it?

SethTisue commented 5 years ago

How can I reproduce [a failure]?

advice for those who want to help is now up at https://github.com/scala/community-builds/wiki/Troubleshooting-a-failure . I can expand and update it in response to questions

you can also help simply by eyeballing a failure and investigating it by reading code, and posting your findings here

SethTisue commented 5 years ago

@etorreborre what do you make of the specs2-more failure? 

[specs2-more] [error]   x h2 id="1"/> ++ <h3/>).updateHeadAttribute("id", 3) === (<h2 id='3'/> ++ <h3/>)
[specs2-more] [error]  <h2 id="3"/><h3/> != List(<h2 id="3"/>, <h3/>) (HtmlxSpec.scala:15)
[specs2-more] [info]   + h2>hello</h2>.addHeadersAnchors.toString must beMatching("<a name=\"hello\"><h2>hello</h2></a>")
[specs2-more] [info]   the headers methods
[specs2-more] [error]     x collects all headers of a document
[specs2-more] [error]  <h1>title1</h1><h2>title2</h2> != List(<h1>title1</h1>, <h2>title2</h2>) (HtmlxSpec.scala:50)
adriaanm commented 5 years ago

I fixed the scala-js issue in https://github.com/scalacommunitybuild/scala-js/tree/community-build-2.13 but forgot to update the ref I guess -- I'll submit as a PR and see

SethTisue commented 5 years ago

@adriaanm I already pushed scala/community-builds@e5c65e2f38bd8b0e7eb50af6282cb119981214a5, let's assume it will go green, I'll come back to it if not

Jasper-M commented 5 years ago

"base64", // StringBuilder stopped allowing assignment to its .length member? is that an intentional change we made?

This is #11158

SethTisue commented 5 years ago

This is #11158

@adriaanm says a proper fix isn't possible right now, so we'd have to just work around it in base64. (but note that base64 is only blocking one downstream project.)

etorreborre commented 5 years ago

@SethTisue I tried a small change to the expectations in HtmlxSpec: b3649e9e7. Let's see how it goes

SethTisue commented 5 years ago

@etorreborre that does it, thanks!

[info] Project specs2----------------------: SUCCESS (project rebuilt ok)
[info] Project specs2-more-----------------: SUCCESS (project rebuilt ok)
joroKr21 commented 5 years ago

The failing tests in kittens test that the derived Hash is the same as Scala's Hashing and hashCode. I guess the cause is scala/scala#7693.

But I'm not sure:

Sciss commented 5 years ago

I don't really understand the scala-swing errors in https://scala-ci.typesafe.com/job/scala-2.13.x-integrate-community-build/1895/consoleFull They all seem to stem from the fact that sbt doesn't see MapWrapper, SetWrapper, BufferWrapper. But they are in the sources: https://github.com/scala/scala-swing/blob/work/src/main/scala-2.13.0-M5/scala/swing/MapWrapper.scala

Could the problem be that sbt is not assuming source directory src/main/scala-2.13.0-M5 simply because the version is no longer -M5?

smarter commented 5 years ago

Could the problem be that sbt is not assuming source directory src/main/scala-2.13.0-M5 simply because the version is no longer -M5?

Yup. To get something more fine-grained you can set up your own rules in your build, e.g.: https://github.com/akka/akka/blob/65ccada280cf4a08701b0b05f7efab46560447ff/project/AkkaBuild.scala#L128-L137

Sciss commented 5 years ago

Ok, we can adopt that. Should I just push another commit to scala-swing repo with this change?

SethTisue commented 5 years ago

Should I just push another commit to scala-swing repo with this change?

Yes please.

(At least, unless it seems to you, having looked at the code that involved, that there's practically zero chance of there being any Scala regression involved that would require delaying RC1. We don't want to make needless busywork for maintainers.)

SethTisue commented 5 years ago

I've bumped the community build SHA to 2.13.0-pre-59975bb, just in case the scala/bug#11450 fix being in might uncover something else downstream.

Sciss commented 5 years ago

@SethTisue done in https://github.com/scala/scala-swing/commit/732c1cc781f3199f3cc8efccc3cc72c85118208d

SethTisue commented 5 years ago

@Sciss thanks, now I get a different error

[scala-swing] [error] /Users/tisue/community.213/target-0.9.16/project-builds/scala-swing-05f61027f8368c12004507d9dfeb2e29933c4d82/src/main/scala-2.13+/scala/swing/BufferWrapper.scala:51:16: method patchInPlace overrides nothing.
[scala-swing] [error] Note: the super classes of class BufferWrapper contain the following, non final members named patchInPlace:
[scala-swing] [error] def patchInPlace(from: Int, patch: scala.collection.IterableOnce[A], replaced: Int): scala.swing.BufferWrapper[A]
[scala-swing] [error]   override def patchInPlace(from: Int, patch: scala.collection.Seq[A], replaced: Int): this.type = {
[scala-swing] [error]                ^
[scala-swing] [error] /Users/tisue/community.213/target-0.9.16/project-builds/scala-swing-05f61027f8368c12004507d9dfeb2e29933c4d82/src/main/scala/scala/swing/Container.scala:28:21: class Content needs to be abstract. Missing implementation for:
[scala-swing] [error]   def patchInPlace(from: Int, patch: scala.collection.IterableOnce[scala.swing.Component], replaced: Int): Content.this.type // inherited from trait Buffer 
[scala-swing] [error] (Note that scala.collection.IterableOnce[A] does not match scala.collection.Seq[A]: trait Seq in package collection is a subclass of trait IterableOnce in package collection, but method parameter types must match exactly.)
[scala-swing] [error]     protected class Content extends BufferWrapper[Component] {
[scala-swing] [error]                     ^
[scala-swing] [error] /Users/tisue/community.213/target-0.9.16/project-builds/scala-swing-05f61027f8368c12004507d9dfeb2e29933c4d82/src/main/scala/scala/swing/TabbedPane.scala:82:10: object creation impossible. Missing implementation for:
[scala-swing] [error]   def patchInPlace(from: Int, patch: scala.collection.IterableOnce[scala.swing.TabbedPane.Page], replaced: Int): scala.swing.TabbedPane.pages.type // inherited from trait Buffer 
[scala-swing] [error] (Note that scala.collection.IterableOnce[A] does not match scala.collection.Seq[A]: trait Seq in package collection is a subclass of trait IterableOnce in package collection, but method parameter types must match exactly.)
[scala-swing] [error]   object pages extends BufferWrapper[Page] {
[scala-swing] [error]          ^

looks like a normal consequence of https://github.com/scala/scala/pull/7155 / https://github.com/scala/scala/pull/7158 ?

SethTisue commented 5 years ago

@xerial wdyt about the airframe failure, could it be caused by a regression in Scala?

[airframe] [info] JDBCCodecTest:
[airframe] [info] - ResultSet to JSON maps *** FAILED ***
[airframe] [info]   "{"id":[2,"name":"yui]"}" was not equal to "{"id":[1,"name":"leo]"}" (JDBCCodecTest.scala:187)
Sciss commented 5 years ago

@SethTisue yes I think so; we can patch this when the RC1 artifact is available, I guess?

SethTisue commented 5 years ago

@Sciss yup

xerial commented 5 years ago

Let me check about the airframe failure

xerial commented 5 years ago

@SethTisue Yeah. This is possibly a regression around TraversableOnce.toIndexedSeq or Iterator in Scala collection.

xerial commented 5 years ago

It seems Iterator's hasNext() is called twice in some code, so if we use code like JDBC's ResultSet.next() that changes the internal state, some elements in an Iterator is skipped at toIndexedSeq. I'll file this as another ticket.

xerial commented 5 years ago

@SethTisue Reproduced this bug and filed at https://github.com/scala/bug/issues/11455

eed3si9n commented 5 years ago

scala-parallel-collections: Thanks to https://github.com/scala/scala/pull/7940, scala-parallel-collections seems to work fine locally using 2.13.0-pre-59975bb.

It's currently pinned to an old commit, so I'll send an update for that once I bump the stability test - https://github.com/scala/scala-parallel-collections/pull/61

eed3si9n commented 5 years ago

scopt: @xuwei-k sent me a PR, which should fix it - https://github.com/scopt/scopt/pull/240

Philippus commented 5 years ago

base64 seems to already have a workaround by @xuwei-k: https://github.com/marklister/base64/pull/19

xerial commented 5 years ago

@SethTisue Fixed this issue at airframe-side for now https://github.com/wvlet/airframe/commit/6c542eb36188293491f9754cc9b010d95bbdff29

Philippus commented 5 years ago

scala-xml-quote fix: https://github.com/densh/scala-xml-quote/pull/18, seems the community build uses the fork at https://github.com/allanrenucci/scala-xml-quote though.

Philippus commented 5 years ago

I think this should fix the scalariform error: https://github.com/scala-ide/scalariform/pull/276

SethTisue commented 5 years ago

@joroKr21 the kittens thing looks like it might better be discussed on a kittens ticket. I've moved kittens to the "not green, but investigated, and no unreported Scala regression seems present" section

SethTisue commented 5 years ago

@xerial in airframe, I'm now getting serialization related test failures:

[airframe] [error] Failed tests:
[airframe] [error]  wvlet.airframe.SerializationTest
[airframe] [error]  wvlet.airframe.DesignSerializationTest
[airframe] [error]  wvlet.airframe.ProviderSerializationTest

this was in a local test but the same error should show up in https://scala-ci.typesafe.com/job/scala-2.13.x-integrate-community-build/1902/

Philippus commented 5 years ago

twotails fix: https://github.com/wheaties/TwoTails/pull/34

SethTisue commented 5 years ago

thanks everyone for all the excellent help! as they come in, I'm pulling in fixes, marking comments as "resolved", and updating the statuses above

SethTisue commented 5 years ago

@Philippus next scala-xml-quote errors are:

[scala-xml-quote] [error] /Users/tisue/community.213/target-0.9.16/project-builds/scala-xml-quote-2ba1cf1077e723d34a4d21d5a898d1ac25fc1aeb/src/test/scala/scala/xml/quote/SimpleNodeSuite.scala:6:30: value ≈ is not a member of scala.xml.NodeSeq
[scala-xml-quote] [error]     assert(xml"<foo/><bar/>" ≈ <foo/><bar/>)
[scala-xml-quote] [error]                              ^
[scala-xml-quote] [error] /Users/tisue/community.213/target-0.9.16/project-builds/scala-xml-quote-2ba1cf1077e723d34a4d21d5a898d1ac25fc1aeb/src/test/scala/scala/xml/quote/SimpleNodeSuite.scala:109:66: value ≈ is not a member of scala.xml.NodeSeq
[scala-xml-quote] [error]     assert(xml"<![CDATA[hello, world]]><![CDATA[hello, world]]>" ≈
[scala-xml-quote] [error]                                                                  ^
[scala-xml-quote] [error] /Users/tisue/community.213/target-0.9.16/project-builds/scala-xml-quote-2ba1cf1077e723d34a4d21d5a898d1ac25fc1aeb/src/test/scala/scala/xml/quote/UnquoteSuite.scala:20:23: type mismatch;
[scala-xml-quote] [error]  found   : scala.xml.NodeBuffer
[scala-xml-quote] [error]  required: String | Seq[scala.xml.Node] | Option[Seq[scala.xml.Node]]
[scala-xml-quote] [error]        
[scala-xml-quote] [error]     assert(xml"<foo a=${<bar/><bat/>}/>" ≈ <foo a={<bar/><bat/>}/>)
[scala-xml-quote] [error]                       ^

note that I switched from allenrenucci to densh

godenji commented 5 years ago

I think this should fix the scalariform error: scala-ide/scalariform#276

Scalariform 0.2.8 released, should be available in 30 minutes or so...

Philippus commented 5 years ago

scapegoat: https://github.com/scalacommunitybuild/scapegoat/pull/1, hopefully it just works without this option enabled.

xerial commented 5 years ago

@SethTisue That serialization error seems related to https://github.com/scala/scala/pull/7624.

SethTisue commented 5 years ago

@Philippus it appears from the next error that scapegoat hasn't moved to the new collections yet:

[scapegoat] [error] /Users/tisue/community.213/target-0.9.16/project-builds/scapegoat-faa3d8e37f7954c9d1aabc690b81dc42fa649c70/src/main/scala/com/sksamuel/scapegoat/Feedback.scala:17:61: type mismatch;
[scapegoat] [error]  found   : scala.collection.mutable.ListBuffer[com.sksamuel.scapegoat.Warning]
[scapegoat] [error]  required: Seq[com.sksamuel.scapegoat.Warning]
[scapegoat] [error]   def warnings(level: Level): Seq[Warning] = warnings.filter(_.level == level)
[scapegoat] [error]                                                             ^
[scapegoat] [error] /Users/tisue/community.213/target-0.9.16/project-builds/scapegoat-faa3d8e37f7954c9d1aabc690b81dc42fa649c70/src/main/scala/com/sksamuel/scapegoat/Feedback.scala:17:61: type mismatch;
[scapegoat] [error]  found   : scala.collection.mutable.ListBuffer[com.sksamuel.scapegoat.Warning]
[scapegoat] [error]  required: Seq[com.sksamuel.scapegoat.Warning]
[scapegoat] [error]   def warnings(level: Level): Seq[Warning] = warnings.filter(_.level == level)
[scapegoat] [error]                                                             ^
SethTisue commented 5 years ago

for scodec, we are using the series/1.10.x branch, but the series/1.11.x branch is where 2.13 support is. so I think the next step would be to try that branch instead, and then also consider backporting the change to the 2.12 community build

UPDATE: scala/community-builds@2b42ac7bb2c42226cabc60b6cbb9b0a3899ecd5d made scodec green in a local test, perhaps it will unblock something downstream

xerial commented 5 years ago

@SethTisue Fixed the serialization error of airframe by forking the JVM: https://github.com/wvlet/airframe/commit/4f9dc062310133ea76b7013951d6b669e01ed9ae

The cause is sbt's class loader issue, which doesn't work well with Java's ObjectSerializer to serialize Scala 2.13.0's new collections. This issue is already addressed in sbt-1.3.0's layered class loader (cc: @eed3si9n)

SethTisue commented 5 years ago

@xerial yup, that does it

94 green projects now :+1:

SethTisue commented 5 years ago

we're at 94 green projects

removed the "blocker" label, as I believe that except for scala-java8-compat, we're in good enough shape here for RC1, though work here will continue up through the 2.13.0 release and beyond

tracking scala-java8-compat-in-the-community-build separately at https://github.com/scala/bug/issues/11452

Philippus commented 5 years ago

looks like these need quite some work to go to scala 2.13.x: "scala-xml-quote" (would need to go to fastparse 2 I think), "scapegoat" and "scribe"

SethTisue commented 5 years ago

@Philippus thanks for checking on it. I removed them from the list.

Philippus commented 5 years ago

@SethTisue I was looking at "scala-logging", I think the specs could be fixed with some clever annotations or asInstanceOf-stuff. But I think it requires a nice proper fix, as it will probably hit real end-users of "scala-logging" eventually. Maybe open an issue on the project?

SethTisue commented 5 years ago

@Philippus yes please :-)

(I would eventually get to it myself, but not this week)

SethTisue commented 5 years ago

3 "http4s-parboiled2", // diverging implicit expansion -- could be a Scala regression, looks tricky to troubleshoot 1 "parboiled2", // same as http4s-parboiled2

this is not eyeball-able, too hairy to troubleshoot ourselves, let's not hold the release up over it

1 "circe-jackson", // appears to have run afoul of recent (Feb? Mar?) changes to overloading resolution? could possibly be a Scala regression

also hairy, very likely fixable in a minor version even if turns out to be Scala's fault, ditto and not holding things up

SethTisue commented 5 years ago

there have been 4 regressions:

Adriaan and team are looking into the remaining two

sjrd commented 5 years ago

scala-js should be back on track, now. Let me know if something breaks again.