search

scala / scala-dev

Scala 2 team issues. Not for user-facing bugs or directly actionable user-facing improvements. For build/test/infra and for longer-term planning and idea tracking. Our bug tracker is at https://github.com/scala/bug/issues
Apache License 2.0
130 stars 15 forks source link

regression: Windows test runs failing in PipelineMainTest, due to sys.process parsing change #814

Closed SethTisue closed 2 years ago

SethTisue commented 2 years ago

@som-snytt this is causing the test suite to time out on Windows — not intermittently, every time, at https://github.com/scala/scala/actions/runs/2907898126 but also subsequent runs since then:

[info] Test scala.tools.nsc.PipelineMainTest.pipelineMainBuildsSeparate started
[3789](https://github.com/scala/scala/runs/8001625648?check_suite_focus=true#step:6:3790)
error: IO error while decoding C:UsersRUNNER~1AppDataLocalTemppipelineBase8762258594693087133projectsm1p1srcm1p1Macro.scala with UTF-8: C:UsersRUNNER~1AppDataLocalTemppipelineBase8762258594693087133projectsm1p1srcm1p1Macro.scala (The system cannot find the file specified)
[3790](https://github.com/scala/scala/runs/8001625648?check_suite_focus=true#step:6:3791)
1 error
[3791](https://github.com/scala/scala/runs/8001625648?check_suite_focus=true#step:6:3792)
Please try specifying another one using the -encoding option
[3792](https://github.com/scala/scala/runs/8001625648?check_suite_focus=true#step:6:3793)
1 error
[3793](https://github.com/scala/scala/runs/8001625648?check_suite_focus=true#step:6:3794)
error: IO error while decoding C:UsersRUNNER~1AppDataLocalTemppipelineBase8762258594693087133projectsm1internalMacroClientsrcm2p2InternalClient.scala with UTF-8: C:UsersRUNNER~1AppDataLocalTemppipelineBase8762258594693087133projectsm1internalMacroClientsrcm2p2InternalClient.scala (The system cannot find the file specified)
[3794](https://github.com/scala/scala/runs/8001625648?check_suite_focus=true#step:6:3795)
Please try specifying another one using the -encoding option
[3795](https://github.com/scala/scala/runs/8001625648?check_suite_focus=true#step:6:3796)

on my Mac, I tried inserting Thread.dumpStack() at the top of sys.process.Parser.tokenize and I see:

    at scala.sys.process.Parser$.tokenize(Parser.scala:34)
    at scala.tools.nsc.settings.MutableSettings.splitParams(MutableSettings.scala:115)
    at scala.tools.nsc.CompilerCommand.expandArg(CompilerCommand.scala:133)
    at scala.tools.nsc.CompilerCommand.$anonfun$processArguments$1(CompilerCommand.scala:142)
    at scala.collection.immutable.List.flatMap(List.scala:293)
    at scala.tools.nsc.CompilerCommand.processArguments(CompilerCommand.scala:141)
    at scala.tools.nsc.CompilerCommand.<init>(CompilerCommand.scala:23)
    at scala.tools.nsc.PipelineMainClass.commandFor$1(PipelineMain.scala:117)

if that's any help. I believe PipelineMain and PipelineMainTest are @retronym territory

som-snytt commented 2 years ago

FSR testAll passes locally running sbt under PowerShell, jdk 15, fresh repo. Not sure yet what differs, presumably the tmp dir location.

But my first thought is to provide a convenience method or interpolator for adding File or Path to a command string with appropriate adjustments (either escaping backslash or converting to forward).

This parsing is done for args supplied as string, not in a list. The quoting feature is for // scalac: options in test files. The current code (on dotty) has a comment that not stripping escape was intended in the sense of accepted limitation. Stripping them is just so it works sanely for common use cases and always the same idiosyncratic shell way.

SethTisue commented 2 years ago

keeping open until we have a green run at https://github.com/scala/scala/actions

som-snytt commented 2 years ago

I can't tell if anything progressed, but there are still path errors in tests. Probably tests that construct a string of args with paths interpolated. I'll take a look during lunch break.

SethTisue commented 2 years ago

For the record, the error currently (at https://github.com/scala/scala/runs/8119344147) is

!!  1 - run/macro_annot.scala                     [output differs]
[13709](https://github.com/scala/scala/runs/8119344147?check_suite_focus=true#step:6:13710)
% diff D:\a\scala\scala\test\scaladoc\run\macro_annot.check D:\a\scala\scala\test\scaladoc\run\macro_annot-run.log--- macro_annot.check
[13710](https://github.com/scala/scala/runs/8119344147?check_suite_focus=true#step:6:13711)
+++ macro_annot-run.log
[13711](https://github.com/scala/scala/runs/8119344147?check_suite_focus=true#step:6:13712)
@@ -1,1 +1,44 @@
[13712](https://github.com/scala/scala/runs/8119344147?check_suite_focus=true#step:6:13713)
-Done.
[13713](https://github.com/scala/scala/runs/8119344147?check_suite_focus=true#step:6:13714)
+D:ascalascalatestscaladocrunmacro_annot-run.obj does not exist or is not a directory
[13714](https://github.com/scala/scala/runs/8119344147?check_suite_focus=true#step:6:13715)
+newSource:5: error: not found: type simulacrum
[13715](https://github.com/scala/scala/runs/8119344147?check_suite_focus=true#step:6:13716)
+@simulacrum trait Applicative[F[_]] {
[13716](https://github.com/scala/scala/runs/8119344147?check_suite_focus=true#step:6:13717)
+ ^
[13717](https://github.com/scala/scala/runs/8119344147?check_suite_focus=true#step:6:13718)
+newSource:10: error: not found: type simulacrum
[13718](https://github.com/scala/scala/runs/8119344147?check_suite_focus=true#step:6:13719)
+  @simulacrum def no1 = ???
[13719](https://github.com/scala/scala/runs/8119344147?check_suite_focus=true#step:6:13720)
+   ^
[13720](https://github.com/scala/scala/runs/8119344147?check_suite_focus=true#step:6:13721)
+newSource:11: error: not found: type simulacrum
[13721](https://github.com/scala/scala/runs/8119344147?check_suite_focus=true#step:6:13722)
+  /** 1 */ @simulacrum def yes1 = ???
[13722](https://github.com/scala/scala/runs/8119344147?check_suite_focus=true#step:6:13723)
+            ^
[13723](https://github.com/scala/scala/runs/8119344147?check_suite_focus=true#step:6:13724)
+java.lang.RuntimeException: Scaladoc Model Test ERROR: No universe generated!
[13724](https://github.com/scala/scala/runs/8119344147?check_suite_focus=true#step:6:13725)
+java.lang.RuntimeException: Scaladoc Model Test ERROR: No universe generated!
[13725](https://github.com/scala/scala/runs/8119344147?check_suite_focus=true#step:6:13726)
+   at scala.tools.partest.ScaladocModelTest.show(ScaladocModelTest.scala:75)
[13726](https://github.com/scala/scala/runs/8119344147?check_suite_focus=true#step:6:13727)
+   at scala.tools.partest.DirectTest.main(DirectTest.scala:117)
SethTisue commented 2 years ago

@som-snytt this is now the only remaining issue preventing us from designating a Scala 2.13.9 release candidate. perhaps we should simply revert https://github.com/scala/scala/pull/10114 (and https://github.com/scala/scala/pull/10123?) and reconsider the change for 2.13.10?

one way or the other I would like to designate a release candidate by the end of the day tomorrow (Fri Sep 2)

som-snytt commented 2 years ago

@SethTisue reverting is OK. Don't let the good interfere with convenience, or whatever the saying is.

SethTisue commented 2 years ago

should I revert 10123 at the same time?

som-snytt commented 2 years ago

@SethTisue that is OK. I did look at fixing the tests but why risk a kerfuffle. "This is how I define 'unwarranted'." To quote Star Trek: The Motion Picture that is playing now.

SethTisue commented 2 years ago

ok will do

SethTisue commented 2 years ago

keeping open until we have a green run at https://github.com/scala/scala/actions

SethTisue commented 2 years ago

green at https://github.com/scala/scala/actions/runs/2974828042 💚