palday commented 4 days ago

529 raised the issue that our Julia compat bounds may not be correct in practice -- although we don't use any language features requiring newer Julia versions, our dependencies may not be resolvable on older Julia version.

In other words, the problem isn't RCall itself but rather the ability to resolve dependencies with appropriate compatibility on a given Julia version. There's also the issue of R 4 compatibility to throw a further wrench into the works, so I did all of the following bisection with R 3.6.1.

Let's try an easy bisection first, just looking at what RCall commits successfully build on which Julia versions.

bash$ git bisect start master 2eac8c9
bash$ git bisect run julia +1.0 --project=. -e'using Pkg; Pkg.update(); Pkg.build()

(2eac8c9794eb86d363f65c77e72c4a1f01cb4bce introduced Project.toml)

julia version	first bad commit	last good RCall version
1.0	52a97c561f98e5a8274da3c610af86bc75b1e41c	0.13.8
1.1	52a97c561f98e5a8274da3c610af86bc75b1e41c	0.13.8
1.2	52a97c561f98e5a8274da3c610af86bc75b1e41c	0.13.8
1.3	52a97c561f98e5a8274da3c610af86bc75b1e41c	0.13.8
1.4	62ae259859f700753284de7d4d046da9313baae1 (the commit that bumps Julia compat!)	-

Hmmm, that's strange, latest RCall builds just fine on Julia 1.4.... Let's strengthen the bisection test to includes passing tests:

bash$ git bisect start master 2eac8c9
bash$ git bisect run julia +1.0 --project=. -e'using Pkg; Pkg.update(); Pkg.build(); Pkg.test()

julia version	first bad commit	last good RCall version
1.0	0829af8845c8a5a47f644118f655b8e974871b9a	0.13.2
1.1	0829af8845c8a5a47f644118f655b8e974871b9a	0.13.2
1.2	0829af8845c8a5a47f644118f655b8e974871b9a	0.13.2
1.3	0829af8845c8a5a47f644118f655b8e974871b9a	0.13.2
1.4	0829af8845c8a5a47f644118f655b8e974871b9a	0.13.2
1.5	0829af8845c8a5a47f644118f655b8e974871b9a	0.13.2
1.6	0829af8845c8a5a47f644118f655b8e974871b9a	0.13.2
1.10	0829af8845c8a5a47f644118f655b8e974871b9a ?!?!?	0.13.2

Okay, everything fails but fails waaaaay too early in the history. This tells me that there's some incompatibility / incorrect compat bounds in the indirect dependencies that leads to things not working as expected and thus failing tests.

One thing that seems to be causing a lot of problems is that the early use of Project.toml used a too-powerful compatibility operator:

https://github.com/JuliaInterop/RCall.jl/blob/2eac8c9794eb86d363f65c77e72c4a1f01cb4bce/Project.toml#L20-L29

That ≥ should be ^. That was fixed in #371, so let's try the blame game again with that version....well, that didn't work out nicely because of #486, so I manually bisected the released versions while fixing that test in each version...

julia version	last good RCall version
1.0	0.13.9
1.1	0.13.9
1.2	0.13.9
1.3	0.13.9
1.4	0.14.0
1.5	0.14.0
1.6	0.14.1

There was a lot of output from all these bisections and I don't have time to review it all, but one thing did pop out: the indirect dependency Formatting.jl which specifies compat on the Logging stdlib as "1", but some older versions of Julia think of their stdlib versions as being "0.0.0". So the compatibility requirements are unresolvable. Formatting.jl is deprecated and read-only, so we can't "just" update its stdlib compat entries.

ERROR: Unsatisfiable requirements detected for package Formatting [59287772]:
 Formatting [59287772] log:
 ├─possible versions are: [0.3.3-0.3.5, 0.4.0-0.4.3] or uninstalled
 ├─restricted to versions 0.4.3 by an explicit requirement, leaving only versions 0.4.3
 └─restricted by compatibility requirements with Logging [56ddb016] to versions: [0.3.3-0.3.5, 0.4.0-0.4.2] or uninstalled — no versions left
   └─Logging [56ddb016] log:
     ├─possible versions are: 0.0.0 or uninstalled
     ├─restricted to versions * by Test [8dfed614], leaving only versions 0.0.0
     │ └─Test [8dfed614] log:
     │   ├─possible versions are: 0.0.0 or uninstalled
     │   └─Test [8dfed614] is fixed to version 0.0.0
     └─Logging [56ddb016] is fixed to version 0.0.0

We can turn this table around to give current Julia compat bounds

RCall version	actual julia compat	reason
0.13.10	1.4	transitive dependencies on things with stdlib compat entries
0.14.1	1.6	dependency on Preferences.jl

[!WARNING] These compat bounds are for R 3.4. If you need R 4, then you're safer with RCall 0.13.6

I don't know what we should do for RCall 0.13.10 - 0.13.18. In some sense they are compatible with Julia 1.0 (and were developed on Julia 1.0!), but Formatting's compat entry for Logging breaks things. We can't just specify an entry on Formatting because it's a doubly indirect dependency:

(RCall) pkg> why Formatting
  DataFrames → PrettyTables → Formatting

We could in principle ask the JuliaString folks to temporarily unarchive the repo and do one final patch release where the Logging compat entry is set to "0, 1" to fix this situation. I know there would be some hesitation to do that for a deprecated package for a very old release of Julia, but I also don't think it's great that we have a breaking-in-practice change without a breaking version bump. We could also bump the compat entry in the registry to 1.4, but that seems to ignore history ...

For RCall 0.14.0 and 0.14.1, we should simply bump the compat entry in the registry to 1.6 and call it a day.

cc @ViralBShah

ViralBShah commented 4 days ago

Also note that we are only a few weeks away from 1.10 potentially being the new LTS. Hence I would do the easiest thing and call it a day.

palday commented 4 days ago

I think the easiest path forward is two parts:

update the compat in the registry for 0.14.x to be Julia 1.6.
add a note here (this comment!) that anybody who wants to use RCall 0.13.10-0.13.19 on Julia < 1.6 should manually add and pin Formatting@0.4.2 to their own project.

palday commented 4 days ago

One more crossref: #530 which updated the compat in Project.toml.

JuliaInterop / RCall.jl

Incorrect julia compat #531

529 raised the issue that our Julia compat bounds may not be correct in practice -- although we don't use any language features requiring newer Julia versions, our dependencies may not be resolvable on older Julia version.