search

invenia / Impute.jl

Imputation methods for missing data in julia
https://invenia.github.io/Impute.jl/latest/
Other
76 stars 11 forks source link

CompatHelper failed to run for a long time #96

Closed norci closed 3 years ago

norci commented 3 years ago

log: https://github.com/invenia/Impute.jl/runs/1540506417?check_suite_focus=true

ERROR: expected package `Impute [f7bf1975]` to exist at path `/Users/rory/repos/invenia/Impute.jl`

The problem is, the package versions in https://github.com/invenia/Impute.jl/Project.toml#L22 are too old.

so the other packages, such as Flux, CUDA, are restricted to an old version too, which contain bugs (have been fixed in new versions).

I don't know how to fix this CI error. what about remove all the entries in the [compat] section? https://github.com/norci/Impute.jl/commit/34b86d6ef8960ed0dfdf35257381605cfa62e07d

rofinn commented 3 years ago

Yeah, looks like it's failing cause I accidentally updated the docs manifest to not use a relative path for the Impute version. I don't know why that didn't error when the change was introduced... :/ Should be an easy fix though. I don't think removing the [compat] section is the correct answer. Can you provide a list of packages that are holding back the Flux / CUDA versions?

norci commented 3 years ago

I'm not sure which one restrict the versions of Flux. this is the reproduce steps:

(@v1.5) pkg> add Flux,CUDA
  Resolving package versions...
Updating `C:\Users\zhexu\.julia\environments\v1.5\Project.toml`
  [052768ef] + CUDA v2.3.0
  [587475ba] + Flux v0.11.2

(@v1.5) pkg> add Impute
  Resolving package versions...
Updating `C:\Users\zhexu\.julia\environments\v1.5\Project.toml`
  [052768ef] ↓ CUDA v2.3.0 ⇒ v1.3.3
  [587475ba] ↓ Flux v0.11.2 ⇒ v0.11.1
  [f7bf1975] + Impute v0.6.0
Updating `C:\Users\zhexu\.julia\environments\v1.5\Manifest.toml`
  [4fba245c] - ArrayInterface v2.14.4
  [ab4f0b2a] - BFloat16s v0.1.0
  [fbb218c0] + BSON v0.2.6
  [336ed68f] + CSV v0.7.10
  [052768ef] ↓ CUDA v2.3.0 ⇒ v1.3.3
  [324d7699] + CategoricalArrays v0.8.3
  [adafc99b] + CpuId v0.2.2
  [124859b0] + DataDeps v0.7.4
  [a93c6f00] + DataFrames v0.22.1
  [864edb3b] ↓ DataStructures v0.18.8 ⇒ v0.17.20
  [b4f34e82] + Distances v0.9.2
  [587475ba] ↓ Flux v0.11.2 ⇒ v0.11.1
  [59287772] + Formatting v0.4.2
  [0c68f7d7] ↓ GPUArrays v6.1.2 ⇒ v5.2.1
  [61eb1bfa] ↓ GPUCompiler v0.8.3 ⇒ v0.6.1
  [cd3eb016] + HTTP v0.8.19
  [0e44f5e4] - Hwloc v1.0.3
  [615f187c] - IfElse v0.1.0
  [f7bf1975] + Impute v0.6.0
  [83e8ac13] + IniFile v0.5.0
  [41ab1584] + InvertedIndices v1.0.0
  [c8e1da08] + IterTools v1.3.0
  [929cbde3] ↓ LLVM v3.4.0 ⇒ v2.0.0
  [bdcacae8] ↓ LoopVectorization v0.9.3 ⇒ v0.8.26
  [739be429] + MbedTLS v1.0.3
  [356022a1] + NamedDims v0.2.29
  [b8a86587] + NearestNeighbors v0.4.8
  [2dfb63ee] + PooledArrays v0.5.3
  [08abe8d2] + PrettyTables v0.10.1
  [21efa798] + SIMDPirates v0.8.26
  [476501e8] ↓ SLEEFPirates v0.6.0 ⇒ v0.5.5
  [6c6a2e73] - Scratch v1.0.3
  [91c51154] + SentinelArrays v1.2.16
  [2913bbd2] ↓ StatsBase v0.33.2 ⇒ v0.32.2
  [856f2bd8] + StructTypes v1.1.0
  [ab02a1b2] + TableOperations v0.2.1
  [3d5dd08c] ↓ VectorizationBase v0.13.8 ⇒ v0.12.33
  [9fa8497b] + Future

(@v1.5) pkg> add CUDA@2
  Resolving package versions...
ERROR: Unsatisfiable requirements detected for package DataFrames [a93c6f00]:
 DataFrames [a93c6f00] log:
 ├─possible versions are: [0.11.7, 0.12.0, 0.13.0-0.13.1, 0.14.0-0.14.1, 0.15.0-0.15.2, 0.16.0, 0.17.0-0.17.1, 0.18.0-0.18.4, 0.19.0-0.19.4, 0.20.0-0.20.2, 0.21.0-0.21.8, 0.22.0-0.22.1] or uninstalled
 ├─restricted by compatibility requirements with StatsBase [2913bbd2] to versions: [0.19.1-0.19.4, 0.20.0-0.20.2, 0.21.0-0.21.8, 0.22.0-0.22.1] or uninstalled
 │ └─StatsBase [2913bbd2] log:
 │   ├─possible versions are: [0.24.0, 0.25.0, 0.26.0, 0.27.0, 0.28.0-0.28.1, 0.29.0, 0.30.0, 0.31.0, 0.32.0-0.32.2, 0.33.0-0.33.2] or uninstalled
 │   └─restricted by compatibility requirements with Flux [587475ba] to versions: 0.33.0-0.33.2
 │     └─Flux [587475ba] log:
 │       ├─possible versions are: [0.4.1, 0.5.0-0.5.4, 0.6.0-0.6.10, 0.7.0-0.7.3, 0.8.0-0.8.3, 0.9.0, 0.10.0-0.10.4, 0.11.0-0.11.2] or uninstalled
 │       ├─restricted to versions * by an explicit requirement, leaving only versions [0.4.1, 0.5.0-0.5.4, 0.6.0-0.6.10, 0.7.0-0.7.3, 0.8.0-0.8.3, 0.9.0, 0.10.0-0.10.4, 0.11.0-0.11.2]
 │       ├─restricted by compatibility requirements with CUDA [052768ef] to versions: [0.4.1, 0.5.0-0.5.4, 0.6.0-0.6.10, 0.7.0-0.7.3, 0.8.0-0.8.3, 0.9.0, 0.10.0-0.10.4, 0.11.2] or uninstalled, leaving only versions: [0.4.1, 0.5.0-0.5.4, 0.6.0-0.6.10, 0.7.0-0.7.3, 0.8.0-0.8.3, 0.9.0, 0.10.0-0.10.4, 0.11.2]
 │       │ └─CUDA [052768ef] log:
 │       │   ├─possible versions are: [0.1.0, 1.0.0-1.0.2, 1.1.0, 1.2.0-1.2.1, 1.3.0-1.3.3, 2.0.0-2.0.2, 2.1.0, 2.2.0-2.2.1, 2.3.0] or uninstalled
 │       │   ├─restricted to versions 2 by an explicit requirement, leaving only versions [2.0.0-2.0.2, 2.1.0, 2.2.0-2.2.1, 2.3.0]
 │       │   └─restricted by compatibility requirements with Flux [587475ba] to versions: [2.1.0, 2.2.0-2.2.1, 2.3.0]
 │       │     └─Flux [587475ba] log: see above
 │       ├─restricted by compatibility requirements with NNlib [872c559c] to versions: [0.8.3, 0.9.0, 0.10.0-0.10.4, 0.11.0-0.11.2] or uninstalled, leaving only versions: [0.8.3, 0.9.0, 0.10.0-0.10.4, 0.11.2]
 │       │ └─NNlib [872c559c] log:
 │       │   ├─possible versions are: [0.3.2, 0.4.0-0.4.3, 0.5.0, 0.6.0-0.6.6, 0.7.0-0.7.7] or uninstalled
 │       │   ├─restricted by compatibility requirements with Flux [587475ba] to versions: [0.3.2, 0.4.0-0.4.3, 0.5.0, 0.6.0-0.6.6, 0.7.0-0.7.7]
 │       │   │ └─Flux [587475ba] log: see above
 │       │   ├─restricted by compatibility requirements with CUDA [052768ef] to versions: [0.6.5-0.6.6, 0.7.0-0.7.7]
 │       │   │ └─CUDA [052768ef] log: see above
 │       │   └─restricted by compatibility requirements with Flux [587475ba] to versions: 0.7.0-0.7.7
 │       │     └─Flux [587475ba] log: see above
 │       └─restricted by compatibility requirements with Adapt [79e6a3ab] to versions: 0.11.0-0.11.2 or uninstalled, leaving only versions: 0.11.2
 │         └─Adapt [79e6a3ab] log:
 │           ├─possible versions are: [0.3.0-0.3.1, 0.4.0-0.4.2, 1.0.0-1.0.1, 1.1.0, 2.0.0-2.0.2, 2.1.0, 2.2.0, 2.3.0] or uninstalled
 │           ├─restricted by compatibility requirements with Flux [587475ba] to versions: [0.3.0-0.3.1, 0.4.0-0.4.2, 1.0.0-1.0.1, 1.1.0, 2.0.0-2.0.2, 2.1.0, 2.2.0, 2.3.0]
 │           │ └─Flux [587475ba] log: see above
 │           └─restricted by compatibility requirements with CUDA [052768ef] to versions: [2.2.0, 2.3.0]
 │             └─CUDA [052768ef] log: see above
 └─restricted by compatibility requirements with Impute [f7bf1975] to versions: [0.15.0-0.15.2, 0.16.0, 0.17.0-0.17.1, 0.18.0-0.18.4] — no versions left
   └─Impute [f7bf1975] log:
     ├─possible versions are: [0.1.0-0.1.1, 0.2.0, 0.3.0, 0.4.0, 0.5.0-0.5.1, 0.6.0] or uninstalled
     ├─restricted to versions * by an explicit requirement, leaving only versions [0.1.0-0.1.1, 0.2.0, 0.3.0, 0.4.0, 0.5.0-0.5.1, 0.6.0]
     └─restricted by compatibility requirements with StatsBase [2913bbd2] to versions: [0.1.0-0.1.1, 0.2.0] or uninstalled, leaving only versions: [0.1.0-0.1.1, 0.2.0]
       └─StatsBase [2913bbd2] log: see above
rofinn commented 3 years ago

Looks like the latest version of Flux only supports StatsBase 0.33. Looking at the diff I don't think the breaking changes should impact Impute.jl, that should be another simple update.

With the extra commit on #97 it seems to work.

(Impute) pkg> add CUDA@2
   Updating registry at `~/.julia/registries/General`
   Updating git-repo `git@github.com:JuliaRegistries/General.git`
   Updating registry at `~/.julia/registries/Invenia`
   Updating git-repo `git@gitlab.invenia.ca:invenia/PackageRegistry.git`
  Resolving package versions...
  Installed BFloat16s ──── v0.1.0
  Installed TimerOutputs ─ v0.5.7
  Installed GPUCompiler ── v0.8.3
  Installed GPUArrays ──── v6.1.2
  Installed LLVM ───────── v3.5.1
  Installed CUDA ───────── v2.3.0
Updating `~/repos/invenia/Impute.jl/Project.toml`
  [052768ef] + CUDA v2.3.0
Updating `~/repos/invenia/Impute.jl/Manifest.toml`
  [621f4979] + AbstractFFTs v0.5.0
  [79e6a3ab] + Adapt v2.3.0
  [ab4f0b2a] + BFloat16s v0.1.0
  [fa961155] + CEnum v0.4.1
  [052768ef] + CUDA v2.3.0
  [ffbed154] - DocStringExtensions v0.8.3
  [e30172f5] - Documenter v0.26.0
  [e2ba6199] + ExprTools v0.1.3
  [0c68f7d7] + GPUArrays v6.1.2
  [61eb1bfa] + GPUCompiler v0.8.3
  [b5f81e59] - IOCapture v0.1.1
  [929cbde3] + LLVM v3.5.1
  [1914dd2f] + MacroTools v0.5.6
  [872c559c] + NNlib v0.7.7
  [6c6a2e73] + Scratch v1.0.3
  [a759f4b9] + TimerOutputs v0.5.7