search

edwinb-ai / LeastSquaresSVM

A Least Squares Support Vector Machine implementation in pure Julia
https://edwinb-ai.github.io/LeastSquaresSVM/dev/
MIT License
1 stars 1 forks source link

Test-only dependencies #34

Closed edwinb-ai closed 3 years ago

edwinb-ai commented 3 years ago

Thanks to @gmagannaDevelop I have come to realize that we are using the old style for declaring test-only dependencies. These dependencies are only called when the test suite is executed. But in reality, we want to use the new style. In this new style, we need to create an environment specifically for the test directory. This should look something like

(test) pkg> activate .
(test) pkg> add "Some dependencies"

and avoid dealing with stuff like https://github.com/edwinb-ai/Elysivm/blob/fb734747070ddcc7c29109d32f1607a8e4acc188/Project.toml#L20-L27 which might make it non-reproducible.

edwinb-ai commented 3 years ago

Using the following Singularity container

https://gist.github.com/edwinb-ai/a8b0604fe2db13efbdeb746693e93e94

I can successfully reproduce and run all tests. The container is using Ubuntu 16.04, base Julia 1.5.3, and very light dependencies. More input on the matter is necessary @gmagannaDevelop .

gmagannaDevelop commented 3 years ago

Hello @edwinb-ai ! I found out that manually installing the dependencies resolved the issue. I did not note specifically which ones caused the instantiation failure. I'll retry it on a fresh clone of the repo and post the results.

I'm running Julia version 1.5.1 on Pop!_OS 20.10 x86_64. Here's more info on my installation :

$ uname -a
Linux reykjavik 5.8.0-7630-generic #32~1609193707~20.10~781bb80-Ubuntu SMP Tue Jan 5 21:29:56 UTC 2 x86_64 x86_64 x86_64 GNU/Linux
edwinb-ai commented 3 years ago

Hello @edwinb-ai ! I found out that manually installing the dependencies resolved the issue. I did not note specifically which ones caused the instantiation failure.

The thing is that dependencies should actually install correctly.

I'll retry it on a fresh clone of the repo and post the results.

Before doing so, try the following. Go to ~/.julia/packages and completely remove this package rm -rf Elysivm If you create another directory with the same environment, it will just pull all of those dependencies, and there will be no problem whatsoever.

edwinb-ai commented 3 years ago

By @gmagannaDevelop Running the tests yields the following outputs ::

$ julia runtests.jl                                                                                          0 (0.002s) < 18:46:08
 Activating environment at `~/Proyects/Collab/Edwin/Elysivm/Project.toml`
Elysivm.jl: Error During Test at /home/gml/Proyects/Collab/Edwin/Elysivm/test/runtests.jl:5
  Got exception outside of a @test
  LoadError: ArgumentError: Package DataFrames not found in current path:
  - Run `import Pkg; Pkg.add("DataFrames")` to install the DataFrames package.

  Stacktrace:
   [1] require(::Module, ::Symbol) at ./loading.jl:893
   [2] include(::String) at ./client.jl:457
   [3] top-level scope at /home/gml/Proyects/Collab/Edwin/Elysivm/test/runtests.jl:8
   [4] top-level scope at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.5/Test/src/Test.jl:1115
   [5] top-level scope at /home/gml/Proyects/Collab/Edwin/Elysivm/test/runtests.jl:6
   [6] include(::Function, ::Module, ::String) at ./Base.jl:380
   [7] include(::Module, ::String) at ./Base.jl:368
   [8] exec_options(::Base.JLOptions) at ./client.jl:296
   [9] _start() at ./client.jl:506
  in expression starting at /home/gml/Proyects/Collab/Edwin/Elysivm/test/integrationtests.jl:1

Test Summary: | Pass  Error  Total
Elysivm.jl    |   14      1     15
  Types       |    8             8
  Classifier  |    3             3
  Regression  |    1             1
  Kernels     |    2             2
ERROR: LoadError: Some tests did not pass: 14 passed, 0 failed, 1 errored, 0 broken.
in expression starting at /home/gml/Proyects/Collab/Edwin/Elysivm/test/runtests.jl:5
edwinb-ai commented 3 years ago

@gmagannaDevelop The problem is that you shouldn't execute tests like that.

Run the following, please julia --project=. test/runtests.jl while having previously instantiated the environment.

The problem you're running into is that Julia is not recognizing the dependencies within the package, i.e. those defined in the Project.toml file.

gmagannaDevelop commented 3 years ago

I removed ~/.julia/packages/Elysivm. Proceeded to open Julia both on a local clone of my fork and on a fresh clone of this repo.

Launching julia correctly recognises and activates the environment on both directories. Entering the dependencies seem to be correctly installed because I get the following output from pkg:

$ julia                                    1 (23.087s) < 19:25:39
 Activating environment at `~/Proyects/Collab/Edwin/Elysivm/Project.toml`

(Elysivm) pkg> instantiate

(Elysivm) pkg> resolve
No Changes to `~/Proyects/Collab/Edwin/Elysivm/Project.toml`
No Changes to `~/Proyects/Collab/Edwin/Elysivm/Manifest.toml`
gmagannaDevelop commented 3 years ago

Problems persist whilst executing tests:

julia --project=. test/runtests.jl     0 (04:00.101) < 19:33:10
 Activating environment at `~/Proyects/Collab/Edwin/Elysivm/Project.toml`
ERROR: LoadError: ArgumentError: Package MLJModels not found in current path:
- Run `import Pkg; Pkg.add("MLJModels")` to install the MLJModels package.

Stacktrace:
 [1] require(::Module, ::Symbol) at ./loading.jl:893
 [2] include(::Function, ::Module, ::String) at ./Base.jl:380
 [3] include(::Module, ::String) at ./Base.jl:368
 [4] exec_options(::Base.JLOptions) at ./client.jl:296
 [5] _start() at ./client.jl:506
in expression starting at /home/gml/Proyects/Collab/Edwin/Elysivm/test/runtests.jl:3

I do not understand why this happens because the dependencies should have been previously met, according to my previous comment on the instantiation of the environment.

Maybe I'm missing something.

edwinb-ai commented 3 years ago

@gmagannaDevelop Can you show the output of the st command within pkg please?

edwinb-ai commented 3 years ago

I removed ~/.julia/packages/Elysivm. Proceeded to open Julia both on a local clone of my fork and on a fresh clone of this repo.

Also, @gmagannaDevelop can you check if there is a similar package in ~/.julia/dev ?

gmagannaDevelop commented 3 years ago

The output of st is identical for both this repo and my fork. It is the following:

(Elysivm) pkg> st
Project Elysivm v0.7.3
Status `~/Proyects/Collab/Edwin/Elysivm/Project.toml`
  [b4f34e82] Distances v0.10.0
  [e30172f5] Documenter v0.26.1
  [ec8451be] KernelFunctions v0.8.17
  [ba0b0d4f] Krylov v0.6.0
  [98b081ad] Literate v2.8.0
  [add582a8] MLJ v0.15.1
  [a7f614a8] MLJBase v0.16.3
  [e80e1ace] MLJModelInterface v0.3.8
  [37e2e46d] LinearAlgebra
gmagannaDevelop commented 3 years ago

There is no dev directory on my Julia installation ~/.julia.

$ ls                                        0 (0.002s) < 19:44:55
artifacts/  conda/         logs/             prefs/
clones/     config/        packages/         registries/
compiled/   environments/  pluto_notebooks/
edwinb-ai commented 3 years ago

@gmagannaDevelop One last try, if we may. First, remove again the directory under ~/.julia/packages. Next, clone the repo on a local directory, this time, enter the REPL directly with julia, go to pkg and run

(@ v1.5) pkg > activate .
(Elysivm) pkg > instantiate
(Elysivm) pkg > test
edwinb-ai commented 3 years ago

The output of st is identical for both this repo and my fork. It is the following:

(Elysivm) pkg> st
Project Elysivm v0.7.3
Status `~/Proyects/Collab/Edwin/Elysivm/Project.toml`
  [b4f34e82] Distances v0.10.0
  [e30172f5] Documenter v0.26.1
  [ec8451be] KernelFunctions v0.8.17
  [ba0b0d4f] Krylov v0.6.0
  [98b081ad] Literate v2.8.0
  [add582a8] MLJ v0.15.1
  [a7f614a8] MLJBase v0.16.3
  [e80e1ace] MLJModelInterface v0.3.8
  [37e2e46d] LinearAlgebra

The thing is that this is totally expected. The dependencies to run the package only have been installed correctly. But here

Problems persist whilst executing tests:

julia --project=. test/runtests.jl     0 (04:00.101) < 19:33:10
 Activating environment at `~/Proyects/Collab/Edwin/Elysivm/Project.toml`
ERROR: LoadError: ArgumentError: Package MLJModels not found in current path:
- Run `import Pkg; Pkg.add("MLJModels")` to install the MLJModels package.

Stacktrace:
 [1] require(::Module, ::Symbol) at ./loading.jl:893
 [2] include(::Function, ::Module, ::String) at ./Base.jl:380
 [3] include(::Module, ::String) at ./Base.jl:368
 [4] exec_options(::Base.JLOptions) at ./client.jl:296
 [5] _start() at ./client.jl:506
in expression starting at /home/gml/Proyects/Collab/Edwin/Elysivm/test/runtests.jl:3

I do not understand why this happens because the dependencies should have been previously met, according to my previous comment on the instantiation of the environment.

Maybe I'm missing something.

the tests dependencies are the ones that are missing. Maybe we are asking to run the tests using only the package dependencies, which will obviously fail, and that is totally expected as well.

edwinb-ai commented 3 years ago

In this new container https://gist.github.com/edwinb-ai/1ed6ae02acb477c8742c351e14d4a32e the package can be instantiated correctly and tested successfully. It is running Ubuntu 20.04. Something is very weird here...

gmagannaDevelop commented 3 years ago

Executing the tests from the REPL, within pkg yielded successful results : image

The problem was surely the naïve approach I was using to install and test the package. I think we can close this issue now.