search

lanl-ansi / MathOptAI.jl

Embed trained machine learning predictors in JuMP
https://lanl-ansi.github.io/MathOptAI.jl/
Other
29 stars 1 forks source link

Doctests fail if CondaPkg is not installed #75

Closed Robbybp closed 2 months ago

Robbybp commented 2 months ago

When trying to run the tests with julia --project=test test/runtests.jl, I get a long error message at the Doctest step:

[ Info: Doctest: running doctests.                                                                                                                                                                                               12:44:06 [384/9299]
┌ Error: doctest failure in ~/research/upstream-dev/MathOptAI.jl/src/predictors/PytorchModel.jl:12-16                     
│                                                                                                                         
│ ```jldoctest                                                                                                            
│ julia> using PythonCall, MathOptAI                                                                                                                                                                                                                
│                                                                                                                         
│ julia> ml_model = PytorchModel("model.pt");                                                                             
│ ```                                                                                                                     
│                                                                                                                         
│ Subexpression:                                                                                                          
│                                                                                                                                                                                                                                                   
│ using PythonCall, MathOptAI                                                                                             
│                                                                                                                                                                                                                                                   
│ Evaluated output:                                                                                                       
│                                                                                                                         
│     CondaPkg Found dependencies: /Users/rbparker/.julia/packages/PythonCall/flx5V/CondaPkg.toml                         
│     CondaPkg Resolving changes                                                                                          
│              ~ python                                                                                                   
│              - torch (pip)                                                                                              
│     CondaPkg Removing Pip packages                                                                                      
│              │ /Users/rbparker/research/upstream-dev/MathOptAI.jl/test/.CondaPkg/env/bin/pip                            
│              │ uninstall                                                                                                
│              │ -y                                                                                                                                                                                                                                 
│              └ torch                                                                                                                                                                                                                              
│ Found existing installation: torch 2.4.0                                                                                
│ Uninstalling torch-2.4.0:                                                                                               
│   Successfully uninstalled torch-2.4.0                                                                                  
│     CondaPkg Installing packages                                                                                        
│              │ /Users/rbparker/.julia/artifacts/98aba21a6802e06a5d2e7c48633d7dc1c3d4a749/bin/micromamba                 
│              │ -r /Users/rbparker/.julia/scratchspaces/0b3b1443-0f03-428d-bdfb-f27f9c1191ea/root                        
│              │ install                                                                                                  
│              │ -y                                                                                                       
│              │ -p /Users/rbparker/research/upstream-dev/MathOptAI.jl/test/.CondaPkg/env                                                                                                                                                           
│              │ --override-channels                         
│              │ --no-channel-priority                                                                                    
│              │ openssl[version='>=3, <3.1']                                                                             
│              │ python[version='>=3.8,<4',channel='conda-forge',build='*cpython*']                                       
│              └ -c conda-forge                                                                                           
│ conda-forge/osx-arm64                                       Using cache                                                 
│ conda-forge/noarch                                          Using cache                                                 
│                                                                                                                         
│ Transaction                                                                                                             
│                                                                                                                         
│   Prefix: /Users/rbparker/research/upstream-dev/MathOptAI.jl/test/.CondaPkg/env                                         
│                                                                                                                                                                                                                                                   
│   Updating specs:                                                                                                       
│                                                                                                                         
│    - openssl[version='>=3, <3.1']                                                                                       
│    - conda-forge::python[version='>=3.8,<4',build=*cpython*]                                                            
│                                                                                                                         
│                                                                                                                                                                                                                                                   
│   Package   Version  Build               Channel           Size                                                         
│ ─────────────────────────────────────────────────────────────────                                                                                                                                                                                 
│   Reinstall:                                                                                                            
│ ─────────────────────────────────────────────────────────────────                                                       
│                                                                                                                         
│   o python   3.11.0  h3ba56d0_1_cpython  conda-forge     Cached                                                         
│                                                                                                                         
│   Summary:                                                                                                              
│                                                                                                                         
│   Reinstall: 1 packages                                                                                                 
│                                                                                                                         
│   Total download: 0 B                                                                                                                                                                                                                             
│                                                                                                                                                                                                                                                   
│ ─────────────────────────────────────────────────────────────────                                                       
│                                                            
│                                                            
│                                                            
│ Transaction starting                                       
│ Reinstalling python-3.11.0-h3ba56d0_1_cpython              
│                                                            
│ Transaction finished                                       
│                                                            
│ To activate this environment, use:                         
│                                                            
│     micromamba activate /Users/rbparker/research/upstream-dev/MathOptAI.jl/test/.CondaPkg/env                           
│                                                            
│ Or to execute a single command in this environment, use:   
│                                                            
│     micromamba run -p /Users/rbparker/research/upstream-dev/MathOptAI.jl/test/.CondaPkg/env mycommand                   
│                                                            
│ Expected output:                                           
│                                                            
│                                                            
│                                                            
│   diff =                                                   
│        CondaPkg Found dependencies: /Users/rbparker/.julia/packages/PythonCall/flx5V/CondaPkg.toml                      
│        CondaPkg Resolving changes                          
│                 ~ python                                   
│                 - torch (pip)                              
│        CondaPkg Removing Pip packages                      
│                 │ /Users/rbparker/research/upstream-dev/MathOptAI.jl/test/.CondaPkg/env/bin/pip                         
│                 │ uninstall                                
│                 │ -y                                       
│                 └ torch                                    
│    Found existing installation: torch 2.4.0                
│    Uninstalling torch-2.4.0:                               
│      Successfully uninstalled torch-2.4.0                  
│        CondaPkg Installing packages                        
│                 │ /Users/rbparker/.julia/artifacts/98aba21a6802e06a5d2e7c48633d7dc1c3d4a749/bin/micromamba              
│                 │ -r /Users/rbparker/.julia/scratchspaces/0b3b1443-0f03-428d-bdfb-f27f9c1191ea/root                     
│                 │ install                                  
│                 │ -y                                       
│                 │ -p /Users/rbparker/research/upstream-dev/MathOptAI.jl/test/.CondaPkg/env                              
│                 │ --override-channels                      
│                 │ --no-channel-priority                    
│                 │ openssl[version='>=3, <3.1']             
│                 │ python[version='>=3.8,<4',channel='conda-forge',build='*cpython*']                                    
│                 └ -c conda-forge                           
│    conda-forge/osx-arm64                                       Using cache                                              
│    conda-forge/noarch                                          Using cache                                              
│                                                            
│    Transaction                                             
│                                                            
│      Prefix: /Users/rbparker/research/upstream-dev/MathOptAI.jl/test/.CondaPkg/env                                      
│                                                            
│      Updating specs:                                       
│                                                            
│       - openssl[version='>=3, <3.1']                       
│       - conda-forge::python[version='>=3.8,<4',build=*cpython*]                                                         
│                                                            
│                                                            
│      Package   Version  Build               Channel           Size                                                      
│    ─────────────────────────────────────────────────────────────────                                                    
│      Reinstall:                                            
│    ─────────────────────────────────────────────────────────────────                                                    
│                                                            
│      o python   3.11.0  h3ba56d0_1_cpython  conda-forge     Cached                                                      
│                                                            
│      Summary:                                              
│                                                            
│      Reinstall: 1 packages                                 
│                                                            
│      Total download: 0 B                                   
│                                                            
│    ─────────────────────────────────────────────────────────────────                                                    
│                                                            
│                                                            
│                                                            
│    Transaction starting                                    
│    Reinstalling python-3.11.0-h3ba56d0_1_cpython           
│                                                            
│    Transaction finished                                    
│                                                                                                                                                                                                                                                   
│    To activate this environment, use:                                                                                                                                                                                                             
│                                                            
│        micromamba activate /Users/rbparker/research/upstream-dev/MathOptAI.jl/test/.CondaPkg/env                        
│                                                                                                                         
│    Or to execute a single command in this environment, use:                                                             
│                                                                                                                         
│        micromamba run -p /Users/rbparker/research/upstream-dev/MathOptAI.jl/test/.CondaPkg/env mycommand                
└ @ Documenter ~/research/upstream-dev/MathOptAI.jl/src/predictors/PytorchModel.jl:12                                     
┌ Error: Doctesting failed                                   
│   exception =                                              
│    `makedocs` encountered a doctest error. Terminating build                                                            
│    Stacktrace:                                             
│      [1] error(s::String)                                  
│        @ Base ./error.jl:35                                
│      [2] runner(::Type{Documenter.Builder.Doctest}, doc::Documenter.Document)                                           
│        @ Documenter ~/.julia/packages/Documenter/qoyeC/src/builder_pipeline.jl:212                                      
│      [3] dispatch(::Type{Documenter.Builder.DocumentPipeline}, x::Documenter.Document)                                  
│        @ Documenter.Selectors ~/.julia/packages/Documenter/qoyeC/src/utilities/Selectors.jl:170                         
│      [4] #86                                                                                                            
│        @ ~/.julia/packages/Documenter/qoyeC/src/makedocs.jl:248 [inlined]                                               
│      [5] withenv(::Documenter.var"#86#88"{Documenter.Document}, ::Pair{String, Nothing}, ::Vararg{Pair{String, Nothing}})
│        @ Base ./env.jl:257                                                                                              
│      [6] #85                                               
│        @ ~/.julia/packages/Documenter/qoyeC/src/makedocs.jl:247 [inlined]                                               
│      [7] cd(f::Documenter.var"#85#87"{Documenter.Document}, dir::String)                                                
│        @ Base.Filesystem ./file.jl:112                     
│      [8] makedocs(; debug::Bool, format::Documenter.HTMLWriter.HTML, kwargs::@Kwargs{root::String, source::String, sitename::String, doctest::Symbol, modules::Vector{Module}, doctestfilters::Vector{Regex}, remotes::Nothing, plugins::Vector{Do
cumenter.Plugin}})                                           
│        @ Documenter ~/.julia/packages/Documenter/qoyeC/src/makedocs.jl:247                                              
│      [9] (::Documenter.var"#all_doctests#131"{Bool, Vector{Regex}, Vector{Documenter.Plugin}, Vector{Module}})()        
│        @ Documenter ~/.julia/packages/Documenter/qoyeC/src/doctest.jl:82                                                
│     [10] macro expansion                                                                                                
│        @ ~/.julia/packages/Documenter/qoyeC/src/doctest.jl:107 [inlined]                                                
│     [11] macro expansion                                   
│        @ ~/.julia/juliaup/julia-1.10.2+0.aarch64.apple.darwin14/share/julia/stdlib/v1.10/Test/src/Test.jl:669 [inlined] 
│     [12] macro expansion                                   
│        @ ~/.julia/packages/Documenter/qoyeC/src/doctest.jl:107 [inlined]                                                
│     [13] macro expansion                                   
│        @ ~/.julia/juliaup/julia-1.10.2+0.aarch64.apple.darwin14/share/julia/stdlib/v1.10/Test/src/Test.jl:1577 [inlined] 
│     [14] doctest(source::Nothing, modules::Vector{Module}; fix::Bool, testset::String, doctestfilters::Vector{Regex}, plugins::Vector{Documenter.Plugin})
│        @ Documenter ~/.julia/packages/Documenter/qoyeC/src/doctest.jl:107                                               
│     [15] doctest                                           
│        @ ~/.julia/packages/Documenter/qoyeC/src/doctest.jl:66 [inlined]                                                 
│     [16] doctest(package::Module; manual::Bool, testset::Nothing, kwargs::@Kwargs{})                                    
│        @ Documenter ~/.julia/packages/Documenter/qoyeC/src/doctest.jl:34                                                
│     [17] macro expansion                                   
│        @ ~/research/upstream-dev/MathOptAI.jl/test/runtests.jl:19 [inlined]                                             
│     [18] macro expansion                                   
│        @ ~/.julia/juliaup/julia-1.10.2+0.aarch64.apple.darwin14/share/julia/stdlib/v1.10/Test/src/Test.jl:1577 [inlined] 
│     [19] top-level scope                                   
│        @ ~/research/upstream-dev/MathOptAI.jl/test/runtests.jl:19                                                       
│     [20] include(mod::Module, _path::String)               
│        @ Base ./Base.jl:495                                
│     [21] exec_options(opts::Base.JLOptions)                
│        @ Base ./client.jl:318                              
│     [22] _start()                                          
│        @ Base ./client.jl:552                              
└ @ Documenter ~/.julia/packages/Documenter/qoyeC/src/doctest.jl:96                                                       
Doctests: MathOptAI: Test Failed at /Users/rbparker/.julia/packages/Documenter/qoyeC/src/doctest.jl:107                   
  Expression: all_doctests()                                 

Stacktrace:                                                  
 [1] macro expansion                                         
   @ ~/.julia/juliaup/julia-1.10.2+0.aarch64.apple.darwin14/share/julia/stdlib/v1.10/Test/src/Test.jl:672 [inlined]       
 [2] macro expansion                                         
   @ ~/.julia/packages/Documenter/qoyeC/src/doctest.jl:107 [inlined]                                                      
 [3] macro expansion                                         
   @ ~/.julia/juliaup/julia-1.10.2+0.aarch64.apple.darwin14/share/julia/stdlib/v1.10/Test/src/Test.jl:1577 [inlined]      
 [4] doctest(source::Nothing, modules::Vector{Module}; fix::Bool, testset::String, doctestfilters::Vector{Regex}, plugins::Vector{Documenter.Plugin})
   @ Documenter ~/.julia/packages/Documenter/qoyeC/src/doctest.jl:107                                                     
Test Summary:         | Fail  Total   Time                   
Docstrings            |    1      1  14.1s                   
  Doctests: MathOptAI |    1      1  13.9s                   
ERROR: LoadError: Some tests did not pass: 0 passed, 1 failed, 0 errored, 0 broken.

This goes away if I install CondaPkg in the test environment. My test/Project.toml is then:

diff --git a/test/Project.toml b/test/Project.toml
index d292d27..d851cbd 100644
--- a/test/Project.toml
+++ b/test/Project.toml
@@ -1,6 +1,7 @@
 [deps]
 AbstractGPs = "99985d1d-32ba-4be9-9821-2ec096f28918"
 CSV = "336ed68f-0bac-5ca0-87d4-7b16caf5d00b"
+CondaPkg = "992eb4ea-22a4-4c89-a5bb-47a3300528ab"
 DataFrames = "a93c6f00-e57d-5684-b7b6-d8193f3e46c0"
 DecisionTree = "7806a523-6efd-50cb-b5f6-3fa6f1930dbb"
 Distributions = "31c24e10-a181-5473-b8eb-7969acd0382f"

However, the offending line (using PythonCall, MathOptAI, I think) works fine in julia --project=test without CondaPkg explicitly installed. Not sure what's going on here. Feel free to close if there's nothing to be done, but thought I'd open in case this happens to someone else.

odow commented 2 months ago

Hmm. Not sure. It all works for me locally. But this will be much easier to diagnose once the repo is public and we can setup CI etc.