Spine Toolbox does not warn about lack of Julia Kernel when running Julia tools like SpineOpt

[tarskul]

I've installed spine tools on a few computers already but on my current setup (Linux mint on a Thinkpad) I'm facing an issue that is difficult to fix because it does not show any errors. The SpineOpt process in my workflow shows a green checkmark but it doesn't actually do anything.

My observations:

• all processes in the workflow show a green checkmark, but SpineOpt does not actually do anything (I'm using the simple system tutorial which I know works)
• In my other setups I installed spine tools globally and in virtual environments (both pipx and conda) whereas in the current setup I only installed spine tools in the virtual environment (conda only).
• @jkiviluo pointed out that I should look at the julia console but there does not seem to be one. In fact, he also told me to create this issue because Spine Toolbox should warn the user that there is an issue with the julia console. Anyway, I'll try to see if that is the fix I need.

For your information, I followed my script below for the installation (I do not actually run it to be able to resolve issues in intermediary steps):

#!/bin/bash
#chmod +x ~/conda_spinedev_install.sh
#sudo apt install libxcb-cursor0
source ~/miniconda3/etc/profile.d/conda.sh
conda create --name spinedev python=3.9 -y
conda activate spinedev
conda install -c conda-forge julia -y

cd ~/miniconda3/envs/spinedev/
mkdir spinetools
cd spinetools
git clone git@github.com:spine-tools/Spine-Toolbox.git
git clone git@github.com:spine-tools/SpineInterface.jl.git
git clone git@github.com:spine-tools/SpineOpt.jl.git

julia -e 'using Pkg; pkg"dev ~/miniconda3/envs/spinedev/spinetools/SpineInterface.jl"; pkg"dev ~/miniconda3/envs/spinedev/spinetools/SpineOpt.jl"; Pkg.add(["PyCall", "Documenter", "HiGHS", "GLPK", "Cbc"])'

cd Spine-Toolbox
python -m pip install --upgrade pip
python -m pip install -r requirements.txt
python -m pip install -r dev-requirements.txt

#Alternative install spine toolbox without git; however conda and pipx do not seem to play nicely together
#python -m pip install --upgrade pip
#python -m pip install --user pipx
#python -m pipx ensurepath
#reset the terminal?
#python -m pipx install spinetoolbox --force

conda env export > ~/OneDrive_KUL/Mopo/Code/batsh/spinedev.yml

#the spinetoolbox command only works after resetting the terminal
spinetoolbox

[jkiviluo]

To clarify: Tars' Spine Toolbox does not see any Julia kernel. In this case, Toolbox should say that you cannot run Julia tools (like SpineOpt) without first setting up Julia kernel (either point to an existing one or install a new one in the settings).

[tarskul]

Yes, thanks. It seems indeed that this was the problem. I installed julia with the 'install julia' button in Spine Toolbox and now I do see a console and the workflow behaves as expected.

As a sidenote, I saw that Spine Toolbox uses jill to install julia. It is a bit weird because I did actually install julia with jill before. Maybe I forgot to add it to the path and that is why SpineToolbox did not see it? It could also be that I'm confusing it with a different computer...

Anyway, I'll continue troubleshooting because now the SpineOpt process shows an error but at least I see an error from which I can continue fixing my installation.

[ptsavol]

I cannot reproduce this. I removed Julia from my PATH and all my Julia kernels so my Tools Settings page looks like this.

These are the default Julia settings for Spine Toolbox when Julia has not been installed yet. Note that users can not select Jupyter Console without also selecting a Julia kernel (the Settings page won't close when clicking OK).

So, with these Settings I tried executing a Julia Tool and this happened.

So it already works like @jkiviluo suggested above. Need some reproducible steps to continue on this.

[tarskul]

hm, I'll add a few remarks:

• for me the basic console looked different; each field filled in with a path but greyed out
• when i solved it for one (mini)conda environment, it was automatically solved for the other conda environment. Does that make sense?
• There is someone else who mentioned that they had the same problem. I'll contact them later to see whether that brings new information on the table.

[ptsavol]

for me the basic console looked different; each field filled in with a path but greyed out

If there's a 'greyd out' path in the red circle, it means that Julia is in your PATH.

when i solved it for one (mini)conda environment, it was automatically solved for the other conda environment. Does that make sense?

Yes. Like Windows, Linux saves the application settings into some global path. All Spine Toolbox installations use the same settings.

There is someone else who mentioned that they had the same problem. I'll contact them later to see whether that brings new information on the table.

I think there's some hicup in installing SpineOpt for the first time (I've seen it too) but it's really difficult to recreate the problem after it finally starts working. But this may be something else of course. Please keep me informed if you get any new information.

[ptsavol]

@tarskul Any new info on this?

[tarskul]

It is so long ago that I can barely remember. I did not get the information from the other person, that is for sure.

As for myself, I eventually got it working and indeed it is hard to recreate the issue once it starts working.

I'm not sure whether it is related but for the longest time I had an issue with multiple versions of Spine Toolbox because I could not configure it correctly. The main issue was with PyCall. But since then I learned what I was doing wrong. I'm trying to document my experience in that regard, but I have to do that in between other stuff so it is slow.

But maybe I can share my improved script:

#!/bin/bash
#chmod +x ~/spinetools/install_spinetools.sh
# uncomment these checks for debugging; check whether python, git and julia are installed; python3 may be needed to changed to python depending on your system
python3 --version
git --version
julia --version

# user settings: install directory, environment names
path_spinetools=$(dirname $0)
path_envs=environments
env_python=penv_eight
# the julia settings need to be adjusted directly in the code (see further below)

# download files from git
cd $path_spinetools
git clone https://github.com:spine-tools/Spine-Toolbox.git
git clone https://github.com:spine-tools/SpineInterface.jl.git
git clone https://github.com:spine-tools/SpineOpt.jl.git

# checkout a specific branch if you want to
cd SpineInterface.jl
git fetch
git checkout -b 0.8-dev origin/0.8-dev
cd ..
cd SpineOpt.jl
git fetch
git checkout -b 0.8-dev origin/0.8-dev
cd ..
cd Spine-Toolbox
git fetch
git checkout -b 0.8-dev origin/0.8-dev
cd ..

# create python environment (for spine toolbox)
mkdir $path_envs
cd $path_envs
python3 -m venv $env_python
source $env_python/bin/activate
cd ..

# alternatively use a conda environment
#path_conda=~/miniconda3/etc/profile.d/conda.sh
#env_conda=cenv_dev
#source $path_conda
#conda create --name $env_conda python=3.9 -y
#conda activate $env_conda

# install python requirements
cd Spine-Toolbox
python3 -m pip install --upgrade pip
python3 -m pip install -r requirements.txt
#python3 -m pip install -r dev-requirements.txt
cd ..

# create julia environment (for SpineOpt and SpineInterface)
# add packages
# configure PyCall (not always necessary because it should be built with the active python environment)
julia -e '
env_julia = joinpath(@__DIR__,"environments/jenv_eight")
path_python = joinpath(@__DIR__,"environments/penv_eight/bin/python3")
path_spineinterface = joinpath(@__DIR__,"SpineInterface.jl")
path_spineopt = joinpath(@__DIR__,"SpineOpt.jl")
import Pkg
Pkg.activate(env_julia)
Pkg.develop(path=path_spineinterface)
Pkg.develop(path=path_spineopt)
Pkg.add(["PyCall", "Documenter", "HiGHS"])
import PyCall
ENV["PYTHON"] = path_python
Pkg.build("PyCall")
println(PyCall.pyprogramname)
'

# manually add julia environment to settings in spine toolbox
#spinetoolbox

# alternatively keep shell open for debugging
$SHELL