Can't find my saved machine artifact

[ablaom]

This used to work for me but doesn't any longer. What's strange is that tests successfully pass locally for me, and I believe saving artifacts is in the tests.

Following the instructions in the README.md:

using MLJ, MLFlowClient

X, y = make_moons()
model = ConstantClassifier()
mach = machine(model, X, y) |> fit!

logger = MLJFlow.Logger("http://127.0.0.1:5000/api")
run = MLJ.save(logger, mach)

service = MLJFlow.service(logger)

artifacts = MLFlowClient.listartifacts(service, run)
@assert !isempty(artifacts)

# ERROR: AssertionError: !(isempty(artifacts))
# Stacktrace:
#  [1] top-level scope
#    @ REPL[27]:1

# julia> versioninfo()
# Julia Version 1.10.3
# Commit 0b4590a5507 (2024-04-30 10:59 UTC)
# Build Info:
#   Official https://julialang.org/ release
# Platform Info:
#   OS: macOS (x86_64-apple-darwin22.4.0)
#   CPU: 12 × Intel(R) Core(TM) i7-8850H CPU @ 2.60GHz
#   WORD_SIZE: 64
#   LIBM: libopenlibm
#   LLVM: libLLVM-15.0.7 (ORCJIT, skylake)
# Threads: 12 default, 0 interactive, 6 GC (on 12 virtual cores)
# Environment:
#   JULIA_LTS_PATH = /Applications/Julia-1.6.app/Contents/Resources/julia/bin/julia
#   JULIA_PATH = /Applications/Julia-1.10.app/Contents/Resources/julia/bin/julia
#   JULIA_EGLOT_PATH = /Applications/Julia-1.7.app/Contents/Resources/julia/bin/julia
#   JULIA_NUM_THREADS = 12
#   JULIA_NIGHTLY_PATH = /Applications/Julia-1.10.app/Contents/Resources/julia/bin/julia

# (jl_I6VBrN) pkg> st
# Status `/private/var/folders/4n/gvbmlhdc8xj973001s6vdyw00000gq/T/jl_I6VBrN/Project.toml`
#   [64a0f543] MLFlowClient v0.5.1
#   [add582a8] MLJ v0.20.5

[ablaom]

@pebeto Can you reproduce?

[pebeto]

Hi @ablaom

This is happening due to a limitation implemented in MLFlowClient.jl. To make it working, you need to define an artifact_location in your Logger instance. I think I can solve this issue, but it belongs to the previously mentioned package (not here).

I'm actually pairing the library with the last mlflow version in a branch (which solves a lot of problems related to this), but it will take time. I will reference this issue in MLFlowClient.jl.

[andreuvall]

I am arriving here via https://github.com/JuliaAI/MLFlowClient.jl/issues/48, but I think that this issue describes better my situation, only that for me, it is using MLFlowClient.jl, not MLJFlow.jl.

I am using MLFlow version 2.7.1 If I remember correctly, I installed it from the conda-forge via Micromamba. I could double-check this. From the Julia side, I am using Julia 1.10 and MLFlowClient v0.5.1.

I start mlflow server (with defaults) from the same location where I then call MLFlowClient. I download this sample image (https://github.com/JuliaLang/julia-logo-graphics/blob/master/images/fontsample.png) to have an artifact to log, and I put it in the same location too. Then, I do

using MLFlowClient
mlf = MLFlow("http://localhost:5000/api")
experiment = createexperiment(mlf)
run = createrun(mlf, experiment)
logartifact(mlf, run, "fontsample.png")
updaterun(mlf, run, "FINISHED")

With this, fontsample.png is copied to

mlflow-artifacts:/472704068679330635/9c9008427165430f92c0a81c53f55d44/artifacts/fontsample.png

where 472704068679330635 is the name assigned to the experiment (I didn't choose one) and 9c9008427165430f92c0a81c53f55d44 is the id of the run. That is to say, the artifact is nicely identified by this path.

However, MLFlow doesn't know about this artifact. Through the API, I get an empty vector

julia> listartifacts(mlf::MLFlow, run)
Union{MLFlowArtifactDirInfo, MLFlowArtifactFileInfo}[]

and on the UI too

Since I am running MLFlow and MLFlowClient from the same location, I wouldn't have thought that the problem has to do with the default location of the artifacts, as addressed in https://github.com/JuliaAI/MLFlowClient.jl/issues/48. Or has it?

[pebeto]

Hi @andreuvall. This issue is happening because of the way MLFlow is handling with artifacts default directory (IMO, it's a horrible one). I don't remember the exact way, but in some of my tests, the REST API is not able to get the mlflow-artifacts because it is out of scope (i.e. mlflow ui is run outside the Julia project directory). One of the solutions I proposed was to let MLFlowClient.jl set a custom default directory ALWAYS. However, some of my tests were throwing some errors. I'm very busy right now to get into it, but this is one of my priorities for this project.

[andreuvall]

Thank you, @pebeto, for your quick reply!

One of the solutions I proposed was to let MLFlowClient.jl set a custom default directory ALWAYS. However, some of my tests were throwing some errors.

I don't understand why this would help, but this may well be because I don't know the internals. For me this is not an urgent matter, and unfortunately I also don't have much time available, but if you can provide a couple of pointers for me to get to understand this better, I will at least give it a look.