Power Models Distribution State Estimation package conflict

[gaadis]

I am trying to install the Power Models Distribution State Estimation package, but I am getting package error conflicts as shown below. Any ideas on how to resolve this?

julia> Pkg.add("PowerModelsDistributionStateEstimation") Resolving package versions... ERROR: Unsatisfiable requirements detected for package KernelDensity [5ab0869b]: KernelDensity [5ab0869b] log: ├─possible versions are: [0.5.0-0.5.1, 0.6.0-0.6.2] or uninstalled ├─restricted by compatibility requirements with StatsPlots [f3b207a7] to versions: [0.5.0-0.5.1, 0.6.0-0.6.2] │ └─StatsPlots [f3b207a7] log: │ ├─possible versions are: [0.10.0-0.10.2, 0.11.0, 0.12.0, 0.13.0, 0.14.0-0.14.17] or uninstalled │ └─restricted to versions by an explicit requirement, leaving only versions [0.10.0-0.10.2, 0.11.0, 0.12.0, 0.13.0, 0.14.0-0.14.17] ├─restricted by compatibility requirements with Optim [429524aa] to versions: 0.6.0-0.6.2 or uninstalled, leaving only versions: 0.6.0-0.6.2 │ └─Optim [429524aa] log: │ ├─possible versions are: [0.15.3, 0.16.0, 0.17.0-0.17.2, 0.18.0-0.18.1, 0.19.0-0.19.7, 0.20.0-0.20.6, 0.21.0, 0.22.0, 1.0.0, 1.1.0, 1.2.0-1.2.1] or uninstalled │ ├─restricted to versions by an explicit requirement, leaving only versions [0.15.3, 0.16.0, 0.17.0-0.17.2, 0.18.0-0.18.1, 0.19.0-0.19.7, 0.20.0-0.20.6, 0.21.0, 0.22.0, 1.0.0, 1.1.0, 1.2.0-1.2.1] │ └─restricted by compatibility requirements with PowerModelsDistributionStateEstimation [d0713e65] to versions: 1.2.0-1.2.1 │ └─PowerModelsDistributionStateEstimation [d0713e65] log: │ ├─possible versions are: [0.1.2, 0.2.0-0.2.2] or uninstalled │ └─restricted to versions * by an explicit requirement, leaving only versions [0.1.2, 0.2.0-0.2.2] └─restricted by compatibility requirements with Distributions [31c24e10] to versions: 0.5.0-0.5.1 or uninstalled — no versions left └─Distributions [31c24e10] log: ├─possible versions are: [0.16.0-0.16.4, 0.17.0, 0.18.0, 0.19.1-0.19.2, 0.20.0, 0.21.0-0.21.3, 0.21.5-0.21.12, 0.22.0-0.22.6, 0.23.0-0.23.12, 0.24.0-0.24.10] or uninstalled ├─restricted by compatibility requirements with StatsPlots [f3b207a7] to versions: [0.16.0-0.16.4, 0.17.0, 0.18.0, 0.19.1-0.19.2, 0.20.0, 0.21.0-0.21.3, 0.21.5-0.21.12, 0.22.0-0.22.6, 0.23.0-0.23.12, 0.24.0-0.24.10] │ └─StatsPlots [f3b207a7] log: see above └─restricted by compatibility requirements with PowerModelsDistributionStateEstimation [d0713e65] to versions: 0.22.3-0.22.6 └─PowerModelsDistributionStateEstimation [d0713e65] log: see above Stacktrace: [1] propagate_constraints!(::Pkg.Resolve.Graph, ::Set{Int64}; log_events::Bool) at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\Pkg\src\Resolve\graphtype.jl:1005 [2] propagate_constraints! at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\Pkg\src\Resolve\graphtype.jl:946 [inlined] (repeats 2 times) [3] simplify_graph!(::Pkg.Resolve.Graph, ::Set{Int64}; clean_graph::Bool) at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\Pkg\src\Resolve\graphtype.jl:1460 [4] simplify_graph! at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\Pkg\src\Resolve\graphtype.jl:1460 [inlined] (repeats 2 times) [5] resolve_versions!(::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}) at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\Pkg\src\Operations.jl:375 [6] targeted_resolve at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\Pkg\src\Operations.jl:1114 [inlined] [7] tiered_resolve(::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}) at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\Pkg\src\Operations.jl:1100 [8] _resolve at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\Pkg\src\Operations.jl:1120 [inlined] [9] add(::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}, ::Array{Base.UUID,1}; preserve::Pkg.Types.PreserveLevel, platform::Pkg.BinaryPlatforms.Windows) at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\Pkg\src\Operations.jl:1135 [10] add(::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}; preserve::Pkg.Types.PreserveLevel, platform::Pkg.BinaryPlatforms.Windows, kwargs::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}) at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\Pkg\src\API.jl:188 [11] add(::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}) at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\Pkg\src\API.jl:139 [12] #add#21 at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\Pkg\src\API.jl:67 [inlined] [13] add at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\Pkg\src\API.jl:67 [inlined] [14] #add#20 at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\Pkg\src\API.jl:66 [inlined] [15] add at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\Pkg\src\API.jl:66 [inlined] [16] add(::String; kwargs::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}) at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\Pkg\src\API.jl:65 [17] add(::String) at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\Pkg\src\API.jl:65 [18] top-level scope at none:1

The result of Pkg.status() is below julia> Pkg.status() Status C:\Users\Gadis\.julia\environments\JuliaPro_v1.5.3-1\Project.toml [c52e3926] Atom v0.12.25 ⚲ [336ed68f] CSV v0.8.2 [da1fd8a2] CodeTracking v1.0.5 [a93c6f00] DataFrames v0.22.2 [1a297f60] FillArrays v0.8.14 [2e9cd046] Gurobi v0.9.7 [7073ff75] IJulia v1.23.1 [b6b21f68] Ipopt v0.6.5 [4076af6c] JuMP v0.21.5 [e5e0dc1b] Juno v0.8.4 ⚲ [2fda8390] LsqFit v0.12.0 [429524aa] Optim v1.2.2 [4722fa14] PkgAuthentication v1.0.2 [91a5bcdd] Plots v1.10.1 [c36e90e8] PowerModels v0.17.3 [d7431456] PowerModelsDistribution v0.10.0 [0448d7d9] RandomizedLinAlg v0.1.0 [295af30f] Revise v3.1.11 [c946c3f1] SCS v0.7.1 [2913bbd2] StatsBase v0.33.2 [f3b207a7] StatsPlots v0.14.17 [028f657a] TotalLeastSquares v1.6.0 [44d3d7a6] Weave v0.10.6 [fdbf4ff8] XLSX v0.7.3 [37e2e46d] LinearAlgebra [de0858da] Printf

[MartaVanin]

Hi @gaadis, looking at your Pkg.status() output, I see that you are installing a lot of packages in the default Julia environment. If a lot of different packages with different compatibility requirements in their dependencies are installed in the same place, it can be quite common that the requirements of a package conflict with those of another package, as it is the case here. It is a good practice to create different environments for different projects, and install in each environment only the packages you need for that specific project. You can find an explanation on how to use environments here: https://towardsdatascience.com/how-to-setup-project-environments-in-julia-ec8ae73afe9c . In this case it is StatsPlots that has a conflict with PMDSE, so two quick possible solutions are the following: 1) create an environment for your state estimation-related worked that has PMDSE installed but not StatsPlots 2) keep things as they are but remove StatsPlots with Pkg.rm("StatsPlots") (but I strongly recommend you use environments)

If you need StatsPlots and PMDSE to be absolutely in the same project, I can look into fixing this dependency issue from its "root", but I might take some time, so 1) or 2) would be faster.

[MartaVanin]

Is this issue solved for you? I would like to close it if so

[gaadis]

I created a different environment for my state estimation project and that resolved the issue. Thanks for your help.

[gaadis]

When I run the State Estimation code, I get the error below at the "add_measurements!(data, msr_path; actual_meas = true, seed = 0)" function

ERROR: LoadError: ArgumentError: provide a valid sink argument, like using DataFrames; CSV.read(source, DataFrame)

I then went to the measurement_parser.jl function, copied the "add_measurement" function and changed meas_df = _CSV.read(meas_file) to meas_df = CSV.read(meas_file, DataFrame) and this solved the problem.

I then got this error "LoadError: ArgumentError: syntax df[column] is not supported use df[!, column] instead" I changed the syntax to what the code suggested and this worked

Then from the read_measurement function, I got the error "LoadError: UndefVarError: _DST not defined", which I don't know how to get rid of.

Ideally, I'd want to use the add_measurement function as it is in the State Estimation code, without having to copy the code but I can't modify the measurement_parser.jl code. Is there any way to fix the error "ERROR: LoadError: ArgumentError: provide a valid sink argument, like using DataFrames; CSV.read(source, DataFrame)" given by the add_measurement function? Your expert suggestion is appreciated.

[MartaVanin]

Hello gaadis, it would be great if you could provide a "minimal working example" of your code, I can better help you if I can reproduce the error on my machine. Without seeing your code, my feeling is that you might be using a more recent version of the CSV or DataFrames packages with respect to what I use, and that some of their functions changed and are causing this. Can you tell me which version of these two packages you are using? I use CSV v0.6.2 and DataFrames v0.21.8 and this works fine for me. A quick fix for you should be to use the same versions, but if it turns out that PMDSE breaks with more recent CSV and DF releases I will also definitely update the code so that this doesn't happen anymore! Regarding the error "LoadError: UndefVarError: _DST not defined": this happens because you are indeed not using the add_measurements! function from within the PMDSE package: _DST is a constant which is defined inside PMDSE, and that indicates the Distributions.jl package. If you try to copy the code from PMDSE you will bump in a lot of errors like this, so I strongly advise against it.

[gaadis]

I'm using CSV v0.6.2 and DataFrames v0.21.8. The code I'm trying to run is the SE code from the library. The code is posted below. Thanks.

module SE

greet() = print("Hello World!")

end # module

A Pluto.jl notebook

v0.12.10

using Markdown using InteractiveUtils using JuMP, Ipopt #Adding many packages on one line might cause long loading times using PowerModels, PowerModelsDistribution using PowerModelsDistributionStateEstimation using Plots using CSV, DataFrames hint(text) = Markdown.MD(Markdown.Admonition("hint", "Hint", [text]));

ntw_path = joinpath(BASE_DIR, "test/data/extra/networks/case3_unbalanced.dss") msr_path = joinpath(BASE_DIR, "test/data/extra/measurements/case3_meas.csv") data = PowerModelsDistribution.parse_file(ntw_path; data_model=MATHEMATICAL) add_measurements!(data, msr_path, actual_meas = true) data["se_settings"] = Dict{String,Any}("criterion" => "rwlav", "rescaler" => 1) slv = JuMP.optimizer_with_attributes(Ipopt.Optimizer, "tol"=>1e-6, "print_level"=>0) se_result = run_acp_mc_se(data, slv) se_result["solution"]["load"] data["meas"] begin pf_results = PowerModelsDistribution.run_mc_pf(data, PowerModels.ACPPowerModel, slv) pf_results["solution"]["bus"] end delta, max_err, avg = calculate_voltage_magnitude_error(se_result, pf_results) scatter(delta, xlabel="Index [-]", ylabel="ϵ [p.u.]", ylims = [0, max_err1.05], title="Absolute voltage difference ϵ", legend = false) begin different_msr_path = pwd()"//tmp_meas.csv" write_measurements!(PowerModels.ACPPowerModel, data, pf_results, different_msr_path) end meas_df = CSV.read(different_msr_path)

begin add_measurements!(data, msr_path, actual_meas = false) data["meas"]["5"] end

meas_df[5, :]

begin update_all_bounds!(data; v_min = 0.85, v_max = 1.15, pg_min=-1.0, pg_max = 1.0, qg_min=-1.0, qg_max=1.0, pd_min=-1.0, pd_max=1.0, qd_min=-1.0, qd_max=1.0 ) assign_start_to_variables!(data) data["bus"] end

[MartaVanin]

The .jl file in examples is a Pluto notebook, so it's maybe more straightforward to run it as such for the first time, instead of copy-pasting to a julia IDE or REPL: all those "begin"s and "end"s are not part of the julia code but are required to run the notebook itself. To see how julia notebooks work: https://github.com/fonsp/Pluto.jl But either way, I have copy pasted the code above line by line (starting from "using JuMP") in the julia REPL and it works fine for me, so I can't reproduce the error. I use Julia1.5.1 for windows and that's my full environment (some packages are not necessary for this code): [336ed68f] CSV v0.6.2 [f65535da] Convex v0.14.1 [a93c6f00] DataFrames v0.21.8 [1313f7d8] DataFramesMeta v0.6.0 [31c24e10] Distributions v0.22.6 [b6b21f68] Ipopt v0.6.5 [4076af6c] JuMP v0.21.4 [f28f55f0] Memento v1.1.2 [429524aa] Optim v1.2.3 [91a5bcdd] Plots v1.10.1 [c3e4b0f8] Pluto v0.12.18 [d7431456] PowerModelsDistribution v0.10.0 https://github.com/lanl-ansi/PowerModelsDistribution.jl.git#0.10-rc [d0713e65] PowerModelsDistributionStateEstimation v0.2.3 [6dd1b50a] Tulip v0.7.1

Could it be that you have something else in your environment which you are using and conflicts with CSV?

[gaadis]

I am using Julia 1.5.3-1 and my full environment is below [336ed68f] CSV v0.6.2 [a93c6f00] DataFrames v0.21.8 [b6b21f68] Ipopt v0.6.3 [4076af6c] JuMP v0.21.4 [91a5bcdd] Plots v1.10.1 [c3e4b0f8] Pluto v0.12.18 [c36e90e8] PowerModels v0.17.3 [d7431456] PowerModelsDistribution v0.10.0 [d0713e65] PowerModelsDistributionStateEstimation v0.2.3

[MartaVanin]

I really don't seem to figure out what's happening, sorry. I got Julia.1.5.3, PowerModelsDistribution0.10.0 and PowerModelsDistributionStateEstimation 0.2.3, and the very minimal code below works fine for me:

using PowerModelsDistribution, PowerModelsDistributionStateEstimation
ntw_path = joinpath(BASE_DIR, "test/data/extra/networks/case3_unbalanced.dss")
msr_path = joinpath(BASE_DIR, "test/data/extra/measurements/case3_meas.csv")
data = PowerModelsDistribution.parse_file(ntw_path; data_model=MATHEMATICAL)
add_measurements!(data, msr_path, actual_meas = true)

If it still doesn't work for you, I see two possible causes: 1) you are using JuliaPro instead of just Julia. Normally this shouldn't be a problem, but it's the only difference I see between our two setups (except maybe the OS?). So I would try to use just a Julia installation instead of JuliaPro and see if that fixes it. 2) maybe there is something old in the memory that you are pointing to. I would try to reboot the computer and try again, if you haven't already

[gaadis]

I really don't seem to figure out what's happening, sorry. I got Julia.1.5.3, PowerModelsDistribution0.10.0 and PowerModelsDistributionStateEstimation 0.2.3, and the very minimal code below works fine for me:

using PowerModelsDistribution, PowerModelsDistributionStateEstimation
ntw_path = joinpath(BASE_DIR, "test/data/extra/networks/case3_unbalanced.dss")
msr_path = joinpath(BASE_DIR, "test/data/extra/measurements/case3_meas.csv")
data = PowerModelsDistribution.parse_file(ntw_path; data_model=MATHEMATICAL)
add_measurements!(data, msr_path, actual_meas = true)

If it still doesn't work for you, I see two possible causes:

1. you are using JuliaPro instead of just Julia. Normally this shouldn't be a problem, but it's the only difference I see between our two setups (except maybe the OS?). So I would try to use just a Julia installation instead of JuliaPro and see if that fixes it.
2. maybe there is something old in the memory that you are pointing to. I would try to reboot the computer and try again, if you haven't already

I uninstalled JuliaPro, reboot the computer, then reinstalled JuliaPro. This fixed the problem. Thanks for your suggestion and help.

[MartaVanin]

FYI I fixed the compatibility issue, so you can have PowerModelsDistributionStateEstimation and StatsPlots in the same environment, starting from PMDSE v0.2.4

[gaadis]

Great thanks.