ByRow and transform not working

BuddhiLW commented 3 years ago

I called, beforehand, with success,

        # Import Packages
        using Pkg
        using DataFrames
        using CSV
        using Plots
        using GLM
        using StatsBase
        using Lathe
        using MLBase
        using ClassImbalance
        using ROCAnalysis
        #using DataFrameTools
        #using DataFramesMeta 
        #using DataFrameMacros

        # Enable printing of 1000 columns
        ENV["COLUMNS"] = 1000

julia> ByRow
ERROR: UndefVarError: ByRow not defined

(Regression) pkg> st
      Status `~/PP/MonitoriaEstatistica/Regression/Project.toml`
  [336ed68f] CSV v0.9.3
  [04a18a73] ClassImbalance v0.8.7
  [a93c6f00] DataFrames v0.20.2
  [38e38edf] GLM v1.4.2
  [38d8eb38] Lathe v0.0.9
  [f0e99cf1] MLBase v0.8.0
  [91a5bcdd] Plots v1.22.0
  [f535d66d] ROCAnalysis v0.3.3
  [2913bbd2] StatsBase v0.32.2

julia> transform
transform (generic function with 2 methods)

julia> DataFrames.transform
ERROR: UndefVarError: transform not defined
Stacktrace:
 [1] getproperty(x::Module, f::Symbol)
   @ Base ./Base.jl:26
 [2] top-level scope
   @ REPL[9]:1

How could I load ByRow and transform?

bkamins commented 3 years ago

[a93c6f00] DataFrames v0.20.2

This shows you that you are on an old version of DataFrames.jl. The current version is 1.2.2.

Do the following:

run add DataFrames@1.2.2 to force installation of the latest version of DataFrames.jl; most likely it will fail
read https://bkamins.github.io/julialang/2020/05/11/package-version-restrictions.html and https://bkamins.github.io/julialang/2020/05/18/project-workflow.html (only the first part; the following sections are more advanced and can be skipped) to understand the reason and typical practices of handling the problems like you have
open a PR to the package that causes the problem (some package you have installed is holding back DataFrames.jl update)

BuddhiLW commented 3 years ago

Could you explain better what should I change in the downing-package (which is Lathe in this case), so I can make a pull request?

I have never contributed to fixing a versioning conflict.

And, thanks for the explanation.

BuddhiLW commented 3 years ago

By the way, the problem persists

  ux = unique(df.Geography);
  df_=transform(df,
        @. :Geography => ByRow(isequal(ux)) .=> Symbol.(ux));
  df_=transform(df_,[true,false] => [1,0])
  first(df_,5);

ERROR: LoadError: UndefVarError: ByRow not defined
Stacktrace:
 [1] top-level scope
   @ /tmp/babel-SiSTTm/julia-vterm-src-gLjwGe:3
 [2] include(fname::String)
   @ Base.MainInclude ./client.jl:444
 [3] top-level scope
   @ REPL[57]:1
in expression starting at /tmp/babel-SiSTTm/julia-vterm-src-gLjwGe:1

(Regression) pkg> status DataFrames
      Status `~/PP/MonitoriaEstatistica/Regression/Project.toml`
  [a93c6f00] DataFrames v1.2.2

nalimilan commented 3 years ago

Could you explain better what should I change in the downing-package (which is Lathe in this case), so I can make a pull request?

A first step would be to edit https://github.com/ChifiSource/Lathe.jl/blob/master/Project.toml to use `DataFrames = "0.22, 1" and see whether tests pass. If not, the failing lines will have to be updated to the latest DataFrames syntax, which in general should be quite easy.

By the way, the problem persists

Have you restarted Julia after updating to DataFrames 1.2.2?

bkamins commented 3 years ago

Also:

@. :Geography => ByRow(isequal(ux)) .=> Symbol.(ux)

should be:

:Geography .=> ByRow(isequal(ux)) .=> Symbol.(ux)

bkamins commented 3 years ago

Could you explain better what should I change in the downing-package

You probably alternatively can just drop an issue in the package and ask if maintainers would be willing to resolve the issue if you are not sure how to do it yourself.

BuddhiLW commented 3 years ago

One of the issues I'm having is with ClassImbalance.jl. The project seems inactive - 2 years without an update in the repository.

I'm willing to do it, but if I had a guide would be great.

In, https://github.com/JuliaLang/Compat.jl/blob/master/Project.toml, I changed the compat to:

[compat]
Compat = "3.37.0"
DataFrames = "1.2.2"
Distributions = "0.25.16"
StatsBase = "0.33.2"
julia = "1.6"

I have forked the project, and I ran

(Package-updates) pkg> up
    Updating registry at `~/.julia/registries/General`
  No Changes to `~/PP/Julia/Package-updates/Project.toml`
  No Changes to `~/PP/Julia/Package-updates/Manifest.toml`
Precompiling project...
  ✓ SortingAlgorithms
  ✓ StatsFuns
  ✓ StatsBase
  ✓ Distributions
  ✓ DataFrames
  7 dependencies successfully precompiled in 16 seconds (32 already precompiled)
  5 dependencies precompiled but different versions are currently loaded. Restart julia to access the new versions

Seems like the tests are great. But, I have trouble installing an R package needed.

(ClassImbalance) pkg> test
     Testing ClassImbalance
      Status `/tmp/jl_niwH69/Project.toml`
  [04a18a73] ClassImbalance v0.8.8 `~/PP/Julia/Package-updates/ClassImbalance.jl`
  [34da2185] Compat v3.37.0
  [8f4d0f93] Conda v1.5.2
  [a93c6f00] DataFrames v1.2.2
  [31c24e10] Distributions v0.25.16
  [38e38edf] GLM v1.5.1
  [2913bbd2] StatsBase v0.33.10
  [37e2e46d] LinearAlgebra `@stdlib/LinearAlgebra`
  [44cfe95a] Pkg `@stdlib/Pkg`
  [9a3f8284] Random `@stdlib/Random`
  [10745b16] Statistics `@stdlib/Statistics`
  [8dfed614] Test `@stdlib/Test`
      Status `/tmp/jl_niwH69/Manifest.toml`
  [d360d2e6] ChainRulesCore v1.4.0
  [04a18a73] ClassImbalance v0.8.8 `~/PP/Julia/Package-updates/ClassImbalance.jl`
  [34da2185] Compat v3.37.0
  [8f4d0f93] Conda v1.5.2
  [a8cc5b0e] Crayons v4.0.4
  [9a962f9c] DataAPI v1.8.0
  [a93c6f00] DataFrames v1.2.2
  [864edb3b] DataStructures v0.18.10
  [e2d170a0] DataValueInterfaces v1.0.0
  [31c24e10] Distributions v0.25.16
  [ffbed154] DocStringExtensions v0.8.5
  [1a297f60] FillArrays v0.12.4
  [59287772] Formatting v0.4.2
  [38e38edf] GLM v1.5.1
  [41ab1584] InvertedIndices v1.1.0
  [92d709cd] IrrationalConstants v0.1.0
  [82899510] IteratorInterfaceExtensions v1.0.0
  [692b3bcd] JLLWrappers v1.3.0
  [682c06a0] JSON v0.21.2
  [2ab3a3ac] LogExpFunctions v0.3.3
  [e1d29d7a] Missings v1.0.2
  [bac558e1] OrderedCollections v1.4.1
  [90014a1f] PDMats v0.11.1
  [69de0a69] Parsers v2.0.3
  [2dfb63ee] PooledArrays v1.3.0
  [21216c6a] Preferences v1.2.2
  [08abe8d2] PrettyTables v1.1.0
  [1fd47b50] QuadGK v2.4.2
  [189a3867] Reexport v1.2.2
  [79098fc4] Rmath v0.7.0
  [1277b4bf] ShiftedArrays v1.0.0
  [a2af1166] SortingAlgorithms v1.0.1
  [276daf66] SpecialFunctions v1.6.1
  [82ae8749] StatsAPI v1.0.0
  [2913bbd2] StatsBase v0.33.10
  [4c63d2b9] StatsFuns v0.9.10
  [3eaba693] StatsModels v0.6.26
  [3783bdb8] TableTraits v1.0.1
  [bd369af6] Tables v1.5.2
  [81def892] VersionParsing v1.2.0
  [efe28fd5] OpenSpecFun_jll v0.5.5+0
  [f50d1b31] Rmath_jll v0.3.0+0
  [0dad84c5] ArgTools `@stdlib/ArgTools`
  [56f22d72] Artifacts `@stdlib/Artifacts`
  [2a0f44e3] Base64 `@stdlib/Base64`
  [ade2ca70] Dates `@stdlib/Dates`
  [8bb1440f] DelimitedFiles `@stdlib/DelimitedFiles`
  [8ba89e20] Distributed `@stdlib/Distributed`
  [f43a241f] Downloads `@stdlib/Downloads`
  [9fa8497b] Future `@stdlib/Future`
  [b77e0a4c] InteractiveUtils `@stdlib/InteractiveUtils`
  [b27032c2] LibCURL `@stdlib/LibCURL`
  [76f85450] LibGit2 `@stdlib/LibGit2`
  [8f399da3] Libdl `@stdlib/Libdl`
  [37e2e46d] LinearAlgebra `@stdlib/LinearAlgebra`
  [56ddb016] Logging `@stdlib/Logging`
  [d6f4376e] Markdown `@stdlib/Markdown`
  [a63ad114] Mmap `@stdlib/Mmap`
  [ca575930] NetworkOptions `@stdlib/NetworkOptions`
  [44cfe95a] Pkg `@stdlib/Pkg`
  [de0858da] Printf `@stdlib/Printf`
  [3fa0cd96] REPL `@stdlib/REPL`
  [9a3f8284] Random `@stdlib/Random`
  [ea8e919c] SHA `@stdlib/SHA`
  [9e88b42a] Serialization `@stdlib/Serialization`
  [1a1011a3] SharedArrays `@stdlib/SharedArrays`
  [6462fe0b] Sockets `@stdlib/Sockets`
  [2f01184e] SparseArrays `@stdlib/SparseArrays`
  [10745b16] Statistics `@stdlib/Statistics`
  [4607b0f0] SuiteSparse `@stdlib/SuiteSparse`
  [fa267f1f] TOML `@stdlib/TOML`
  [a4e569a6] Tar `@stdlib/Tar`
  [8dfed614] Test `@stdlib/Test`
  [cf7118a7] UUIDs `@stdlib/UUIDs`
  [4ec0a83e] Unicode `@stdlib/Unicode`
  [e66e0078] CompilerSupportLibraries_jll `@stdlib/CompilerSupportLibraries_jll`
  [deac9b47] LibCURL_jll `@stdlib/LibCURL_jll`
  [29816b5a] LibSSH2_jll `@stdlib/LibSSH2_jll`
  [c8ffd9c3] MbedTLS_jll `@stdlib/MbedTLS_jll`
  [14a3606d] MozillaCACerts_jll `@stdlib/MozillaCACerts_jll`
  [83775a58] Zlib_jll `@stdlib/Zlib_jll`
  [8e850ede] nghttp2_jll `@stdlib/nghttp2_jll`
  [3f19e933] p7zip_jll `@stdlib/p7zip_jll`
     Testing Running tests...
[ Info: Running `conda config --add channels r --file /home/buddhilw/.julia/conda/3/condarc-julia.yml --force` in root environment
Warning: 'r' already in 'channels' list, moving to the top
[ Info: Running `conda install -y r-base` in root environment
Collecting package metadata (current_repodata.json): done
Solving environment: done

# All requested packages already installed.

    Updating registry at `~/.julia/registries/General`
   Resolving package versions...
    Updating `/tmp/jl_niwH69/Project.toml`
  [6f49c342] + RCall v0.13.12
    Updating `/tmp/jl_niwH69/Manifest.toml`
  [324d7699] + CategoricalArrays v0.10.0
  [6f49c342] + RCall v0.13.12
  [3cdcf5f2] + RecipesBase v1.1.2
  [ae029012] + Requires v1.1.3
  [856f2bd8] + StructTypes v1.7.3
  [1b915085] + WinReg v0.3.1
    Building Conda → `~/.julia/scratchspaces/44cfe95a-1eb2-52ea-b672-e2afdf69b78f/299304989a5e6473d985212c28928899c74e9421/build.log`
    Building RCall → `~/.julia/scratchspaces/44cfe95a-1eb2-52ea-b672-e2afdf69b78f/80a056277142a340e646beea0e213f9aecb99caa/build.log`
┌ Warning: ignoring error: 
│   e =
│    REvalError: Loading required package: DMwR
│    Warning in library(package, lib.loc = lib.loc, character.only = TRUE, logical.return = TRUE,  :
│      there is no package called 'DMwR'
│    Error in install.packages("DMwR", lib = Sys.getenv("R_LIBS_USER"), repos = "http://cran.r-project.org/",  : 
│      type 'binary' is not supported on this platform
└ @ Main ~/PP/Julia/Package-updates/ClassImbalance.jl/test/install_dmwr.jl:17
┌ Warning: RCall.jl: Loading required package: DMwR
│ Warning in library(package, lib.loc = lib.loc, character.only = TRUE, logical.return = TRUE,  :
│   there is no package called 'DMwR'
│ Warning: package 'DMwR' is not available (for R version 3.6.1)
└ @ RCall ~/.julia/packages/RCall/iMDW2/src/io.jl:160
ERROR: LoadError: LoadError: REvalError: Error in library("DMwR") : there is no package called 'DMwR'
in expression starting at /home/buddhilw/PP/Julia/Package-updates/ClassImbalance.jl/test/install_dmwr.jl:32
in expression starting at /home/buddhilw/PP/Julia/Package-updates/ClassImbalance.jl/test/runtests.jl:9
ERROR: Package ClassImbalance errored during testing

I will make a pull request there.

In the meantime, how can I use this local version, which is updated?

BuddhiLW commented 3 years ago

(ClassImbalance) pkg> st
     Project ClassImbalance v0.8.8
      Status `~/PP/Julia/Package-updates/ClassImbalance.jl/Project.toml`
  [34da2185] Compat v3.37.0
  [a93c6f00] DataFrames v1.2.2
  [31c24e10] Distributions v0.25.16
  [2913bbd2] StatsBase v0.33.10
  [37e2e46d] LinearAlgebra
  [9a3f8284] Random
  [10745b16] Statistics

bkamins commented 3 years ago

In the meantime, how can I use this local version, which is updated?

Do ] dev PACKAGE_NAME

and then you can change whatever you want in the dev directory of the package.

JuliaData / DataFrames.jl

ByRow and transform not working #2884