search

JuliaData / JuliaDBMeta.jl

Metaprogramming tools for JuliaDB
Other
33 stars 2 forks source link

@where problem for filtering with list #26

Closed pierrethiriet closed 5 years ago

pierrethiriet commented 6 years ago

I really like the macro provided by JuliaDBMeta. But I am currently having an issue converting a code working with JuliaBox to a local JuliaPro 0.6.2.2 and JuliaDBMeta 0.2.0. I have to filter a table based on a list of values. When passing the directly the list [...] in the macro @where, it works well while when using a name for the list, it is not:

julia> using JuliaDB
julia> using JuliaDBMeta
julia> t = table(@NT(a = [1,2,3], b = ["x","y","z"]))
Table with 3 rows, 2 columns:
a  b
──────
1  "x"
2  "y"
3  "z"
julia> @where t :b in ["x", "z"]
Table with 2 rows, 2 columns:
a  b
──────
1  "x"
3  "z"

is working while : 

julia> list = ["x", "z"]
2-element Array{String,1}:
 "x"
 "z"
julia> @where t :b in list
ERROR: BoundsError: attempt to access 3-element Array{Int64,1} at index [Any[tru
e, false, true]]

And with the usual JuliaDB syntax, it is working too:

julia> filter(c->c.b in list, t)
Table with 2 rows, 2 columns:
a  b
──────
1  "x"
3  "z"

Strangely, it is working on JuliaBox. Or maybe I did somthing wrong ?

piever commented 6 years ago

Thanks for reporting. I think you may be on an old version of IndexedTables that had some issues with map and type inference. What's your Pkg.status() ? Could you update to the latest version of IndexedTables and see if you still have this issue?

pierrethiriet commented 6 years ago

Thanks for your answer. The current status of Julia is:

julia> Pkg.status()
199 required packages:
...
 - IndexedTables                 0.6.0
...

It seems to be the latest standard version and I can not get the master one:

julia> Pkg.checkout("IndexedTables")
INFO: Checking out IndexedTables master...
INFO: Pulling IndexedTables latest master...
WARNING: IndexedTables is fixed at 0.7.3+ conflicting with top-level requirement
: [0.6.0,0.6.0+)
ERROR: fixed packages introduce conflicting requirements for PooledArrays:
         IndexedTables requires versions [0.2.0,8)
       available versions are 0.0.1, 0.0.2, 0.1.0, 0.1.1, 0.2.0 and 0.2.1

This issue happens with two with two of our computer both running with JuliaPro 6.2.2

piever commented 6 years ago

I'm not sure to what extent JuliaPRO allows you to update packages, but the latest JuliaDBMeta requires IndexedTables 0.7.2 and I would recommend trying to update IndexedTables.

I don't understand the error message though, if IndexedTables requires versions [0.2.0, 0.8), and available versions are 0.0.1, 0.0.2, 0.1.0, 0.1.1, 0.2.0 and 0.2.1, how is this conflicting?

Can you post your complete Pkg.status() so that I can get an idea whether there may be some packages holding you back from updating?

pierrethiriet commented 6 years ago

So here is complete Pkg.status()

julia> Pkg.status()
199 required packages:
 - ASTInterpreter2               0.1.1
 - AbstractFFTs                  0.3.0
 - Adapt                         0.2.0
 - Atom                          0.6.10
 - AutoGrad                      0.0.10
 - AutoHashEquals                0.2.0
 - AxisAlgorithms                0.2.0
 - AxisArrays                    0.2.1
 - BinDeps                       0.8.7
 - Blink                         0.6.2
 - Blosc                         0.3.0
 - BufferedStreams               0.3.3
 - BusinessDays                  0.8.2
 - CSV                           0.1.5
 - Calculus                      0.2.2
 - CatIndices                    0.1.0
 - CategoricalArrays             0.1.6
 - Cbc                           0.3.2
 - Clustering                    0.9.1
 - CodeTools                     0.5.0
 - CodecZlib                     0.4.2
 - Codecs                        0.4.0
 - ColorTypes                    0.6.7
 - ColorVectorSpace              0.5.2
 - Colors                        0.8.2
 - Combinatorics                 0.5.0
 - CommonSubexpressions          0.0.1
 - Compat                        0.54.0
 - Compose                       0.5.5
 - ComputationalResources        0.2.0
 - Conda                         0.7.1
 - Contour                       0.4.0
 - CoordinateTransformations     0.4.1
 - CoupledFields                 0.0.1
 - CustomUnitRanges              0.1.0
 - DBAPI                         0.1.0
 - DSP                           0.3.4
 - Dagger                        0.5.2
 - DataArrays                    0.6.2
 - DataFlow                      0.3.0
 - DataFrames                    0.10.1
 - DataStreams                   0.1.3
 - DataStructures                0.7.4
 - DataValues                    0.3.3
 - DebuggerFramework             0.1.2
 - DecisionTree                  0.6.5
 - DiffEqDiffTools               0.4.0
 - DiffResults                   0.0.3
 - DiffRules                     0.0.3
 - Distances                     0.5.0
 - Distributions                 0.15.0
 - DocSeeker                     0.1.0
 - DualNumbers                   0.3.0
 - FFTViews                      0.1.0
 - FFTW                          0.0.4
 - FileIO                        0.7.0
 - FixedPointNumbers             0.4.6
 - Flux                          0.4.1
 - Formatting                    0.3.0
 - ForwardDiff                   0.7.3
 - GLM                           0.8.1
 - GZip                          0.3.0
 - Gadfly                        0.6.5
 - Gallium                       0.1.0
 - Glob                          1.1.1
 - GraphPlot                     0.2.0
 - Graphics                      0.2.0
 - Gurobi                        0.4.0
 - HDF5                          0.8.8
 - Hexagons                      0.1.0
 - Hiccup                        0.1.1
 - HttpCommon                    0.4.0
 - HttpParser                    0.3.1
 - HttpServer                    0.3.0
 - HypothesisTests               0.6.0
 - IJulia                        1.7.0
 - IdentityRanges                0.1.0
 - ImageAxes                     0.4.0
 - ImageCore                     0.5.0
 - ImageFiltering                0.2.3
 - ImageMetadata                 0.4.0
 - ImageMorphology               0.0.2
 - ImageTransformations          0.4.1
 - Images                        0.12.0
 - IndexedTables                 0.6.0              master
 - IndirectArrays                0.3.0
 - Interact                      0.6.4
 - Interpolations                0.7.3
 - IntervalSets                  0.1.1
 - IterTools                     0.2.1
 - IteratorInterfaceExtensions   0.0.2
 - Iterators                     0.3.1
 - JDBC                          0.2.1
 - JLD                           0.8.3
 - JSON                          0.17.1
 - JavaCall                      0.5.2
 - JuMP                          0.18.0
 - JuliaDB                       0.7.2
 - JuliaDBMeta                   0.2.0
 - JuliaWebAPI                   0.3.6
 - Juno                          0.4.0
 - KernelDensity                 0.4.1
 - Knet                          0.9.0
 - LNR                           0.0.2
 - LaTeXStrings                  0.3.0
 - Lazy                          0.12.0
 - LearnBase                     0.1.6
 - LegacyStrings                 0.3.0
 - LibExpat                      0.4.1
 - Libz                          0.2.4
 - LightGraphs                   0.12.0
 - LightXML                      0.6.0
 - LineSearches                  3.2.5
 - Loess                         0.3.0
 - Logging                       0.3.1
 - LossFunctions                 0.2.0
 - MLBase                        0.7.0
 - MNIST                         0.0.2
 - MacroTools                    0.4.0
 - MappedArrays                  0.0.7
 - MathProgBase                  0.7.0
 - MbedTLS                       0.5.5
 - Measures                      0.1.0
 - Media                         0.3.0
 - MemPool                       0.0.6
 - Missings                      0.2.7
 - Mustache                      0.3.1
 - Mux                           0.2.3
 - NLSolversBase                 4.3.0
 - NLopt                         0.3.6
 - NNlib                         0.2.3
 - NaNMath                       0.3.1
 - NamedArrays                   0.7.0
 - NamedTuples                   4.0.0
 - NearestNeighbors              0.3.0
 - Nettle                        0.3.0
 - NullableArrays                0.1.2
 - Nullables                     0.0.3
 - Nulls                         0.1.2
 - ODBC                          0.5.2
 - OffsetArrays                  0.5.0
 - OnlineStats                   0.15.4
 - OnlineStatsBase               0.5.1
 - Optim                         0.14.0
 - PDMats                        0.8.0
 - PaddedViews                   0.3.0
 - Parameters                    0.8.1
 - ParserCombinator              1.7.11
 - PenaltyFunctions              0.0.2
 - Polynomials                   0.2.1
 - PooledArrays                  0.1.1
 - PositiveFactorizations        0.1.0
 - Primes                        0.2.0
 - ProtoBuf                      0.5.0
 - PyCall                        1.15.0             master (dirty)
 - PyPlot                        2.3.2
 - QuadGK                        0.2.0
 - QuantEcon                     0.14.1
 - RCall                         0.8.1
 - RangeArrays                   0.2.0
 - Ratios                        0.2.0
 - Reactive                      0.6.0
 - RecipesBase                   0.2.3
 - Reexport                      0.1.0
 - Requests                      0.5.1
 - Requires                      0.4.3
 - ReverseDiffSparse             0.8.1
 - Rmath                         0.3.2
 - Roots                         0.5.0
 - Rotations                     0.6.1
 - SHA                           0.5.6
 - SIUnits                       0.1.0
 - SQLite                        0.4.5
 - ScikitLearnBase               0.3.0
 - ShowItLikeYouBuildIt          0.2.0
 - Showoff                       0.1.1
 - SimpleTraits                  0.6.0
 - SortingAlgorithms             0.2.0
 - SpecialFunctions              0.3.8
 - StaticArrays                  0.6.6
 - StatsBase                     0.20.1
 - StatsFuns                     0.5.0
 - StringDistances               0.2.0
 - SweepOperator                 0.1.0
 - TableTraits                   0.1.0
 - TableTraitsUtils              0.1.3
 - TexExtensions                 0.1.0
 - TextParse                     0.4.0
 - TiledIteration                0.1.0
 - TimeZones                     0.6.2
 - Tokenize                      0.4.2
 - TranscodingStreams            0.4.1
 - URIParser                     0.3.0
 - WeakRefStrings                0.2.0
 - WebSockets                    0.4.0
 - WinRPM                        0.3.1
 - WinReg                        0.3.0
 - WoodburyMatrices              0.2.2
 - ZMQ                           0.5.1
22 additional packages:
 - BinaryProvider                0.2.3
 - Cairo                         0.5.2
 - DecFP                         0.4.3
 - FreeType                      2.0.0
 - FreeTypeAbstraction           0.1.0
 - GLAbstraction                 0.6.1
 - GLFW                          2.0.0-alpha
 - GLVisualize                   0.7.0
 - GLWindow                      0.7.1
 - GeometryTypes                 0.4.5
 - GraphLayout                   0.3.1
 - GraphVisualize                0.0.0-             master (unregistered)
 - Junet                         0.0.0-             master (unregistered)
 - MeshIO                        0.1.3
 - Mocking                       0.5.0
 - ModernGL                      0.2.1
 - Packing                       0.1.0
 - Quaternions                   0.3.0
 - SignedDistanceFields          0.2.0
 - Suppressor                    0.1.0
 - UnicodeFun                    0.2.0
 - vOptGeneric                   0.1.1+             master
piever commented 6 years ago

It doesn't look like there's any pinned package, so I'd think it should update successfully the next time you run Pkg.update()

pierrethiriet commented 6 years ago

Thanks for your help. For some reason, the packages are not updated with the command Pkg.update() but I guess it is not related to the JuliaDBMeta package. I'll try to install a clean Julia without JuliaPro and see if the situation is improved.

piever commented 6 years ago

Good idea! Please do let me know if the problem goes away on a fresh Julia install with up to date packages.