search

JuliaText / CorpusLoaders.jl

A variety of loaders for various NLP corpora.
Other
32 stars 13 forks source link

Tests failing #48

Open ablaom opened 3 years ago

ablaom commented 3 years ago

Started on #47 but having problem getting the current release to pass tests:

     Testing Running tests...
Test Summary: | Pass  Total
types         |   15     15
┌ Warning: Error parsing "<wf cmd=verify_tag pos=VBP ot=notag>'ve</wf>". AssertionError("No parser for \"<wf cmd=verify_tag pos=VBP ot=notag>'ve</wf>\"")
└ @ CorpusLoaders ~/.julia/packages/CorpusLoaders/RkzqR/src/apply_subparsers.jl:23
Test Summary: | Pass  Total
SemCor        |   35     35
Test Summary: | Pass  Total
Senseval3     |    4      4
Test Summary: | Pass  Total
CoNLL         |    9      9
Test Summary: | Pass  Total
CoNLL2000     |    7      7
basic use train_pos: Test Failed at /Users/anthony/.julia/packages/CorpusLoaders/RkzqR/test/test_IMDB.jl:15
  Expression: typeof(words) == Vector{String}
   Evaluated: Vector{Any} == Vector{String}
``` Stacktrace: [1] macro expansion @ ~/.julia/packages/CorpusLoaders/RkzqR/test/test_IMDB.jl:15 [inlined] [2] top-level scope @ /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.6/Test/src/Test.jl:1226 [3] include(fname::String) @ Base.MainInclude ./client.jl:444 [4] macro expansion @ ~/.julia/packages/CorpusLoaders/RkzqR/test/runtests.jl:20 [inlined] [5] top-level scope @ /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.6/Test/src/Test.jl:1226 [6] include(fname::String) @ Base.MainInclude ./client.jl:444 [7] top-level scope @ none:6 [8] eval @ ./boot.jl:360 [inlined] [9] exec_options(opts::Base.JLOptions) @ Base ./client.jl:261 [10] _start() @ Base ./client.jl:485 basic use train_pos: Test Failed at /Users/anthony/.julia/packages/CorpusLoaders/RkzqR/test/test_IMDB.jl:18 Expression: typeof(sentences) == #= /Users/anthony/.julia/packages/CorpusLoaders/RkzqR/test/test_IMDB.jl:18 =# @NestedVector(String, 2) Evaluated: Vector{Any} == Vector{Vector{String}} Stacktrace: [1] macro expansion @ ~/.julia/packages/CorpusLoaders/RkzqR/test/test_IMDB.jl:18 [inlined] [2] top-level scope @ /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.6/Test/src/Test.jl:1226 [3] include(fname::String) @ Base.MainInclude ./client.jl:444 [4] macro expansion @ ~/.julia/packages/CorpusLoaders/RkzqR/test/runtests.jl:20 [inlined] [5] top-level scope @ /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.6/Test/src/Test.jl:1226 [6] include(fname::String) @ Base.MainInclude ./client.jl:444 [7] top-level scope @ none:6 [8] eval @ ./boot.jl:360 [inlined] [9] exec_options(opts::Base.JLOptions) @ Base ./client.jl:261 [10] _start() @ Base ./client.jl:485 basic use train_pos: Error During Test at /Users/anthony/.julia/packages/CorpusLoaders/RkzqR/test/test_IMDB.jl:19 Test threw exception Expression: sum(length.(sentences)) == length(words) MethodError: no method matching zero(::Type{Any}) Closest candidates are: zero(::Type{Union{Missing, T}}) where T at missing.jl:105 zero(::Union{Type{P}, P}) where P<:Dates.Period at /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.6/Dates/src/periods.jl:53 zero(::SparseArrays.AbstractSparseArray) at /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.6/SparseArrays/src/SparseArrays.jl:55 ... Stacktrace: [1] zero(#unused#::Type{Any}) @ Base ./missing.jl:106 [2] reduce_empty(#unused#::typeof(+), #unused#::Type{Any}) @ Base ./reduce.jl:311 [3] reduce_empty(#unused#::typeof(Base.add_sum), #unused#::Type{Any}) @ Base ./reduce.jl:320 [4] mapreduce_empty(#unused#::typeof(identity), op::Function, T::Type) @ Base ./reduce.jl:343 [5] reduce_empty(op::Base.MappingRF{typeof(identity), typeof(Base.add_sum)}, #unused#::Type{Any}) @ Base ./reduce.jl:329 [6] reduce_empty_iter @ ./reduce.jl:355 [inlined] [7] mapreduce_empty_iter(f::Function, op::Function, itr::Vector{Any}, ItrEltype::Base.HasEltype) @ Base ./reduce.jl:351 [8] _mapreduce(f::typeof(identity), op::typeof(Base.add_sum), #unused#::IndexLinear, A::Vector{Any}) @ Base ./reduce.jl:400 [9] _mapreduce_dim @ ./reducedim.jl:318 [inlined] [10] #mapreduce#672 @ ./reducedim.jl:310 [inlined] [11] mapreduce @ ./reducedim.jl:310 [inlined] [12] #_sum#682 @ ./reducedim.jl:878 [inlined] [13] _sum @ ./reducedim.jl:878 [inlined] [14] #_sum#681 @ ./reducedim.jl:877 [inlined] [15] _sum @ ./reducedim.jl:877 [inlined] [16] #sum#679 @ ./reducedim.jl:873 [inlined] [17] sum(a::Vector{Any}) @ Base ./reducedim.jl:873 [18] macro expansion @ ~/.julia/packages/CorpusLoaders/RkzqR/test/test_IMDB.jl:19 [inlined] [19] top-level scope @ /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.6/Test/src/Test.jl:1226 [20] include(fname::String) @ Base.MainInclude ./client.jl:444 [21] macro expansion @ ~/.julia/packages/CorpusLoaders/RkzqR/test/runtests.jl:20 [inlined] [22] top-level scope @ /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.6/Test/src/Test.jl:1226 [23] include(fname::String) @ Base.MainInclude ./client.jl:444 [24] top-level scope @ none:6 [25] eval @ ./boot.jl:360 [inlined] [26] exec_options(opts::Base.JLOptions) @ Base ./client.jl:261 [27] _start() @ Base ./client.jl:485 basic use train_pos: Test Failed at /Users/anthony/.julia/packages/CorpusLoaders/RkzqR/test/test_IMDB.jl:21 Expression: typeof(flatten_levels(docs, (!lvls)(IMDB, :documents)) |> full_consolidate) == Vector{String} Evaluated: Vector{Any} == Vector{String} Stacktrace: [1] macro expansion @ ~/.julia/packages/CorpusLoaders/RkzqR/test/test_IMDB.jl:21 [inlined] [2] top-level scope @ /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.6/Test/src/Test.jl:1226 [3] include(fname::String) @ Base.MainInclude ./client.jl:444 [4] macro expansion @ ~/.julia/packages/CorpusLoaders/RkzqR/test/runtests.jl:20 [inlined] [5] top-level scope @ /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.6/Test/src/Test.jl:1226 [6] include(fname::String) @ Base.MainInclude ./client.jl:444 [7] top-level scope @ none:6 [8] eval @ ./boot.jl:360 [inlined] [9] exec_options(opts::Base.JLOptions) @ Base ./client.jl:261 [10] _start() @ Base ./client.jl:485 basic use train_pos: Test Failed at /Users/anthony/.julia/packages/CorpusLoaders/RkzqR/test/test_IMDB.jl:22 Expression: typeof(flatten_levels(docs, (!lvls)(IMDB, :sentences)) |> full_consolidate) == Vector{String} Evaluated: Vector{Any} == Vector{String} Stacktrace: [1] macro expansion @ ~/.julia/packages/CorpusLoaders/RkzqR/test/test_IMDB.jl:22 [inlined] [2] top-level scope @ /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.6/Test/src/Test.jl:1226 [3] include(fname::String) @ Base.MainInclude ./client.jl:444 [4] macro expansion @ ~/.julia/packages/CorpusLoaders/RkzqR/test/runtests.jl:20 [inlined] [5] top-level scope @ /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.6/Test/src/Test.jl:1226 [6] include(fname::String) @ Base.MainInclude ./client.jl:444 [7] top-level scope @ none:6 [8] eval @ ./boot.jl:360 [inlined] [9] exec_options(opts::Base.JLOptions) @ Base ./client.jl:261 [10] _start() @ Base ./client.jl:485 Test Summary: | Pass Fail Error Total IMDB | 19 4 1 24 basic use train_pos | 1 4 1 6 basic use train_neg | 6 6 basic use test_pos | 6 6 basic use test_neg | 6 6 Test Summary: | Pass Fail Error Total IMDB | 19 4 1 24 basic use train_pos | 1 4 1 6 basic use train_neg | 6 6 basic use test_pos | 6 6 basic use test_neg | 6 6 ERROR: LoadError: Some tests did not pass: 19 passed, 4 failed, 1 errored, 0 broken. in expression starting at /Users/anthony/.julia/packages/CorpusLoaders/RkzqR/test/runtests.jl:19 caused by: Some tests did not pass: 19 passed, 4 failed, 1 errored, 0 broken. ERROR: Package CorpusLoaders errored during testing ``` 
(jl_4k2h6c) pkg> st
      Status `/private/var/folders/4n/gvbmlhdc8xj973001s6vdyw00000gq/T/jl_4k2h6c/Project.toml`
  [214a0ac2] CorpusLoaders v0.3.2

julia> versioninfo()
Julia Version 1.6.0
Commit f9720dc2eb (2021-03-24 12:55 UTC)
Platform Info:
  OS: macOS (x86_64-apple-darwin19.6.0)
  CPU: Intel(R) Core(TM) i7-8850H CPU @ 2.60GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-11.0.1 (ORCJIT, skylake)
Environment:
  JULIA_LTS_PATH = /Applications/Julia-1.0.app/Contents/Resources/julia/bin/julia
  JULIA_PATH = /Applications/Julia-1.6.app/Contents/Resources/julia/bin/julia
  JULIA_EGLOT_PATH = /Applications/Julia-1.5.app/Contents/Resources/julia/bin/julia
  JULIA_NUM_THREADS = 5
  JULIA_NIGHTLY_PATH = /Applications/Julia-1.7.app/Contents/Resources/julia/bin/julia
oxinabox commented 3 years ago

Weird. I wonder if it i a bug here, or in MultiResolutionIterators.jl.

Could it be that the IMDB data isn't loading an an empty collection can't infer?