search

queryverse / Query.jl

Query almost anything in julia
Other
394 stars 50 forks source link

Type too large issue #90

Closed tcovert closed 7 years ago

tcovert commented 7 years ago

I'm trying to migrate some code away from the DataFrames/DataFramesMeta paradigm over to DataTables and query, but the very first query I tried did not work:

using DataTables, Query
lt = readtable("/path/to/file.csv")
@from i in lt begin
@select {i.FileNo, i.LogsOnFile}
@collect DataTable
end

This generates one error the first time I tried it:

ERROR: type too large
 in create_tuple(::Array{Symbol,1}) at /Users/tcovert/.julia/v0.5/NamedTuples/src/NamedTuples.jl:105
 in make_tuple(::Array{Expr,1}) at /Users/tcovert/.julia/v0.5/NamedTuples/src/NamedTuples.jl:145
 in query(::DataTables.DataTable) at /Users/tcovert/.julia/v0.5/Query/src/sources/source_datatable.jl:21

and the second time I got something different:

ERROR: MethodError: no method matching NamedTuples._NT_APIFileNoOriginalOperatorOriginalWellNameCurrentOperatorCurrentWellNameFieldNameCountyNameTownshipRangeSectionQQFootagesHole_TypeLATITUDELONGITUDEFirstSpudDateMeasuredTDKBElevDFElevGRElevGLElevTY_BCHRTY_BCHNK_PK_PJRSK_PESK_NBK_GHK_MK_NK_IKJ_SJ_RT_SPM_MKPM_OPPM_EBAPM_BCPN_TM_EBSM_BSM_KLM_MDUNM_MDM_MDRM_MDLSM_MDFAM_MDFYM_MDSAM_MDTIM_MDLPMD_BD_DVD_TFD_BBD_DPD_SRD_DBD_PED_WS_IS_CLO_GO_STO_RRO_WRO_WIO_BICO_DPCLogsOnFile{Nullable{Int64},Nullable{Int64},Nullable{String},Nullable{String},Nullable{String},Nullable{String},Nullable{String},Nullable{String},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{String},Nullable{String},Nullable{String},Nullable{Float64},Nullable{Float64},Nullable{String},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{String}}(::Nullable{Int64}, ::Nullable{Int64}, ::Nullable{String}, ::Nullable{String}, ::Nullable{String}, ::Nullable{String}, ::Nullable{String}, ::Nullable{String}, ::Nullable{Int64}, ::Nullable{Int64}, ::Nullable{Int64}, ::Nullable{String}, ::Nullable{String}, ::Nullable{String}, ::Nullable{Float64}, ::Nullable{Float64}, ::Nullable{String}, ::Nullable{Int64}, ::Nullable{Int64}, ::Nullable{Int64}, ::Nullable{Int64}, ::Nullable{Int64}, ::Nullable{Int64}, ::Nullable{Int64}, ::Nullable{Int64}, ::Nullable{Int64}, ::Nullable{Int64}, ::Nullable{Int64}, ::Nullable{Int64}, ::Nullable{Int64}, ::Nullable{Int64}, ::Nullable{Int64}, ::Nullable{Int64}, ::Nullable{Int64}, ::Nullable{Int64}, ::Nullable{Int64}, ::Nullable{Int64}, ::Nullable{Int64}, ::Nullable{Int64}, ::Nullable{Int64}, ::Nullable{Int64}, ::Nullable{Int64}, ::Nullable{Int64}, ::Nullable{Int64}, ::Nullable{Int64}, ::Nullable{Int64}, ::Nullable{Int64}, ::Nullable{Int64}, ::Nullable{Int64}, ::Nullable{Int64}, ::Nullable{Int64}, ::Nullable{Int64}, ::Nullable{Int64}, ::Nullable{Int64}, ::Nullable{Int64}, ::Nullable{Int64}, ::Nullable{Int64}, ::Nullable{Int64}, ::Nullable{Int64}, ::Nullable{Int64}, ::Nullable{Int64}, ::Nullable{Int64}, ::Nullable{Int64}, ::Nullable{Int64}, ::Nullable{Int64}, ::Nullable{Int64}, ::Nullable{Int64}, ::Nullable{Int64}, ::Nullable{Int64}, ::Nullable{Int64}, ::Nullable{Int64}, ::Nullable{String})
Closest candidates are:
  NamedTuples._NT_APIFileNoOriginalOperatorOriginalWellNameCurrentOperatorCurrentWellNameFieldNameCountyNameTownshipRangeSectionQQFootagesHole_TypeLATITUDELONGITUDEFirstSpudDateMeasuredTDKBElevDFElevGRElevGLElevTY_BCHRTY_BCHNK_PK_PJRSK_PESK_NBK_GHK_MK_NK_IKJ_SJ_RT_SPM_MKPM_OPPM_EBAPM_BCPN_TM_EBSM_BSM_KLM_MDUNM_MDM_MDRM_MDLSM_MDFAM_MDFYM_MDSAM_MDTIM_MDLPMD_BD_DVD_TFD_BBD_DPD_SRD_DBD_PED_WS_IS_CLO_GO_STO_RRO_WRO_WIO_BICO_DPCLogsOnFile{Nullable{Int64},Nullable{Int64},Nullable{String},Nullable{String},Nullable{String},Nullable{String},Nullable{String},Nullable{String},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{String},Nullable{String},Nullable{String},Nullable{Float64},Nullable{Float64},Nullable{String},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{String}}{T}(::Any) at sysimg.jl:53
 in macro expansion at /Users/tcovert/.julia/v0.5/Query/src/sources/source_datatable.jl:56 [inlined]
 in next(::Query.EnumerableDataTable{NamedTuples._NT_APIFileNoOriginalOperatorOriginalWellNameCurrentOperatorCurrentWellNameFieldNameCountyNameTownshipRangeSectionQQFootagesHole_TypeLATITUDELONGITUDEFirstSpudDateMeasuredTDKBElevDFElevGRElevGLElevTY_BCHRTY_BCHNK_PK_PJRSK_PESK_NBK_GHK_MK_NK_IKJ_SJ_RT_SPM_MKPM_OPPM_EBAPM_BCPN_TM_EBSM_BSM_KLM_MDUNM_MDM_MDRM_MDLSM_MDFAM_MDFYM_MDSAM_MDTIM_MDLPMD_BD_DVD_TFD_BBD_DPD_SRD_DBD_PED_WS_IS_CLO_GO_STO_RRO_WRO_WIO_BICO_DPCLogsOnFile{Nullable{Int64},Nullable{Int64},Nullable{String},Nullable{String},Nullable{String},Nullable{String},Nullable{String},Nullable{String},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{String},Nullable{String},Nullable{String},Nullable{Float64},Nullable{Float64},Nullable{String},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{String}},Tuple{NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{String,1},NullableArrays.NullableArray{String,1},NullableArrays.NullableArray{String,1},NullableArrays.NullableArray{String,1},NullableArrays.NullableArray{String,1},NullableArrays.NullableArray{String,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{String,1},NullableArrays.NullableArray{String,1},NullableArrays.NullableArray{String,1},NullableArrays.NullableArray{Float64,1},NullableArrays.NullableArray{Float64,1},NullableArrays.NullableArray{String,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{String,1}}}, ::Int64) at /Users/tcovert/.julia/v0.5/Query/src/sources/source_datatable.jl:48
 in next at /Users/tcovert/.julia/v0.5/Query/src/enumerable/enumerable_select.jl:28 [inlined]
 in macro expansion at /Users/tcovert/.julia/v0.5/Query/src/sinks/sink_datatable.jl:13 [inlined]
 in _filldt(::Tuple{NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{String,1}}, ::Query.EnumerableSelect{NamedTuples._NT_FileNoLogsOnFile{Nullable{Int64},Nullable{String}},Query.EnumerableDataTable{NamedTuples._NT_APIFileNoOriginalOperatorOriginalWellNameCurrentOperatorCurrentWellNameFieldNameCountyNameTownshipRangeSectionQQFootagesHole_TypeLATITUDELONGITUDEFirstSpudDateMeasuredTDKBElevDFElevGRElevGLElevTY_BCHRTY_BCHNK_PK_PJRSK_PESK_NBK_GHK_MK_NK_IKJ_SJ_RT_SPM_MKPM_OPPM_EBAPM_BCPN_TM_EBSM_BSM_KLM_MDUNM_MDM_MDRM_MDLSM_MDFAM_MDFYM_MDSAM_MDTIM_MDLPMD_BD_DVD_TFD_BBD_DPD_SRD_DBD_PED_WS_IS_CLO_GO_STO_RRO_WRO_WIO_BICO_DPCLogsOnFile{Nullable{Int64},Nullable{Int64},Nullable{String},Nullable{String},Nullable{String},Nullable{String},Nullable{String},Nullable{String},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{String},Nullable{String},Nullable{String},Nullable{Float64},Nullable{Float64},Nullable{String},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{String}},Tuple{NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{String,1},NullableArrays.NullableArray{String,1},NullableArrays.NullableArray{String,1},NullableArrays.NullableArray{String,1},NullableArrays.NullableArray{String,1},NullableArrays.NullableArray{String,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{String,1},NullableArrays.NullableArray{String,1},NullableArrays.NullableArray{String,1},NullableArrays.NullableArray{Float64,1},NullableArrays.NullableArray{Float64,1},NullableArrays.NullableArray{String,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{String,1}}},FunctionWrappers.FunctionWrapper{NamedTuples._NT_FileNoLogsOnFile{Nullable{Int64},Nullable{String}},Tuple{NamedTuples._NT_APIFileNoOriginalOperatorOriginalWellNameCurrentOperatorCurrentWellNameFieldNameCountyNameTownshipRangeSectionQQFootagesHole_TypeLATITUDELONGITUDEFirstSpudDateMeasuredTDKBElevDFElevGRElevGLElevTY_BCHRTY_BCHNK_PK_PJRSK_PESK_NBK_GHK_MK_NK_IKJ_SJ_RT_SPM_MKPM_OPPM_EBAPM_BCPN_TM_EBSM_BSM_KLM_MDUNM_MDM_MDRM_MDLSM_MDFAM_MDFYM_MDSAM_MDTIM_MDLPMD_BD_DVD_TFD_BBD_DPD_SRD_DBD_PED_WS_IS_CLO_GO_STO_RRO_WRO_WIO_BICO_DPCLogsOnFile{Nullable{Int64},Nullable{Int64},Nullable{String},Nullable{String},Nullable{String},Nullable{String},Nullable{String},Nullable{String},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{String},Nullable{String},Nullable{String},Nullable{Float64},Nullable{Float64},Nullable{String},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{String}}}}}) at /Users/tcovert/.julia/v0.5/Query/src/sinks/sink_datatable.jl:5
 in collect(::Query.EnumerableSelect{NamedTuples._NT_FileNoLogsOnFile{Nullable{Int64},Nullable{String}},Query.EnumerableDataTable{NamedTuples._NT_APIFileNoOriginalOperatorOriginalWellNameCurrentOperatorCurrentWellNameFieldNameCountyNameTownshipRangeSectionQQFootagesHole_TypeLATITUDELONGITUDEFirstSpudDateMeasuredTDKBElevDFElevGRElevGLElevTY_BCHRTY_BCHNK_PK_PJRSK_PESK_NBK_GHK_MK_NK_IKJ_SJ_RT_SPM_MKPM_OPPM_EBAPM_BCPN_TM_EBSM_BSM_KLM_MDUNM_MDM_MDRM_MDLSM_MDFAM_MDFYM_MDSAM_MDTIM_MDLPMD_BD_DVD_TFD_BBD_DPD_SRD_DBD_PED_WS_IS_CLO_GO_STO_RRO_WRO_WIO_BICO_DPCLogsOnFile{Nullable{Int64},Nullable{Int64},Nullable{String},Nullable{String},Nullable{String},Nullable{String},Nullable{String},Nullable{String},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{String},Nullable{String},Nullable{String},Nullable{Float64},Nullable{Float64},Nullable{String},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{String}},Tuple{NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{String,1},NullableArrays.NullableArray{String,1},NullableArrays.NullableArray{String,1},NullableArrays.NullableArray{String,1},NullableArrays.NullableArray{String,1},NullableArrays.NullableArray{String,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{String,1},NullableArrays.NullableArray{String,1},NullableArrays.NullableArray{String,1},NullableArrays.NullableArray{Float64,1},NullableArrays.NullableArray{Float64,1},NullableArrays.NullableArray{String,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{String,1}}},FunctionWrappers.FunctionWrapper{NamedTuples._NT_FileNoLogsOnFile{Nullable{Int64},Nullable{String}},Tuple{NamedTuples._NT_APIFileNoOriginalOperatorOriginalWellNameCurrentOperatorCurrentWellNameFieldNameCountyNameTownshipRangeSectionQQFootagesHole_TypeLATITUDELONGITUDEFirstSpudDateMeasuredTDKBElevDFElevGRElevGLElevTY_BCHRTY_BCHNK_PK_PJRSK_PESK_NBK_GHK_MK_NK_IKJ_SJ_RT_SPM_MKPM_OPPM_EBAPM_BCPN_TM_EBSM_BSM_KLM_MDUNM_MDM_MDRM_MDLSM_MDFAM_MDFYM_MDSAM_MDTIM_MDLPMD_BD_DVD_TFD_BBD_DPD_SRD_DBD_PED_WS_IS_CLO_GO_STO_RRO_WRO_WIO_BICO_DPCLogsOnFile{Nullable{Int64},Nullable{Int64},Nullable{String},Nullable{String},Nullable{String},Nullable{String},Nullable{String},Nullable{String},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{String},Nullable{String},Nullable{String},Nullable{Float64},Nullable{Float64},Nullable{String},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{Int64},Nullable{String}}}}}, ::Type{DataTables.DataTable}) at /Users/tcovert/.julia/v0.5/Query/src/sinks/sink_datatable.jl:31

Any idea what I am doing wrong? The file I am trying to query as a DataTable is available in CSV format below. Also, for what its worth, using a DataFrame + Query also fails in the same way (i.e., Type too large error and then the longer error next).

https://www.dropbox.com/s/wipoy72fmbh5638/logtops_20160811.csv?dl=0

tcovert commented 7 years ago

I just updated my (master) version of Query.jl and now I get a different error with this query:

julia> @from i in lt begin
       @select {i.FileNo, i.LogsOnFile}
       @collect DataTable
       end
ERROR: Can only collect a NamedTuple iterator into a DataFrame
 in collect(::Query.EnumerableSelect{Any,Query.EnumerableDataTable{NamedTuples._NT_API_FileNo_OriginalOperator_OriginalWellName_CurrentOperator_CurrentWellName_FieldName_CountyName_Township_Range_Section_QQ_Footages_Hole__Type_LATITUDE_LONGITUDE_FirstSpudDate_MeasuredTD_KBElev_DFElev_GRElev_GLElev_TY__BCHR_TY__BCHN_K__P_K__PJRS_K__PES_K__NB_K__GH_K__M_K__N_K__IK_J__S_J__R_T__S_PM__MK_PM__OP_PM__EBA_PM__BC_PN__T_M__EBS_M__BS_M__KL_M__MDUN_M__MD_M__MDR_M__MDLS_M__MDFA_M__MDFY_M__MDSA_M__MDTI_M__MDLP_MD__B_D__DV_D__TF_D__BB_D__DP_D__SR_D__DB_D__PE_D__W_S__I_S__CL_O__G_O__ST_O__RR_O__WR_O__WI_O__BI_CO__D_PC_LogsOnFile{Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{String},Query.DataValue{String},Query.DataValue{String},Query.DataValue{String},Query.DataValue{String},Query.DataValue{String},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{String},Query.DataValue{String},Query.DataValue{String},Query.DataValue{Float64},Query.DataValue{Float64},Query.DataValue{String},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{String}},Tuple{NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{String,1},NullableArrays.NullableArray{String,1},NullableArrays.NullableArray{String,1},NullableArrays.NullableArray{String,1},NullableArrays.NullableArray{String,1},NullableArrays.NullableArray{String,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{String,1},NullableArrays.NullableArray{String,1},NullableArrays.NullableArray{String,1},NullableArrays.NullableArray{Float64,1},NullableArrays.NullableArray{Float64,1},NullableArrays.NullableArray{String,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{Int64,1},NullableArrays.NullableArray{String,1}}},FunctionWrappers.FunctionWrapper{Any,Tuple{NamedTuples._NT_API_FileNo_OriginalOperator_OriginalWellName_CurrentOperator_CurrentWellName_FieldName_CountyName_Township_Range_Section_QQ_Footages_Hole__Type_LATITUDE_LONGITUDE_FirstSpudDate_MeasuredTD_KBElev_DFElev_GRElev_GLElev_TY__BCHR_TY__BCHN_K__P_K__PJRS_K__PES_K__NB_K__GH_K__M_K__N_K__IK_J__S_J__R_T__S_PM__MK_PM__OP_PM__EBA_PM__BC_PN__T_M__EBS_M__BS_M__KL_M__MDUN_M__MD_M__MDR_M__MDLS_M__MDFA_M__MDFY_M__MDSA_M__MDTI_M__MDLP_MD__B_D__DV_D__TF_D__BB_D__DP_D__SR_D__DB_D__PE_D__W_S__I_S__CL_O__G_O__ST_O__RR_O__WR_O__WI_O__BI_CO__D_PC_LogsOnFile{Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{String},Query.DataValue{String},Query.DataValue{String},Query.DataValue{String},Query.DataValue{String},Query.DataValue{String},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{String},Query.DataValue{String},Query.DataValue{String},Query.DataValue{Float64},Query.DataValue{Float64},Query.DataValue{String},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{Int64},Query.DataValue{String}}}}}, ::Type{DataTables.DataTable}) at /Users/tcovert/.julia/v0.5/Query/src/sinks/sink_datatable.jl:26
IlyaOrson commented 7 years ago

I get a similar error with a 5822×87 DataFrames.DataFrame table and the subsequent errors also.

------ ErrorException --------------````---- Stacktrace (most recent call last)

 [1] — query(::DataFrames.DataFrame) at source_dataframe.jl:25

 [2] — make_tuple(::Array{Expr,1}) at NamedTuples.jl:145

 [3] — create_tuple(::Array{Symbol,1}) at NamedTuples.jl:105

type too large
davidanthoff commented 7 years ago

I believe the errors you saw on master are unrelated. The NamedTuples released an incompatible update and I only updated the REQUIRE file here a few hours ago. I think if you were to use the current master or the latest tagged version you should get the original error again.

I'll try to look into that soon, but have a lot of other things on my plate right now, so no promise, I'm afraid.

davidanthoff commented 7 years ago

Tracking this in blackrock/NamedTuples.jl#25 for now.

davidanthoff commented 7 years ago

Alright, this error goes away on julia 0.6. I know that Query doesn't work on julia 0.6 yet, but unfortunately that is the best I can offer at this point.