bkamins
commented
3 years ago This is a PrettyTables.jl issue and it seems to be a bug that ideally should get resolved before 1.0 release of PrettyTables.jl.
Closed EconometricsBySimulation closed 3 years ago
bkamins
commented
3 years ago This is a PrettyTables.jl issue and it seems to be a bug that ideally should get resolved before 1.0 release of PrettyTables.jl.
bkamins
commented
3 years ago Also could you please check if in your data there are no special non-standard characters that potentially could mix up display? Thank you!
ronisbr
commented
3 years ago Hi @EconometricsBySimulation ,
Can you please describe better your scenario? What terminal, Julia version, DataFrames version and PrettyTables version are installed?
I just can't reproduce the error, and there is some strange things happening in you output. For example, look how <<NO TEXT FOUND FOR SPECIFIED DA… is correctly truncated whereas Lorem ipsum dolor sit amet, consectet... isn't.
I create a similar DataFrame using:
julia> using DataFrames
julia> using Dates
julia> dates = Date.(["2020-12-27", "2020-12-28", "2020-12-29", "2020-12-30"]);
julia> month = [12, 12, 12, 12];
julia> days = [27,28,29,30];
julia> text = ["<<NO TEXT FOUND FOR SPECIFIED DATA>>", "<<NO TEXT FOUND FOR SPECIFIED DATA>>", "Lorem ipsum dolor sit amet, consectetur adipiscing elit", "<<NO TEXT FOUND FOR SPECIFIED DATA>>"];
julia> df = DataFrame(date = dates, day = days, month = month, txt = text)
4×4 DataFrame
Row │ date day month txt
│ Date Int64 Int64 String
─────┼─────────────────────────────────────────────────────────────
1 │ 2020-12-27 27 12 <<NO TEXT FOUND FOR SPECIFIED DA…
2 │ 2020-12-28 28 12 <<NO TEXT FOUND FOR SPECIFIED DA…
3 │ 2020-12-29 29 12 Lorem ipsum dolor sit amet, cons…
4 │ 2020-12-30 30 12 <<NO TEXT FOUND FOR SPECIFIED DA…If I type Base.displaysize() = (10, 80) nothing should happen because we get the display size from displaysize(stdout) instead:
However, I can redefine Base.displaysize(::Base.TTY) = (10,80). In this case, it works perfectly here:
ronisbr
commented
3 years ago I couldn't reproduce it also in IJulia:
ronisbr
commented
3 years ago The closest I got from this bug was reported here:
https://github.com/ronisbr/PrettyTables.jl/issues/118
However, to reproduce it, we need to have a totally empty row ("" or nothing in all elements and "" in the column name). It also required to show the DataFrame without showing the element types (show(df, eltypes = false)). However, this does not seem to be happening here. Nevertheless, can you try with the latest PrettyTables.jl version (v0.12.1) that was just released?
EconometricsBySimulation
commented
3 years ago Thanks for all your feedback! I think special characters are very likely as this had been a problem before.
The loren ipsul text I substituted because the original text is sensitive.
I'm writing from my phone so I don't know all the exact version information but I am currently running julia 1.6 and Dataframes#main that was just advertised on Slack (Bogmil)
I just added #main to my project and I had no problem with the code previously (been running various versions of it for the last two years).
Hope this is helpful. I can look for specific special characters when I get back on my computer.
Francis
On Fri, Apr 16, 2021, 10:16 PM Ronan Arraes Jardim Chagas < @.***> wrote:
The closest I got from this bug was reported here:
ronisbr/PrettyTables.jl#118 https://github.com/ronisbr/PrettyTables.jl/issues/118
However, to reproduce it, we need to have a totally empty row ("" or nothing in all elements and "" in the column name). It also required to show the DataFrame without showing the element types (show(df, eltypes = false)). However, this does not seem to be happening here. Nevertheless, can you try with the latest PrettyTables.jl version (v0.12.1) that was just released?
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/JuliaData/DataFrames.jl/issues/2722#issuecomment-821750719, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABCYU44L4V66HY5CSPG2G3LTJDVOTANCNFSM43CC5L3A .
EconometricsBySimulation
commented
3 years ago Also I'll try with the latest pretty tables
On Sat, Apr 17, 2021, 12:44 AM Francis Smart @.***> wrote:
Thanks for all your feedback! I think special characters are very likely as this had been a problem before.
The loren ipsul text I substituted because the original text is sensitive.
I'm writing from my phone so I don't know all the exact version information but I am currently running julia 1.6 and Dataframes#main that was just advertised on Slack (Bogmil)
I just added #main to my project and I had no problem with the code previously (been running various versions of it for the last two years).
Hope this is helpful. I can look for specific special characters when I get back on my computer.
Francis
On Fri, Apr 16, 2021, 10:16 PM Ronan Arraes Jardim Chagas < @.***> wrote:
The closest I got from this bug was reported here:
ronisbr/PrettyTables.jl#118 https://github.com/ronisbr/PrettyTables.jl/issues/118
However, to reproduce it, we need to have a totally empty row ("" or nothing in all elements and "" in the column name). It also required to show the DataFrame without showing the element types (show(df, eltypes = false)). However, this does not seem to be happening here. Nevertheless, can you try with the latest PrettyTables.jl version (v0.12.1) that was just released?
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/JuliaData/DataFrames.jl/issues/2722#issuecomment-821750719, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABCYU44L4V66HY5CSPG2G3LTJDVOTANCNFSM43CC5L3A .
bkamins
commented
3 years ago Thank you for looking into this. Hopefully 0.12.1 release of PrettyTables.jl that @ronisbr just made resolves things.
ronisbr
commented
3 years ago Btw, are you showing the DataFrame by a custom show function? If so, can you share the code please? I would like to see if this can be cause by a combination of arguments.
EconometricsBySimulation
commented
3 years ago I'm not using any custom show function.
On Sat, Apr 17, 2021 at 3:18 PM Ronan Arraes Jardim Chagas < @.***> wrote:
Btw, are you showing the DataFrame by a custom show function? If so, can you share the code please?
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/JuliaData/DataFrames.jl/issues/2722#issuecomment-821873745, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABCYU45JQ5YKTZAVNQDUWPLTJHNHPANCNFSM43CC5L3A .
PhD Candidate- Michigan State University (563) 223-8108
ronisbr
commented
3 years ago OK thanks! Let's hope that the problem was occurring due to a DataFrame with empty columns, which were fixed with PrettyTables v0.12.1.
EconometricsBySimulation
commented
3 years ago Find a minimal scrambled csv that produces the error. reproducible error.zip
using Pkg
Pkg.activate(".")
using DataFrames, CSVActivating environment at ...\Project.toml
Pkg.status()Status ...\Project.toml
[336ed68f] CSV
[a93c6f00] DataFrames
Base.displaysize() = (10, 80)x = CSV.read("error.csv", DataFrame)
@show x
xx = 3×2 DataFrame Row │ txt2 month │ String Int64 ─────┼────────────────────────────────────────── 1 │ <<NO TEXT FOUND FOR SPECIFIED DA… 12 2 │ apoo od f n@N2enuo ttc aEon0t 0T… 12 3 │ <<NO TEXT FOUND FOR SPECIFIED DA… 12
ArgumentError: can't repeat a string -1 times
Stacktrace: [1] repeat(s::String, r::Int64) @ Base .\strings\substring.jl:226 [2] ^ @ .\strings\basic.jl:718 [inlined] [3] _crop_str(str::String, crop_size::Int64, lstr::Int64) @ PrettyTables ~.julia\packages\PrettyTables\9DHd9\src\backends\text\string.jl:54 [4] _str_aligned(data::String, alignment::Symbol, field_size::Int64, lstr::Int64) @ PrettyTables ~.julia\packages\PrettyTables\9DHd9\src\backends\text\string.jl:211 [5] _str_aligned @ ~.julia\packages\PrettyTables\9DHd9\src\backends\text\string.jl:206 [inlined] [6] _draw_continuation_row(display::PrettyTables.Display, io::IOContext{IOBuffer}, tf::PrettyTables.TextFormat, text_crayon::Crayons.Crayon, border_crayon::Crayons.Crayon, cols_width::Vector{Int64}, vlines::Vector{Int64}, alignment::Symbol) @ PrettyTables ~.julia\packages\PrettyTables\9DHd9\src\backends\text\display.jl:34 [7] _print_table_data(buf::IOContext{IOBuffer}, display::PrettyTables.Display, data::PrettyTables.ColumnTable, data_str::Matrix{Vector{String}}, id_cols::Vector{Int64}, id_rows::Vector{Int64}, Δc::Int64, cols_width::Vector{Int64}, vlines::Vector{Int64}, row_printing_recipe::Vector{Union{NTuple{4, Int64}, Symbol}}, col_printing_recipe::Vector{Union{Int64, Symbol}}, alignment::Vector{Symbol}, body_hlines_format::NTuple{4, Char}, cell_alignment_override::Dict{Tuple{Int64, Int64}, Symbol}, continuation_row_alignment::Symbol, ellipsis_line_skip::Int64, highlighters::Base.RefValue{Any}, noheader::Bool, show_row_names::Bool, show_row_number::Bool, tf::PrettyTables.TextFormat, border_crayon::Crayons.Crayon, row_name_crayon::Crayons.Crayon, text_crayon::Crayons.Crayon) @ PrettyTables ~.julia\packages\PrettyTables\9DHd9\src\backends\text\print_aux.jl:187 [8] _pt_text(r_io::Base.RefValue{Any}, pinfo::PrettyTables.PrintInfo; border_crayon::Crayons.Crayon, header_crayon::Crayons.Crayon, subheader_crayon::Crayons.Crayon, rownum_header_crayon::Crayons.Crayon, text_crayon::Crayons.Crayon, omitted_cell_summary_crayon::Crayons.Crayon, alignment_anchor_fallback::Symbol, alignment_anchor_fallback_override::Dict{Int64, Symbol}, alignment_anchor_regex::Dict{Int64, Vector{Regex}}, autowrap::Bool, body_hlines::Vector{Int64}, body_hlines_format::Nothing, continuation_row_alignment::Symbol, crop::Symbol, crop_subheader::Bool, crop_num_lines_at_beginning::Int64, columns_width::Int64, display_size::Tuple{Int64, Int64}, equal_columns_width::Bool, ellipsis_line_skip::Int64, highlighters::Tuple{PrettyTables.Highlighter}, hlines::Vector{Symbol}, linebreaks::Bool, maximum_columns_width::Vector{Int64}, minimum_columns_width::Int64, newline_at_end::Bool, overwrite::Bool, noheader::Bool, nosubheader::Bool, row_name_crayon::Crayons.Crayon, row_name_header_crayon::Crayons.Crayon, row_number_alignment::Symbol, show_omitted_cell_summary::Bool, sortkeys::Bool, tf::PrettyTables.TextFormat, title_autowrap::Bool, title_crayon::Crayons.Crayon, title_same_width_as_table::Bool, vcrop_mode::Symbol, vlines::Vector{Int64}) @ PrettyTables ~.julia\packages\PrettyTables\9DHd9\src\backends\text\print.jl:477 [9] #_pt#74 @ ~.julia\packages\PrettyTables\9DHd9\src\private.jl:425 [inlined] [10] _pretty_table(io::IO, data::DataFrame; header::Tuple{Vector{String}, Vector{String}}, kwargs::Base.Iterators.Pairs{Symbol, Any, NTuple{24, Symbol}, NamedTuple{(:alignment, :alignment_anchor_fallback, :alignment_anchor_regex, :compact_printing, :crop, :crop_num_lines_at_beginning, :ellipsis_line_skip, :formatters, :header_alignment, :hlines, :highlighters, :maximum_columns_width, :newline_at_end, :nosubheader, :row_name_alignment, :row_name_crayon, :row_name_column_title, :row_names, :row_number_alignment, :row_number_column_title, :show_row_number, :title, :vcrop_mode, :vlines), Tuple{Vector{Symbol}, Symbol, Dict{Int64, Vector{Regex}}, Bool, Symbol, Int64, Int64, Tuple{typeof(DataFrames._pretty_tables_general_formatter)}, Symbol, Vector{Symbol}, Tuple{PrettyTables.Highlighter}, Vector{Int64}, Bool, Bool, Symbol, Crayons.Crayon, String, Nothing, Symbol, String, Bool, String, Symbol, Vector{Int64}}}}) @ PrettyTables ~.julia\packages\PrettyTables\9DHd9\src\private.jl:332 [11] #pretty_table#63 @ ~.julia\packages\PrettyTables\9DHd9\src\print.jl:719 [inlined] [12] _show(io::IOContext{IOBuffer}, df::DataFrame; allrows::Bool, allcols::Bool, rowlabel::Symbol, summary::Bool, eltypes::Bool, rowid::Nothing, truncate::Int64, kwargs::Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}) @ DataFrames ~.julia\packages\DataFrames\9luaM\src\abstractdataframe\show.jl:304 [13] #show#696 @ ~.julia\packages\DataFrames\9luaM\src\abstractdataframe\show.jl:398 [inlined] [14] show(io::IOContext{IOBuffer}, df::DataFrame) @ DataFrames ~.julia\packages\DataFrames\9luaM\src\abstractdataframe\show.jl:398 [15] #show#711 @ ~.julia\packages\DataFrames\9luaM\src\abstractdataframe\io.jl:50 [inlined] [16] show @ ~.julia\packages\DataFrames\9luaM\src\abstractdataframe\io.jl:50 [inlined] [17] limitstringmime(mime::MIME{Symbol("text/plain")}, x::DataFrame) @ IJulia ~.julia\packages\IJulia\e8kqU\src\inline.jl:43 [18] display_mimestring @ ~.julia\packages\IJulia\e8kqU\src\display.jl:71 [inlined] [19] display_dict(x::DataFrame) @ IJulia ~.julia\packages\IJulia\e8kqU\src\display.jl:102 [20] #invokelatest#2 @ .\essentials.jl:708 [inlined] [21] invokelatest @ .\essentials.jl:706 [inlined] [22] execute_request(socket::ZMQ.Socket, msg::IJulia.Msg) @ IJulia ~.julia\packages\IJulia\e8kqU\src\execute_request.jl:112 [23] #invokelatest#2 @ .\essentials.jl:708 [inlined] [24] invokelatest @ .\essentials.jl:706 [inlined] [25] eventloop(socket::ZMQ.Socket) @ IJulia ~.julia\packages\IJulia\e8kqU\src\eventloop.jl:8 [26] (::IJulia.var"#15#18")() @ IJulia .\task.jl:406
@EconometricsBySimulation can you please send the .zip again or put in a online server? It seems to be corrupted:
╰─ curl -O https://github.com/JuliaData/DataFrames.jl/files/6337529/reproducible.error.zip
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 606 100 606 0 0 404 0 0:00:01 0:00:01 --:--:-- 404
╭─ ~/tmp 1.21 2.31G 45% 13:31:46
╰─ unzip reproducible.error.zip
Archive: reproducible.error.zip
End-of-central-directory signature not found. Either this file is not
a zipfile, or it constitutes one disk of a multi-part archive. In the
latter case the central directory and zipfile comment will be found on
the last disk(s) of this archive.
unzip: cannot find zipfile directory in one of reproducible.error.zip or
reproducible.error.zip.zip, and cannot find reproducible.error.zip.ZIP, period.I think the file is encrypted, it is asking password. I am not seeing the new link.
EconometricsBySimulation
commented
3 years ago Just in case you need it again. error.zip
EconometricsBySimulation
commented
3 years ago Sorry, work computer wants to encrypt everything. Tell me if that works.
ronisbr
commented
3 years ago Yes! Now I can see the files :) thanks!
EconometricsBySimulation
commented
3 years ago bkamins
commented
3 years ago I could reproduce it using your Manifest.toml, but under PrettyTables.jl 0.12.1 it works for me. Can you check that if you update PrettyTables.jl on your side also everything works?
Thank you!
ronisbr
commented
3 years ago I could reproduce it using your Manifest.toml, but under PrettyTables.jl 0.12.1 it works for me. Can you check that if you update PrettyTables.jl on your side also everything works?
That's precisely what I was about to write! I could reproduce the bug using that Manifest.toml, but I couldn't with latest DataFrames and PrettyTables v0.12.1.
EconometricsBySimulation
commented
3 years ago Not sure what I need to do?
EconometricsBySimulation
commented
3 years ago I added PrettyTables then ran input [22] again. Bogumil do you need my manefest.toml and project.toml?
ronisbr
commented
3 years ago Please, check in your Manifest.toml if you have PrettyTables.jl v0.12.1 or v0.12.0.
EconometricsBySimulation
commented
3 years ago [[PrettyTables]] deps = ["Crayons", "Formatting", "Markdown", "Reexport", "Tables"] git-tree-sha1 = "b46905b7def282b72c2a3a5b385947a01835e10a" uuid = "08abe8d2-0d0c-5749-adfa-8a2ac140af0d" version = "0.12.1"
On Mon, Apr 19, 2021 at 1:14 PM Ronan Arraes Jardim Chagas < @.***> wrote:
Please, check in your Manifest.toml if you have PrettyTables.jl v0.12.1 or v0.12.0.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/JuliaData/DataFrames.jl/issues/2722#issuecomment-822635614, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABCYU42WV4ABJIW27W4XFP3TJRQIDANCNFSM43CC5L3A .
PhD Candidate- Michigan State University (563) 223-8108
ronisbr
commented
3 years ago Did you restarted IJulia?
EconometricsBySimulation
commented
3 years ago Will do
On Mon, Apr 19, 2021 at 1:17 PM Ronan Arraes Jardim Chagas < @.***> wrote:
Did you restarted IJulia?
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/JuliaData/DataFrames.jl/issues/2722#issuecomment-822637640, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABCYU44QLA5EXM4RMPL26MDTJRQUDANCNFSM43CC5L3A .
PhD Candidate- Michigan State University (563) 223-8108
EconometricsBySimulation
commented
3 years ago That did it! Thank you
ronisbr
commented
3 years ago Side note: I have no idea why this was failing inside the text backend and, when fixed, it started to show an HTML table...
EconometricsBySimulation
commented
3 years ago It was very idiosyncratic.
On Mon, Apr 19, 2021 at 1:28 PM Ronan Arraes Jardim Chagas < @.***> wrote:
Side note: I have no idea why this was failing inside the text backend and, when fixed, it started to show an HTML table...
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/JuliaData/DataFrames.jl/issues/2722#issuecomment-822644361, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABCYU4ZHQFRVHKNG3G5Q5LDTJRR4ZANCNFSM43CC5L3A .
PhD Candidate- Michigan State University (563) 223-8108
bkamins
commented
3 years ago I have no idea why this was failing inside the text backend and, when fixed, it started to show an HTML table
This might be some IJulia.jl mechanics for detecting the supported output type. Maybe IJulia.jl is trying to print some dummy thing to check?
bkamins
commented
3 years ago OK. I am closing this as I understand the issue is resolved. Thank you all!
After setting
Base.displaysize() = (10, 80)And defining a way too complex function that takes a string and other arguments and turns them into a DataFrame
Within the function the above table is printed (for debugging purposes) before being returned. Upon being returned it produces the following error. I shouldn't share the code as it might have some sensitive data inserted for testing purposes.
Setting
Base.displaysize() = (12, 80)completely removes the error.I afraid I was not able to recreate the error using generated data.