would like to try this out in nushell

[fdncred]

hey @zhiburt, i'd love to try your tabled crate out in nushell just to see how it stacks up and if it could work for us, what the performance is like, how it handles emojis, etc. are you interested and available to help with this?

if so, i'd see it initially working this way as a prototype. right now most commands automatically call table which creates the normal nushell tables. i figured we could create a new command called tabled and then just do ls | tabled and blah | tabled just to see how it works.

what are your thoughts?

[zhiburt]

Hi @fdncred

what are your thoughts?

I was actually thinking about creating a binary of this kind a bunch of times. Though I haven't came with the format which must be required by stdin. What approach do you use in nushell?

I'd love to try your tabled crate out in nushell just to see how it stacks up and if it could work for us, what the performance is like, how it handles emojis, etc. are you interested and available to help with this?

Sure, Personally the performance would be my biggest concern here. I want to believe the performance is OK; but who knows... It would be interesting to see how things are really going. *We do a bunch of allocations on each `fmt::Display which ideally should be eliminated; (maybe #123 can help)

Then do you propose me to create a prototype?

[fdncred]

Then do you propose me to create a prototype?

Yes, exactly. :) Feel free to say, "No!".

What I was thinking and hoping for was to create a tabled internal command inside of nushell just as there is a table internal command inside of nushell. I wouldn't want to use an external command because I don't think it would work right.

My biggest concern is that we truncate columns with ... sometimes and we also wrap text when there isn't enough space. I'm not sure if tabled supports either of those yet. But it still may be worth a try to look at any internal command that uses the input variable and see if a tabled command could take that input, figure out the column names and create rows, just as a prototype.

[zhiburt]

My biggest concern is that we truncate columns with ... sometimes and we also wrap text when there isn't enough space. I'm not sure if tabled supports either of those yet.

Must be supported.

Better look at my concern :smiling_face_with_tear: :disappointed_relieved:

---

Seems like a lot of allocations.

[zhiburt]

A bit Improved allocations, but it still high.

[fdncred]

nice

[zhiburt]

(surprisingly) Getting closer.

[fdncred]

how exactly are you generating these cool graphs?

[zhiburt]

Well it's not me. All the credit to the https://docs.rs/criterion/latest/criterion/ (apparently it uses gnuplot?)

Also see #139

I've put there a few more

[zhiburt]

Hi @fdncred

I've created a small patchwork here https://github.com/zhiburt/nushell/tree/checkout-tabled. You can check this out.

I did not made any refactoring and stuff like this;

As I see there's a difference in logic of alignment to terminal width. Current logic I guess is not applicable for nu_shell. So it may deserve a change.

[fdncred]

oh, nice. I'm shocked at how few changes it took to hook up tabled to nushell. nice job.

I noticed that it doesn't have the footers that nushell has. I'm running it now. it's pretty fun.

some of the coloring is different too (the lines). maybe tabled doesn't support that yet.

numbers aren't left-aligned but I'm pretty sure tabled supports that. nushell tables

tabled tables

[fdncred]

This alignment/wrapping is interesting too, like you already mentioned.

[fdncred]

and oh, I finally understand your second screenshot. you made the terminal smaller and wow that makes columns kind of disappear. very interesting.

it's still cool though, you're doing a great job with tabled. Congrats! I wonder if we could/should try to make tabled work more like nushell with alignment and wrapping?

[zhiburt]

I noticed that it doesn't have the footers that nushell has

We could do it by pushin headers to the data. Though to mimic the nu_table theme we would need create a custom ones to consider this last rows.

Am I right that nushell just prints headers and the bottom and the top?

ref: #126

some of the coloring is different too (the lines). maybe tabled doesn't support that yet.

Quite interesting. Could you point out where they're different?

numbers aren't left-aligned but I'm pretty sure tabled supports that.

I did a right alignment

and oh, I finally understand your second screenshot. you made the terminal smaller and wow that makes columns kind of disappear. very interesting.

Correct

---

By the way I've just noticed that on one of your screenshots table has colored borders. We are considered to support that but, How do you do that?

I mean as I understand this theme is not in a default list.

[zhiburt]

I wonder if we could/should try to make tabled work more like nushell with alignment and wrapping?

Couldn't we do both? If it being exposed by config or in the args to table.

By certainly the default must be decided.

---

After some work; (not yet finished).

I feel like this is similar to your approach right?

[fdncred]

Am I right that nushell just prints headers and the bottom and the top?

ya, but there's a config setting in nushell that controls when to print it. footer_mode: always #always, never, number_of_rows, auto

Could you point out where they're different?

in the two screenshots of the regular ls i have the lines/borders configured to print in purple on the nushell tables but in the tabled tables they print in white.

I'm using a custom theme to make the borders purple and change some other colors. I have this in my config.nu.

let my_theme = {
    separator: yd
    # leading_trailing_space_bg: red
    leading_trailing_space_bg: { attr: n}
    header: cb
    date: pu
    filesize: ub
    row_index: yb
    hints: dark_gray
    bool: red
    int: green
    duration: blue_bold
    range: purple
    float: red
    #string: red
    nothing: red
    binary: red
    cellpath: red

    # just take defaults for shapes
}

and then later on in the let-env config = { section I do

    color_config: $my_theme

[fdncred]

Couldn't we do both?

we could probably have a config setting to use tabled versus nushell tables. I'm just wondering if tabled needs to support column truncation where we show ... as a column and the other columns are just truncated?

I feel like this is similar to your approach right?

I think it's looking better. size isn't right aligned.

It would be nice to have a side-by-side nushell table vs tabled table - it's hard to spot differences without comparing, but it's looking good!

[fdncred]

oh, i almost forgot, we have this option too on nushell tables to turn off the first column.

    disable_table_indexes: true # set to true to remove the index column from tables

[zhiburt]

I'm just wondering if tabled needs to support column truncation where we show ... as a column and the other columns are just truncated?

So we can either truncate or wrap; We don't do both; We can but not out of the box.

See

ref: #152

[zhiburt]

So yes there're quite a few things to do to cover the whole nu_table usage.

It would be nice to have a side-by-side nushell table vs tabled table - it's hard to spot differences without comparing, but it's looking good!

Don't say we need tests :smile:

[fdncred]

I'd like to be able to prove that tabled can replace nushell tables somehow, even if tabled doesn't have all the features. I'm wondering if we can make it configurable to use tabled in nushell. Either put it behind a feature or add a config.nu flag. At least then we can spot issues and log them on your repo.

If you're not really interested in doing this, that's ok too. I just really like tabled. :)

yes, we'd need tests somehow. lol.

[zhiburt]

I tend to think I've covered most of the mentioned things?

You can check this out on the same branch https://github.com/zhiburt/nushell/tree/checkout-tabled.

[zhiburt]

1 thing which I didn't figured out is why on your screenshot size column has a right alignment when it's not an index, maybe you can clarify it?

[zhiburt]

todo: handle the case when table can't be rendered because not enough space.

[fdncred]

IIRC, the way it works in nushell is that data types are assigned an alignment. It's in the color_config.rs https://github.com/nushell/nushell/blob/f5519e2a0958112ce371fcab57b4bdbbc8f929ef/crates/nu-color-config/src/color_config.rs#L234-L385

[zhiburt]

IIRC, the way it works in nushell is that data types are assigned an alignment. It's in the color_config.rs https://github.com/nushell/nushell/blob/f5519e2a0958112ce371fcab57b4bdbbc8f929ef/crates/nu-color-config/src/color_config.rs#L234-L385

Great;

[zhiburt]

Also added the support for the notion if we can't print table.

[fdncred]

nice job. are these changes in your latest checkout-tabled branch? I'll try to try later if so.

[zhiburt]

are these changes in your latest checkout-tabled branch? I'll try to try later if so.

Yes they are.

Take care then :) til later

[fdncred]

looking very nice! good work.

the next challenge, if you choose to accept it, is to get this working a good as possible. Run your tabled version of nushell and open this file. Scroll around and look at all the column lines that are drawn wrong.

To be clear, this is a big problem in nushell and I'm not sure if there even is a "fix" for it since it depends on how each terminal renders emojis.

I'm also interested in caching, maybe related to above. for instance, read the first 1000 lines of the emoji file (or other files) before deciding how wide to set the columns. so, this caching isn't necessarily for speed but for determining proper column width.

[fdncred]

one of the core team members asked if string truncation is possible in tabled. for instance, if you're running nushell and type env you get a large table and the config items is pretty big. It may be nice to only show the first xx chars of any item. is that possible with tabled?

[fdncred]

I've also noticed that $env.config displays wrong. Are tables without column headers not a thing in tabled?

[zhiburt]

the next challenge, if you choose to accept it, is to get this working a good as possible. Run your tabled version of nushell and open this file. Scroll around and look at all the column lines that are drawn wrong.

I will.

one of the core team members asked if string truncation is possible in tabled. for instance, if you're running nushell and type env you get a large table and the config items is pretty big. It may be nice to only show the first xx chars of any item. is that possible with tabled?

Yes.

I'm also interested in caching, maybe related to above. for instance, read the first 1000 lines of the emoji file (or other files) before deciding how wide to set the columns. so, this caching isn't necessarily for speed but for determining proper column width.

That's interesting point. I was thinking about switching to Iterator<Iterator<String>> + Clone to eliminate allocations for Vec<Vec<String>>. Though we would need to do Clone once in a while to calculate dimensions. But we would not keep a big memory footprint.

https://github.com/zhiburt/tabled/issues/123

Your proposal is good. I've heard the same idea recently here https://github.com/wfxr/csview/issues/61.

But maybe it's just 2 different things.

To sum up it is something worth to consider.

I've also noticed that $env.config displays wrong. Are tables without column headers not a thing in tabled?

It is a thing;

Could you show the output?

[fdncred]

Yes.

Any ideas how to do that and make it configurable? Like we put truncate_table_strings_at: 50 in the config.nu to truncate at 50 chars/graphemes?

Your proposal is good. I've heard the same idea recently here https://github.com/wfxr/csview/issues/61.

Yes, agreed. That --sniff option sounds like the same thing.

Could you show the output?

Nushell

Nushell Tabled

It also looks like all datatypes aren't getting colored properly.

[zhiburt]

I've addressed truncate and header.

Personally I see this truncate like not well applicable in some cases. For example for a history command. When I run it I want the whole command no matter what no? Otherwise it will be necessary to go increase the limit etc. Or creating some alias.

How emojies look on your screen? On mine is fine but my font just doesn't support most of them.

BTW: where have you found this file with emojies? Or you've created it?

[fdncred]

I've addressed truncate and header.

nice work!

When I run it I want the whole command no matter what no?

yes, you would. I agree with you. I'm no sure how the functionality would/should work. for instance, when do we trigger string truncation vs when we don't want to trigger it? I'm not sure exactly.

also, notice that coloring doesn't appear to be working. notice my keys in the nushell version of $env.config are green. the tabled version is white.

How emojies look on your screen?

This is the problem with emoji's, each terminal renders them differently. notice how the columns aren't wide enough in some places.

BTW: where have you found this file with emojies? Or you've created it?

That file is downloaded from GitHub. It's their emoji list.

[zhiburt]

also, notice that coloring doesn't appear to be working. notice my keys in the nushell version of $env.config are green. the tabled version is white.

fixed

This is the problem with emoji's, each terminal renders them differently. notice how the columns aren't wide enough in some places.

I see.

But do nu-table handle it better?

I think maybe try to use width_cjk instead of width. It must better cover hieroglyphs. I just don't know if it will affect other locales and emojies.

[zhiburt]

Just installed pacman -S noto-fonts-emoji

Maybe we could estimate the width of emojie by font (.ttf)? Or figure it out from something like https://www.freedesktop.org/wiki/Software/fontconfig/

[fdncred]

Maybe we could estimate the width of emojie by font (.ttf)?

This concept took me a minute to understand. It's not really about measuring the width of a particular emoji. It's about how the terminal and font renders the emoji. So, if you try kitty, iterm2, wezterm, warp, you'll probably get different results. Some emoji have characters called ZWJ zero-width-joiners that are actually characters but they are zero width. There are other oddities with emoji too.

[fdncred]

I found another strange thing today following an issue in nushell.

nushell

echo ["now is the time for all good men", "hello"]
╭──────────────────────────────────╮
│ now is the time for all good men │
│ hello                            │
╰──────────────────────────────────╯

nushell tabled - notice that this isn't a list but a table with an empty column header

echo ["now is the time for all good men", "hello"]
╭────────────────────────────╮
│                            │
├────────────────────────────┤
│ now is the time for all .. │
│ hello                      │
├────────────────────────────┤
│                            │
╰────────────────────────────╯

I understand why it's truncated with .., is there any way to configure that right now?

And this was the real problem in nushell that I want to test and need to change the truncation length to test it.

nushell

echo ["জী._ডি._ব্লক_সল্টলেক_দূর্গা_পুজো_২০১৮.jpg", "Hello"]
╭───────────────────────────────────────────╮
│ জী._ডি._ব্লক_সল্টলেক_দূর্গা_পুজো_২০১৮.jpg │ <-- this is the error in nushell right now, it should be lined up.
│ Hello                                     │
╰───────────────────────────────────────────╯

nushell tabled

echo ["জী._ডি._ব্লক_সল্টলেক_দূর্গা_পুজো_২০১৮.jpg", "Hello"] 
╭────────────────────────────╮
│                            │
├────────────────────────────┤
│ জী._ডি._ব্লক_সল্টলেক_দূর্গা_.. │ <-- this looked fine in the terminal, when I pasted it here it moved left?
│ Hello                      │
├────────────────────────────┤
│                            │
╰────────────────────────────╯

[zhiburt]

I found another strange thing today following an issue in nushell.

Addressed

I understand why it's truncated with .., is there any way to configure that right now?

Added truncate_table_strings_suffix in config.

[fdncred]

sorry I wasn't more clear. It's cool to be able to change the truncation characters but what I was really meaning was a way to make it not truncate at all and/or change the width so it doesn't truncate.

[zhiburt]

/or change the width so it doesn't truncate.

Width can be changed by truncate_table_strings_at

[zhiburt]

Made a default version to non cut.

[fdncred]

thanks. this is what i was wanting to test in tabled.

regular nushell

[zhiburt]

Could you copy & paste a string here?

I'd test it.

[fdncred]

Could you copy & paste a string here?

I'd test it.

it's above https://github.com/zhiburt/tabled/issues/143#issuecomment-1149301162

[zhiburt]

Interestingly, on my end tabled is OK but nu-table isn't?

tabled

nu-table

[fdncred]

My above screenshots were Windows. These are now Mac.

nushell - wezterm

tabled - wezterm

the font I'm using is FiraCode Nerd Font Mono on Mac. tabled - iterm2 - it seems like iterm2 looks good but others don't

[zhiburt]

Yes I've just checked it. And the same string just looks differently depending terminal/font.

Can't say which font alacritty uses wezterm uses https://aur.archlinux.org/packages/ttf-freebanglafont aparently

wezterm

alacritty

BTW: Maybe this font on wezterm is not Mono and that's the reason?

[zhiburt]

I was thinking recently about this sniff configuration.

And can we do it?

I mean we can do it for width estimation. But we still will have to check each row to build a correct height of a row. Otherwise we will not able to do vertical alignment properly.

Maybe when it's used these limitations must be assumed. That certain things will be not possible to do.