Is the terminal version 'tabview' relevant anymore in light of the 'Visidata' project

[firecat53]

Hey all. Just wondering what your thoughts are on continued development of the terminal version of tabview given that Visidata seems to have sprung up and taken off nicely. It's not identical, but seems to have all the features of tabview and then some.

I just noticed the Visidata project recently. Given that I no longer use tabview almost daily, development has obviously been very slow!

Thoughts? Are there things that tabview does better to make it worth continuing?

Thanks! Scott

@wavexx and @scls19fr, this was primarly aimed at you since you've been most active in the project.

[wavexx]

On Sat, Nov 11 2017, Scott Hansen wrote:

Hey all. Just wondering what your thoughts are on continued development of the terminal version of tabview given that Visidata seems to have sprung up and taken off nicely. It's not identical, but seems to have all the features of tabview and then some.

Whoa there, I gave it a quick try and it's very nice! It even loads text files asynchronously!

There's a lot to like in there :)

I just noticed the Visidata project recently.

How did you discover it?

Given that I no longer use tabview almost daily, development has obviously been very slow!

So what? ;)

Development for gtabview has been slow as well, but I use both tabview and gtabview almost daily despite the flaws.

The python API similarity makes using them from a notebook and/or script surprisingly convenient.

Thoughts? Are there things that tabview does better to make it worth continuing?

I like how visidata uses underline to avoid the extra wasted line at the top (after headers). This is similar to my old branch "packed_layout".

tabview "[" "]" commands are super-convenient for skipping through categorical data in a column.

stdin slurping (tabview - < data) is also convenient when working with pipes.

I doubt both are hard to make available into visidata.

That being said, tabview is way smaller. And sometimes small things are good by themselves.

[wavexx]

On Sun, Nov 12 2017, Yuri D'Elia wrote:

Thoughts? Are there things that tabview does better to make it worth continuing?

Maybe we could ask if visidata wants to join TabViewer? Seems very fitting to me :)

[saulpw]

Hi @firecat53,

Thanks for noticing VisiData and for sharing it among your users. =) This means a lot, especially considering the work you put into Tabview and how much it has served the terminal community over these years.

If you do decide to discontinue active development, I would like to ensure that the needs of you and your users are met by VisiData. If anyone needs help with learning how to use it, or has workflow or feature requests, you can reach me on all of the communication channels mentioned here. Additionally, a new iteration of the website will be released soon, which will include more workflow tutorials.

@wavexx, thanks for the kind words. I agree that skipping through categorical data is very useful; you can do that in VisiData with < and >. ([ and ] are used for sorting by column). If you want to keep the tabview keybindings, you can make a local .visidatarc config file for yourself with some aliases.

Also, stdin slurping is already supported, both with - and without it. (ls | vd will work fine)

What other features do you use that tabview or gtabview support, that you haven't been able to find in VisiData? I would love to make them available if possible.

[wavexx]

On Sun, Nov 12 2017, Saul Pwanson wrote:

@wavexx, thanks for the kind words. I agree that skipping through categorical data is very useful; you can do that in VisiData with < and >. ([ and ] are used for sorting by column). If you want to

Nice!

Also, stdin slurping is already supported, both with - and without it. (ls | vd will work fine)

"vd -" seems to fail for me (but " ...| vd" works): This is from git master of yesterday:

$ ls | vd - Traceback (most recent call last): File "/usr/local/bin/vd", line 137, in main() File "/usr/local/bin/vd", line 97, in main vs = openSource(src) File "/usr/local/lib/python3.6/dist-packages/visidata/data.py", line 133, in openSource return openSource(Path(p), filetype) # convert to Path and recurse File "/usr/local/lib/python3.6/dist-packages/visidata/data.py", line 147, in openSource vs = getGlobals()openfunc File "/usr/local/lib/python3.6/dist-packages/visidata/data.py", line 172, in open_txt with p.open_text() as fp: File "/usr/local/lib/python3.6/dist-packages/visidata/vdtui.py", line 2244, in open_text return open(self.resolve(), mode=mode, encoding=options.encoding, errors=options.encoding_errors) FileNotFoundError: [Errno 2] No such file or directory: '-'

What other features do you use that tabview or gtabview support, that you haven't been able to find in VisiData? I would love to make them available if possible.

Some random remarks following some basic usage of vd:

If this isn't already available, it would be nice if you could use the terminal default colors: this is done using curses.use_default_colors().

This ensures the "normal" and "background" color use the current definition of the terminal. If you have a black-on-white terminal, you'd definitely like to keep it that way instead of reversing it. I set the background color depending on the host I'm connecting to, so changing the vd color scheme doesn't do the same thing.

The man page installed by setup.py is in lib/.../visidata/man/vd.1. Works for F1, but this should at least be hard-linked to share/man/man1 for "man" itself. The manpage is great!

Opening text files with a simple delimiter which is not "tab" is perhaps more annoying than it should. Having to change the delimiter is incredibly common. It seems the shortest way is:

vd -f csv --csv-delimiter '|'

except that I would also need to change/disable quotechar (these are dumb text files, not csv with a different delimiter).

Having "vd --delimiter '|'" influence the "tsv" format would be nice. In fact, tsv would probably be a specific case of a generic "txt" file with tab as a delimiter.

Also, --delimiter as an alias/replacement to --csv-delimiter to make it shorter would be cool. It's the most common flag you need to use. We used '-d' to copy unix tools, but I see you used that for delay already.

You should perhaps mention what "dialect" accepts in the man page, since it's already so neatly documented. As python developers we might know, but others will definitely not.

The list of dialects is not listed nearby the "-f" flag, just a dumb list or a "See SUPPORTED SOURCES" at the bottom would be nice.

In "SUPPORTED SOURCES" the sentence "For these multi..." seems incorrectly indented.

I don't see a way to set an initial position for the cursor (in terms of row/column). I'd recommend to see what we did in tabview/gtabview, which support the flag:

gtabview +row[:column]

which mimic the traditional seek flag in vi/emacs. Very nice to point users to specific cells.

The amount of features that vd packs is amazing. It's very well designed, and I like how you reuse the table layout for the settings screen and so on.

As I said before, if you want to be part of the "TabViewer" project group there's no problem [not sure if that would make a difference in terms of visibility!]

[saulpw]

Thanks for all the input, @wavexx! I've created issues for most of the suggestions you made, and we'll knock them down as we have time. Of course, if you have any interest in fixing any of these, I'd be happy to provide guidance and support, so we can get them done sooner. (Some are already on the roadmap, like installing manpages--they can't be installed via PyPi, so that requires creating system-level packages for brew/apt/etc, which are useful and convenient for other reasons as well).

The only tricky request here is for use_default_colors(). It might be possible, but I'd want to have a chat with you about the design, since the default VisiData color scheme may not work well with certain default terminal colors. Maybe there can be a --boring option that uses a color scheme which won't conflict. (Or maybe there is a simple color theme that we could create for your .visidatarc which will solve your problem with minimal changes to the core).

Thank you again for the kind words and all the suggestions. VisiData has been my full-time project over the past year, and I've put a lot of work into it without any real chance of compensation (I just want it to exist, to make my own life easier), so it means a lot to me that it's used and appreciated. I'd like to keep VisiData under my own account for now, unless/until it needs to be maintained by someone else.

BTW, if you are interested, I made a 5 minute demo of graphing functionality that will hopefully be in the upcoming v0.98 release. I got scrollwheel zoom working last night, and am working today on row selection from graph points. I think it should be pretty cool :)

[firecat53]

@saulpw nice project, BTW. Seems like you've taken the best features from tabview/gtabview and terminal spreadsheets like sc and teapot and mashed them together nicely!

@wavexx thanks for your thoughts. I'll continue making bug fixes to tabview but probably won't add too much in the way of features. Maybe I'll rewrite it in Go to make it even smaller and faster and give myself a reason to finally learn Go :smile:

I'll go ahead and close this 'issue' now. Thanks guys.

[wavexx]

On Sun, Nov 12 2017, Saul Pwanson wrote:

are already on the roadmap, like installing manpages--they can't be installed via PyPi, so that requires creating system-level packages

what's wrong with:

data_files[('share/man/man1', 'visidata/man/vd.1')]

?

The only tricky request here is for use_default_colors(). It might be possible, but I'd want to have a chat with you about the design, since the default VisiData color scheme may not work well with certain default terminal colors. Maybe there can be a --boring option that

This will always be true, even if you don't use use_default_colors(), since the terminal palette can be arbitrarily changed by the user.

There's no magic in this: call use_default_colors() and let users change the color scheme (which you already do). People that care will tweak to remove the conflict for their scheme, while the others are unlikely to have customized the terminal palette at all and thus will not notice any difference.

[saulpw]

Okay, these both seem to be working reasonably if not perfectly. They'll be in v0.98 which will be released in the next week or so. Thanks for making the points, they are well received.

[scls19fr]

I was personaly looking for Blaze support as an optional dependency https://github.com/saulpw/visidata/issues/4 so we could have MySQL/MariaDB support (but also all databases supported by SQLAlchemy)

[wavexx]

On Sun, Nov 12 2017, Saul Pwanson wrote:

BTW, if you are interested, I made a 5 minute demo of graphing functionality that will hopefully be in the upcoming v0.98 release. I got scrollwheel zoom working last night, and am working today on row selection from graph points. I think it should be pretty cool :)

I downloaded the video and I finally had time to watch it now.

It's always fun to abuse character graphics :P. I think the histogram display is very good for a number of scenarios and perfectly usable.

Line graphics are of course a bit chunky. Are you aware of sixel graphics?

https://en.wikipedia.org/wiki/Sixel

For some terminal porn, see: http://41j.com/hterm/

It's a bit sad my current favourite terminal (urxvt) doesn't support it, but it's very compelling when properly used. It's actually much better than tek4014 emulation (which requires a separate window), and support is actually growing (iterm2 on mac supports it natively, along with xterm and others!).

[wavexx]

On Sun, Nov 19 2017, Yuri D'Elia wrote:

It's always fun to abuse character graphics :P. I think the histogram display is very good for a number of scenarios and perfectly usable.

Also, for some more ideas, when using terminals to display data I had some success displaying horizontal inline graphs (or "sparklines") using the following unicode set: " ▁▂▃▄▅▆▇"

[wavexx]

On Sun, Nov 19 2017, Yuri D'Elia wrote:

It's a bit sad my current favourite terminal (urxvt) doesn't support it, but it's very compelling when properly used.

I just gave mlterm a try and now I wished ranger would support sixel previews as well as opposed to the ugly w3m-img hack =)

https://github.com/ranger/ranger/issues/723