Edit cell in-place

[firecat53]

Add the ability to edit a cell in-place and save.

Considerations:

• Tabview can be used with just tabular data. Would have to return the altered data instead of saving a file
• Should we overwrite the existing file, save changes into a new file, or create a backup of the original and save changes to the original filename?

[firecat53]

From another issue opened by @sametmax:

tabview file.csv should create the file if it doesn't exist, set dlimiters to a sane default and start editing the file as a blank slate.

[wavexx]

On top of that, if we allow editing, we should try to resave with the same format/style of the input. In some cases, this might require sanitizing (unix-like tab delimited would require stripping of newlines/tabs in the cells). In other cases (csv) it's even subtler: the quoting style is a PITA to handle.

[firecat53]

@wavexx: the delimiter is already discovered in the csv_sniff() method so it shouldn't be a problem to re-save with the same delimiter. I believe (not checked yet) that the cell contents should already be escaped properly for the given delimiter. I'm hoping the quoting won't be as difficult as you fear :) We also will already know the correct encoding from set_encoding().

I had been planning to use this module for the editing: https://github.com/firecat53/py_curses_editor

It's already got basic keyboard shortcuts, and we could use a conditional import to test whether or not editing is available:

try:
    import editor
    EDIT = True
except ImportError:
    EDIT = False
......
def keys():
    ....
    def edit():
        if EDIT is False:
    return

Or something like that. If I remember correctly, you can pass it an existing curses object so it could be integrated into the tabview window somehow (per discussion in #38). It's got a few bugs that need squashing but it's been pretty usable so far as the input method for keepassc.

Open to other thoughts!

[wavexx]

On 01/02/2015 08:06 PM, Scott Hansen wrote:

@wavexx: the delimiter is already discovered in the csv_sniff() method so it shouldn't be a problem to re-save with the same delimiter. I believe (not checked yet) that the cell contents should already be escaped properly for the given delimiter. I'm hoping the quoting won't be as difficult as you fear :) We also will already know the correct encoding from set_encoding().

I was thinking about the more subtle field quoting rules, such as quotes around strings (mandatory, never, or only with newlines), distinction between ints/strings/empty values.

I had been planning to use this module for the editing: https://github.com/firecat53/py_curses_editor

Yesterday I was thinking about urwid. It comes with a decent editor widget built-in which I've used in the past, and could simplify the layout code in a few places without having to reinvent the wheel.

However I doubt I'll have time to switch the tabview code to it.

[wavexx]

Speaking of which, the help panel doesn't fit a standard 80x25 terminal anymore.

[firecat53]

Ugh...yeah I think the split-window idea for (like mutt, etc) for displaying/editing cell contents will need to be scrollable.

[firecat53]

Testing out the py_curses_editor in the 'editor' branch for use with the popups (help, show cell and search) and for editing cells (no saving yet...).

You can install py_curses_editor from pypi (it's one commit behing master, but it's just a couple of minor things).

Let me know what you think....

[wavexx]

On 02/19/2015 04:39 PM, Scott Hansen wrote:

You can install py_curses_editor from pypi (it's one commit behing master, but it's just a couple of minor things).

Let me know what you think....

It makes the terminal flash/refresh the beginning and at each keypress, which is definitely not pleasant to use.

Resizing the terminal seems to work only partially: the editor is updated correctly, but the rest is not repainted.

I was able to get an addstr exception while fiddling with text, so there's some text wrapping issue hidden in there.

[firecat53]

Ugh, thanks for your honest feedback! :smile:

I never noticed the flashing on my machine...I guess I'll need to maybe test on a different computer and look at when I actually need to refresh.

I know about the second issue...on the bug list already.

Any chance you can figure out exactly when the addstr error occurs?

Thanks!

I guess one thing I'm still wondering even after doing all this is whether or not adding cell editing even makes sense for this "viewer". Am I getting a case of feature creep?

[wavexx]

On 02/19/2015 05:09 PM, Scott Hansen wrote:

Ugh, thanks for your honest feedback! :smile:

I didn't intent it to be so violent :P Just spun it up and had a few tests.

I never noticed the flashing on my machine...I guess I'll need to maybe test on a different computer and look at when I actually need to refresh.

On urxvt the refresh is very noticeable, less so on xterm, but still quite visible.

Any chance you can figure out exactly when the addstr error occurs?

I'll try more with it later.

I just created a file with a large cell of text and tried resizing the terminal randomly. That also triggered it.

[wavexx]

On 02/19/2015 05:09 PM, Scott Hansen wrote:

I guess one thing I'm still wondering even after doing all this is whether or not adding cell editing even makes sense for this "viewer". Am I getting a case of feature creep?

It's a good question.

Editing would be nice, but then again I rarely need to edit the content of a single cell.

Per-cell/column/row regex substitution, some number manipulation functions would be much more useful. The problem is that there's no end to it.

As a viewer, I still miss the ability to sort columns, move rows/columns, and add simple display filters. Those are frequent operations you need to do when looking at data effectively.

Maybe we should try to make the viewer class just a table-cell widget and re-implement tabview on top of it using callbacks.

For example, you could just hook 's' to view.register_key('s', func) and implement a function that resorts the array and updates it with view.set_data(...). Similarly you can get events as view.register('move_left', func), etc in order to implement custom behavior, much like you currently do with a normal gui widget.

At this point, tabview would be the simplest usage of the widget, and adding more powerful features in 'tabedit' would make sense.

[firecat53]

Sounds like a great idea. I'll have to go do some research, as I'm basically clueless about standard GUI design practices.

We should for now probably just retool the existing popup help, search and cell-view windows to act similar to say, mutt (full screen help, half-screen show cell, search input at the bottom/top) and use internal code for that instead of relying on an external library. It would be nice to have better editing for text input than the Textbox widget, but searching doesn't typically involve much text.

I wonder if by using the gui widget concept you could just use $EDITOR for cell editing instead.

[wavexx]

On 02/19/2015 06:53 PM, Scott Hansen wrote:

I wonder if by using the gui widget concept you could just use $EDITOR for cell editing instead.

I don't think so, even though I would like full vi-like editing.

Tabular data is often a one-liner. The current way of showing a pop-up for larger cells when you need isn't actually too bad honestly. With the larger info-line at the top I had to use the pop-up maybe once so far (using such an important key as enter for it doesn't seem warranted).

And I also still think that search should be incremental with instant feedback, so that shuns an external editor and would also favor the smallest search box possible in order to show the results immediately.

[firecat53]

Changed up the textboxes in the 'textbox' branch. Used ideas from your original fix_search branch and from @gurisko 's show_cell branch. This should set up nicely for the incremental search using the _search_validator() function. Thoughts?

[multimeric]

What happened with this? I noticed the textbox branch doesn't exist anymore. Did you merge it into develop? Where is help needed?

[firecat53]

Well, the project is rather on hold right now. I'll review and merge reasonable pull requests, but I haven't been able to work on any real development for quite some time now. Feel free to pick an issue and start hacking! The textbox branch was effectively merged with 3342ce58. Thanks!

[MstWntd]

@firecat53 and @wavexx many thanks for supporting this amazing project, the amount of times tabview has helped me out is insane, even vim totally sucks at csv viewing, having a purpose built cmdline tool with all the cool features super helpful.