search

nkh / vimkat

Renders a file via Vim, vimcat replacement
26 stars 0 forks source link

Terminal cursor disappears and some other things happen after running vimkat on a file #3

Closed Hammermaniac closed 1 year ago

Hammermaniac commented 1 year ago

Hi there!

I'm really liking vimkat, but I seem to get an odd issue where after running vimkat to read a file:

  1. My terminal cursor (the blinking cursor) disappears and ;
  2. My mouse scroll-wheel function changes to scroll through my command history, instead of scrolling the printed output.
  3. Pressing Shift+PageUp types ~2 into the prompt.
  4. Afterwards opening vim and then quitting, the vimkat output is gone, along with the line where I ran the vimkat command.

I'm not sure what else gets changed, but it generally seems like the terminal goes into a weird state, I can get it back to normal by either:

  1. Opening vim and then quitting.
  2. Resetting the terminal by running reset.

Any idea why that might be?

I use Terminator as my emulator and gvim, but I've tested it on gnome-console along with normal vim as well and it's the same behavior. Let me know if I can provide something to give more insight.

I'm on Arch (6.1.12-arch1-1) and my ~/.vimrc:

" All system-wide defaults are set in $VIMRUNTIME/archlinux.vim (usually just
" /usr/share/vim/vimfiles/archlinux.vim) and sourced by the call to :runtime
" you can find below.  If you wish to change any of those settings, you should
" do it in this file (/etc/vimrc), since archlinux.vim will be overwritten
" everytime an upgrade of the vim packages is performed.  It is recommended to
" make changes after sourcing archlinux.vim since it alters the value of the
" 'compatible' option.

" This line should not be removed as it ensures that various options are
" properly set to work with the Vim-related packages.
runtime! archlinux.vim
" If you prefer the old-style vim functionalty, add 'runtime! vimrc_example.vim'
" Or better yet, read /usr/share/vim/vim80/vimrc_example.vim or the vim manual
" and configure vim to your own liking!

set rtp+=~/.vim/bundle/Vundle.vim
call vundle#begin()

Plugin 'VundleVim/Vundle.vim'
Plugin 'preservim/nerdtree'

call vundle#end()

" do not load defaults if ~/.vimrc is missing
"let skip_defaults_vim=1
set shell=bash
filetype plugin indent on
filetype indent on
syntax on
set expandtab
set tabstop=4
set shiftwidth=4
colo void
set nopaste
autocmd FileType * setlocal formatoptions-=c formatoptions-=r formatoptions-=o
highlight LineNr ctermfg=green
nkh commented 1 year ago

Hi, what a weird and unexpected error!

it would be good if you provided the command you use, version, version of script, ...

Does this happen on a specific input file?

vimkat is a "best attempt" using 1 line of code, the rest is fluff; trying to do the best of vim terminal output (which depends on the terminal and settings). But vimkat doesn't manipulate anything but the output it generates.

A wild, but educated, guess is that you get the output directly on your terminal, if you run you command but redirect it in a file, do you get the same type of behavior? if you don't then I'd need the input you give to vimkat (the shortest possible input that breaks vimkat)

There are 3 ways to implement a vim cat properly and vimkat isn't it :). When I get enough people complaining I'll try another implementation (yes congratulation you're the first error report of an application that seems to be used by 20 persons :) )

Hammermaniac commented 1 year ago

Thank you for replying!

vim version: 9.0 (1-1302) vimkat version: Latest master branch version

I've tried both your latest master branch version, as well as its initial commit. I'm essentially pointing to your vimkat executable and giving it to a file like this:

~/vimkat/vimkat ~/.bashrc

So after some further testing, things to note:

This seems to be a file-type thing actually! When using this on a shell OR Puppet (Ruby) file:

  1. My cursor and shell functionality stay normal, BUT ;
  2. The output seems to mangled a bit, where some lines completely disappear and others get cut and concatenated into other lines.

When using this on a javascript file:

  1. Terminal cursor disappears and the shell functionality gets mixed up.
  2. The output is perfectly fine! Nothing gets mixed up.

When redirecting the output to a file, the ansi codes get printed to it and using cat on that file results in the same behavior described above, specifically the same file-type.

Odd! Would you like me to provide you with the files perhaps?

nkh commented 1 year ago

Providing the files is certainly a good idea; both the source code and the files generated by vimkat.

The problem is that some codes should not be there and mess up your terminal. Don't expect miracles, the whole thing is a hack and what vim generates is mind boggling.

Hammermaniac commented 1 year ago

I'm at work right now so I'll send them through once I'm home.

May I ask what OS you're using? I'm getting the same behavior on my machine at work, an Ubuntu 20.04 LTS system, using vim version 8.1.

Hammermaniac commented 1 year ago

Hi there, attached are the files:

console.log("Whack-a-Mole!");

/
// void // / document.addEventListener('DOMContentLoaded', initialize);

/ Global variables / let table = document.getElementsByTagName('table')[0]; let tableData; let rowCount = table.rows.length; let columnCount = table.rows[0].cells.length; let imgElement = document.createElement('img'); let imgWidth; let imgHeight; let cell = document.getElementsByTagName('td'); let randRow; let randCol; let randTime; let hideTime;

/ Functions / function initialize () { console.log("Whack-a-Mole starting up..."); console.log(table); console.log('initialize'); tableData = new Array(rowCount); for (let r = 0 ; r < rowCount ; r++) { tableData[r] = new Array(columnCount); for (let c = 0 ; c < columnCount ; c++) { tableData[r][c] = table.rows[r].cells[c]; tableData[r][c].style.resize = "none"; } } imgWidth = tableData[0][0].clientWidth 0.92; imgHeight = tableData[0][0].clientHeight 0.92; randStart(); document.addEventListener('mousedown', function(event) { if (event.detail > 1) { event.preventDefault(); }; }, false); };

function randStart () { console.log('randStart'); console.log('randStart:', tableData); randRow = Math.floor(Math.random() rowCount); randCol = Math.floor(Math.random() columnCount); tableData[randRow][randCol].appendChild(imgElement).src = './mole.PNG'; tableData[randRow][randCol].appendChild(imgElement).width = imgWidth; console.log( 'randStart::width:', tableData[randRow][randCol].appendChild(imgElement).width ); tableData[randRow][randCol].appendChild(imgElement).height = imgHeight; console.log( 'randStart::height:', tableData[randRow][randCol].appendChild(imgElement).height ); table.rows[randRow].cells[randCol].addEventListener('click', removeClick); clearTimeout(randTime); clearTimeout(hideTime); waitTime(); };

function waitTime () { console.log('waitTime'); randTime = Math.floor((Math.random() * 2500) + 1000); if ( randTime > 2999 ) { randTime = 2500; } else if ( randTime < 1000 ) { randTime = 750; }; console.log('randTime:', randTime); setTimeout(hideMole, randTime); };

function hideMole () { console.log('hideMole'); table.rows[randRow].cells[randCol].removeEventListener('click', removeClick); tableData[randRow][randCol].removeChild(imgElement); hideTime = Math.floor((Math.random() * 2000) + 1000); if ( hideTime > 1500 ) { hideTime = 1500; } else if ( hideTime < 1000 ) { hideTime = 750; }; console.log('hideTime:', hideTime); setTimeout(randStart, hideTime); };

function removeClick () { console.log('removeClick'); clearTimeout(randTime); clearTimeout(hideTime); let audio = new Audio('./audio.wav'); let randShow = Math.floor((Math.random() * 2000) + 1000); if ( randShow > 1500) { randShow = 1500; } else if ( randShow < 1000 ) { randShow = 1000; }; audio.play(); table.rows[randRow].cells[randCol].removeEventListener('click', removeClick); tableData[randRow][randCol].removeChild(imgElement); console.log(table.rows[randRow].cells[randCol]); setTimeout(randStart, randShow); };


- Redirected bashrc file: [vimkat-redirected-bashrc.txt](https://github.com/nkh/vimkat/files/10800686/vimkat-redirected-bashrc.txt)
- Source bashrc:
```shell
# ~/.bashrc: executed by bash(1) for non-login shells.
# see /usr/share/doc/bash/examples/startup-files (in the package bash-doc)
# for examples

# If not running interactively, don't do anything
case $- in
    *i*) ;;
      *) return;;
esac

# don't put duplicate lines or lines starting with space in the history.
# See bash(1) for more options
HISTCONTROL=ignoreboth

# append to the history file, don't overwrite it
shopt -s histappend

# for setting history length see HISTSIZE and HISTFILESIZE in bash(1)
HISTSIZE=1000
HISTFILESIZE=2000

# check the window size after each command and, if necessary,
# update the values of LINES and COLUMNS.
shopt -s checkwinsize

# If set, the pattern "**" used in a pathname expansion context will
# match all files and zero or more directories and subdirectories.
#shopt -s globstar

# make less more friendly for non-text input files, see lesspipe(1)
[ -x /usr/bin/lesspipe ] && eval "$(SHELL=/bin/sh lesspipe)"

# set variable identifying the chroot you work in (used in the prompt below)
if [ -z "${debian_chroot:-}" ] && [ -r /etc/debian_chroot ]; then
    debian_chroot=$(cat /etc/debian_chroot)
fi

# set a fancy prompt (non-color, unless we know we "want" color)
case "$TERM" in
    xterm-color|*-256color) color_prompt=yes;;
esac

# uncomment for a colored prompt, if the terminal has the capability; turned
# off by default to not distract the user: the focus in a terminal window
# should be on the output of commands, not on the prompt
#force_color_prompt=yes

if [ -n "$force_color_prompt" ]; then
    if [ -x /usr/bin/tput ] && tput setaf 1 >&/dev/null; then
    # We have color support; assume it's compliant with Ecma-48
    # (ISO/IEC-6429). (Lack of such support is extremely rare, and such
    # a case would tend to support setf rather than setaf.)
    color_prompt=yes
    else
    color_prompt=
    fi
fi

if [ "$color_prompt" = yes ]; then
    PS1='${debian_chroot:+($debian_chroot)}\[\033[01;32m\]\u@\h\[\033[00m\]:\[\033[01;34m\]\w\[\033[00m\]\$ '
else
    PS1='${debian_chroot:+($debian_chroot)}\u@\h:\w\$ '
fi
unset color_prompt force_color_prompt

# If this is an xterm set the title to user@host:dir
case "$TERM" in
xterm*|rxvt*)
    PS1="\[\e]0;${debian_chroot:+($debian_chroot)}\u@\h: \w\a\]$PS1"
    ;;
*)
    ;;
esac

# enable color support of ls and also add handy aliases
if [ -x /usr/bin/dircolors ]; then
    test -r ~/.dircolors && eval "$(dircolors -b ~/.dircolors)" || eval "$(dircolors -b)"
    alias ls='ls --color=auto'
    #alias dir='dir --color=auto'
    #alias vdir='vdir --color=auto'

    alias grep='grep --color=auto'
    alias fgrep='fgrep --color=auto'
    alias egrep='egrep --color=auto'
fi

# colored GCC warnings and errors
#export GCC_COLORS='error=01;31:warning=01;35:note=01;36:caret=01;32:locus=01:quote=01'

# some more ls aliases
alias ll='ls -alF'
alias la='ls -A'
alias l='ls -CF'

# Add an "alert" alias for long running commands.  Use like so:
#   sleep 10; alert
alias alert='notify-send --urgency=low -i "$([ $? = 0 ] && echo terminal || echo error)" "$(history|tail -n1|sed -e '\''s/^\s*[0-9]\+\s*//;s/[;&|]\s*alert$//'\'')"'

# Alias definitions.
# You may want to put all your additions into a separate file like
# ~/.bash_aliases, instead of adding them here directly.
# See /usr/share/doc/bash-doc/examples in the bash-doc package.

if [ -f ~/.bash_aliases ]; then
    . ~/.bash_aliases
fi

# enable programmable completion features (you don't need to enable
# this, if it's already enabled in /etc/bash.bashrc and /etc/profile
# sources /etc/bash.bashrc).
if ! shopt -oq posix; then
  if [ -f /usr/share/bash-completion/bash_completion ]; then
    . /usr/share/bash-completion/bash_completion
  elif [ -f /etc/bash_completion ]; then
    . /etc/bash_completion
  fi
fi

# Created by `pipx` on 2022-07-23 11:10:57
export PATH="$PATH:/home/void/.local/bin"

if [ -f ~/.aliases ]; then
    . ~/.aliases
fi

export VK_ICD_FILENAMES=/usr/share/vulkan/icd.d/radeon_icd.i686.json:/usr/share/vulkan/icd.d/radeon_icd.x86_64.json

I thought it might be better to attach the redirected output instead of pasting them in the comment, it's less scrolling and might keep the characters intact, please let me know if I should take a different approach!

I realize you might not be able to help and that's fine :) I come from using vimcat from rkitover/vimpager but vimcat specifically got broken after an OS update (vimpager and vim still works great though) and whilst looking for a fix I found this alternative, which is much faster as well. So I'm hopeful! But I understand if you can't help.

nkh commented 1 year ago

Hi,

the good news, for me but bad for you, vimkat outputs both the java and bash file without any problem on my box (Kubuntu, konsole, tmux if you want to give it a shot). also your vimkat generated file for the bash file can be "cat"ed without problem on my box.

the bad news, for me but with some hope for you, your vimkat generated java file output does hide the cursor (or rather doesn't set it back I think), the whack-a-mole file gives me some hint.

I'll diff the outputs of the java code (the only one I have source and file generated on your machine that gives a difference) and see what I can make out of it. If you have other files that don't render properly please send some to me, as files and named so I know what is what).

Hammermaniac commented 1 year ago

Interesting! Alright here are 3 files, I seem to need to rename the files to txt in order to be able to upload them, otherwise github doesn't allow me to upload them, do you think you could perharps remove the .txt file extensions of the files for your testing, if that would make any difference? So .js for Javascript and .pp for Ruby, please :)

  1. inflate-the-unicorn.js:

console.log("Inflate The Unicorn!") document.addEventListener('DOMContentLoaded', initialize);

function initialize () { console.log('Show the Unicorns!'); startImages = document.getElementsByClassName('inflate-an-image'); console.log('Image:', startImages[0].src); for (var k = 0 ; k < startImages.length ; k++) { startImages[k].src = './images/unicorn-0.png'; } clickFirst(document.getElementsByClassName('inflate-an-image')[0]); clickSecond(document.getElementsByClassName('inflate-an-image')[1]); clickThird(document.getElementsByClassName('inflate-an-image')[2]); }

function clickFirst (evt) { evt.addEventListener('click', showUnicorns); } function clickSecond (evt) { evt.addEventListener('click', showUnicorns); } function clickThird (evt) { evt.addEventListener('click', showUnicorns); }

function showUnicorns (whichUni) { console.log(whichUni.target); evt = document.getElementsByClassName('inflate-an-image'); console.log('showUnicorns::evt:', evt); let avaiImages = { start: './images/unicorn-0.png', first: './images/unicorn-1.png', second: './images/unicorn-2.png', third: './images/unicorn-3.png' } let avaiImagesVal = { start: avaiImages.start.slice(-5, -4), first: avaiImages.first.slice(-5, -4), second: avaiImages.second.slice(-5, -4), third: avaiImages.third.slice(-5, -4) } let unicorns; let curImage = whichUni.target.src.slice(-5, -4); let nextImageVal = whichUni.target.src.slice(-5, -4); nextImageVal++; let nextImage = './images/unicorn-' + nextImageVal + '.png'; for (var u = 0 ; u < document.getElementsByClassName('inflate-an-image').length ; u++) { unicorns = document.createAttribute('unicorn' + u); if (curImage < avaiImagesVal.third) { console.log('showUnicorn::curImage:', curImage); unicorns.value = nextImage; whichUni.target.src = unicorns.value; } else { whichUni.target.src = avaiImages.start; } console.log('forloop::unicorn-image:', unicorns.value); } }

2. `game.js`
- `/tmp` generated file: [vimkat_typescript_game.js_2214.txt](https://github.com/nkh/vimkat/files/10802335/vimkat_typescript_game.js_2214.txt)
- Redirected: [vimkat-redirected-game.js.txt](https://github.com/nkh/vimkat/files/10802339/vimkat-redirected-game.js.txt)
- Source:
```javascript
// Don't change or delete this line! It waits until the DOM has loaded, then calls 
// the start function. More info: 
// https://developer.mozilla.org/en-US/docs/Web/Events/DOMContentLoaded
document.addEventListener('DOMContentLoaded', start)

function start () {
  bindEventListeners(document.getElementsByClassName('board')[0].children);
}

function bindEventListeners (dots) {
  for (var i = 0 ; i < dots.length ; i++) {
    // BIND YOUR EVENT LISTENERS HERE
    // The first one is provided for you
    dots[i].addEventListener('contextmenu', makeGreen);
    dots[i].addEventListener('click', makeBlue);
    dots[i].addEventListener('dblclick', hide);
  }
}

// CREATE FUNCTION makeBlue HERE
function makeBlue (evt) {
    if ( evt.target.className.includes('green') ) {
        evt.target.classList.toggle('green');
    }
    evt.target.classList.toggle('blue');
    updateCounts();
}

function makeGreen (evt) {
  evt.preventDefault()
  if ( evt.target.className.includes('blue') ) {
      evt.target.classList.toggle('blue');
  }
  evt.target.classList.toggle('green');
  updateCounts();
}

// CREATE FUNCTION hide HERE
function hide (evt) {
  if ( evt.target.className.includes('blue') ) {
      evt.target.classList.toggle('blue');
  } else if ( evt.target.className.includes('green') ) {
      evt.target.classList.toggle('green');
  }
  evt.target.classList.toggle('invisible');
  updateCounts();
}

function updateCounts () {
  var totals = {
    blue: 0,
    green: 0,
    invisible: 0
  }
  // WRITE CODE HERE TO COUNT BLUE, GREEN, AND INVISIBLE DOTS
    function countTotals (dot_array) {
        console.log('dot_array is:', dot_array);
        for (var n = 0 ; n < dot_array.length ; n++) {
            if ( dot_array[n].className.includes('invisible') ) {
                totals.invisible = totals.invisible + 1;
                console.log('Invisible changed to:', totals.invisible);
            } else if ( dot_array[n].className.endsWith('green') ) {
                totals.green = totals.green + 1;
                console.log('Green changed to:', totals.green);
            } else if ( dot_array[n].className.endsWith('blue') ) {
                totals.blue = totals.blue + 1;
                console.log('Blue changed to:', totals.blue); 
            }
        }
        console.log('Totals are:', totals);
    }
    countTotals(document.getElementsByClassName('board')[0].children);
  // Once you've done the counting, this function will update the display
  displayTotals(totals);
}

function displayTotals (totals) {
  for (var key in totals) {
    document.getElementById(key + '-total').innerHTML = totals[key];
  }
}

  1. chrome.pp:

    • /tmp generated file: vimkat_typescript_chrome.pp_2039.txt
    • Redirected: vimkat-redirected-chrome.pp.txt
    • Source: 
      
# Make Chrome and Chromium launch with proxy
class node_management::chrome {
# copy chrome_proxy.sh script over to node 
$chrome_proxy_source = 'puppet:///modules/node_management/usr/local/bin/chrome_proxy.sh'
file { '/usr/local/bin/chrome_proxy.sh':
source => $chrome_proxy_source,
owner  => root,
group  => root,
mode   => '0755'
}
# replace "Exec" in chromium file to run with the proxy file
# $chromium_file = '/usr/share/applications/chromium-browser.desktop'
exec { 'chrome_and_chromium_proxy':
command => '/usr/local/bin/chrome_proxy.sh',
# onlyif  => "/usr/bin/test -f ${chromium_file}",
}

    ###############################################

    check if file chromium-browser.desktop exists

    $chromium_file = '/usr/share/applications/chromium-browser.desktop' $chromium_file_exists = find_file($chromium_file) $chromium_source_file = 'puppet:///modules/node_management/usr/share/applications/chromium-browser.desktop' if $chromium_file_exists { file { '/usr/share/applications/chromium-browser.desktop': source => $chromium_source_file, owner => root, group => root, mode => '0644' notify => File['/usr/share/applications/chromium-browser.desktop'], } }

    overwrite chromium-browser.desktop file

    file { '/usr/share/applications/chromium-browser.desktop': source => $chromium_source_file, owner => root, group => root, mode => '0644' }

    Facter.add('chromium_file_exists') do setcode do File.exists?('/usr/share/applications/chromium-browser.desktop') end end

    $chromium_source_file = 'puppet:///modules/node_management/usr/share/applications/chromium-browser.desktop' if $facts['chromium_file_exists'] { file { 'chromium-browser.desktop': path => '/usr/share/applications/chromium-browser.desktop', source => $chromium_source_file, owner => root, group => root, mode => '0644' } } }

Please note that even though it doesn't hide the cursor after running vimkat on the chrome.pp file, it still mangles the output, but it also seems to be doing that on the game.js file, which is strange since the inflate-the-unicorn.js javascript file doesn't seem to be resulting in mangled output.

One key difference I'm noticing though is that the mangled output seems to be from files that are larger than what the terminal screen space can display, without needing to scroll. I don't know if that is relevant? For example if I use it on my howto script it cuts away a lot of output. See here: vimkat-redirected-howto.sh.txt

Hammermaniac commented 1 year ago

A note on the above .sh script, using vimkat on it does not make the cursor disappear. Do you perhaps see a specific code that might be doing that?

nkh commented 1 year ago

"\e[?25l" hides the cursor, the problem I was thinking about is that the cursor is not re-enabled

When sending files please also send all the files as an attachment; having to copy it, open an editor, paste, save, doesn't make things easier. could you please resend everything named as suggested below?

file.type.txt vimkat.file.type.txt vimkat_typescript_file.type_number.txt

There's an option to limit the number of lines rendered that I use to generate a shorter preview so it's faster, you may want to try that. just run vimkat without arguments for the help.

I need to setup a better diff environment, trying a few programs right now, to find out what the differences are, I hope I don't have to write something specific for it (but looks that way). The differences can only come from one place and it's vim which also depends to some extents on what terminal you're using.

I also suggest that you run with the vimrc file that comes with vimkat for a test, that's also in the help.

nkh commented 1 year ago

also, for each file triplet tell me exactly what you have for problem, for example I have no idea of what game.js gives for bad result and on my box vimkat works perfectly fine and the vimkat generated file can be output without problem.

also generate a file with vimkat's vimrc file, we have different color schemes and that makes diffing files more difficult.

file.type.txt vimkat.file.type.txt vimkat_default_vimrc.file.type.txt vimkat_typescript_file.type_number.txt

It would be a good idea if you give me an example of a file that may render wrong because of its size (and name it accordingly)

nkh commented 1 year ago

this could give me the easiest way to compare:

vimkat game.js -1 ./vimkatrc

where ./vimkatrc is a file you create and that just contains "syntax off"

Hammermaniac commented 1 year ago

Ohh I'll attach them all, I didn't think at the time it would be an issue, but it would be inconvenient for me too, sorry about that.

Please find the files below, I've named the redirected files with the problem it's having: MANGLED - indicates "cut" and mixed up output CURSOR - indicates the cursor disappearing and the terminal acting odd.

bashrc (MANGLED): Source: bashrc.txt Redirected: vimkat.MANGLED_bashrc.txt /tmp: vimkattypescript.bashrc_2947.txt

chrome.pp (MANGLED): Source: chrome.pp.txt Redirected: vimkat.MANGLED_chrome.pp.txt /tmp: vimkat_typescript_chrome.pp_3028.txt

howto.sh (MANGLED): Source: howto.sh.txt Redirected: vimkat.MANGLED_howto.sh.txt /tmp: vimkat_typescript_howto_3265.txt

game.js (CURSOR): Source: game.js.txt Redirected: vimkat.CURSOR_game.js.txt /tmp: vimkat_typescript_game.js_3200.txt

inflate-the-unicorn.js (CURSOR): Source: inflate-the-unicorn.js.txt Redirected: vimkat.CURSOR_inflate-the-unicorn.js.txt /tmp: vimkat_typescript_inflate-the-unicorn.js_2853.txt

whack-a-mole.js (CURSOR): Source: whack-a-mole.js.txt Redirected: vimkat.CURSOR_whack-a-mole.js.txt /tmp: vimkat_typescript_whack-a-mole.js_3109.txt

These were generated with:

$ vimkat ~/personal/howto/howto > vimkat.MANGED_howto.sh.txt
$ which vimkat                                        
vimkat () {
    ~/vimkat/vimkat $1 -1 /home/void/personal/testie/testie2/vimkatrc
}

And the vimkatrc file:

void:testie2/ $ cat vimkatrc
syntax off

I use terminator, but as I've mentioned before, this is the case with gnome-terminal and a TTY line, happens on different machines for me as well:

Let me know if you need anything else! Thank you so much for your time :)

Hammermaniac commented 1 year ago

By TTY I simply mean pressing Ctrl+Alt+F2 and running the commands in a non-graphical TTY line, so not running them through an installed terminal emulator, sorry I should have specified that.

Also, I've tested the TTY behavior on both my Arch installation and the Kubuntu 22.04 install - same behavior.

The Ubuntu 20.04 install is on an entirely different machine, a laptop specifically, whereas the other 2 installations are on my tower.

Hammermaniac commented 1 year ago

I didn't attach files with my default vimrc, but I don't think it's necessary because the files processed with the vimkatrc file has the same result and behavior as the ones with my vimrc.

Did you specifically need ones with my default vimrc as well? I realized after leaving the house that you might want to diff the difference between those 2. Let me know if you need those as well and I'll edit my comment when I'm able to!

nkh commented 1 year ago

it's easier to have the smallest vimrc that make vimkat fail, once that is fixed we can look at a more complex vimrc and if it has any impact.

I run "VIM - Vi IMproved 8.2 (2019 Dec 12, compiled Jan 11 2023 23:53:12)", do you have that version, any older version, on one of your machines?

I hope I'll have time during the WE to look at this, been swamped in another project (Asciio) this week.

nkh commented 1 year ago

OK, I have bad news and more bad news.

I found why some files are mangled ... they are not! clear your terminal and cat a mangled file, chances are it will look ok, fill your terminal with text (ls, find, whatnot), cat your mangled file and chances are you will get a mangled output.

There are multiple reasons for that, first vim generates codes it sends to the terminal, those code will depend on what's in your termcap or terminfo, it also depends on what vim thinks works in a terminal and it also depends on the terminal setting you add (https://vim-jp.org/vimdoc-en/term.html). Vim's codes generation boggles the mind, I'm sure there's a good reason and it is optimized but even with my version its codes generation leaves me perplexed. You have another version so chances are the generation is different (well it is different I can see that when comparing what you send and what I generate). So we're more or less att the mercy of vim. Don't give up hope yet.

Then we have 'script' which records an interactive session, that's not the same as being a terminal emulator, script just passes codes down to whatever terminal you have. This means that script recorded 'interactive' output works best in the same terminal, which makes sense.

Then I come along and try to make a terminal emulator out of an interactive session recorder, that's bound to fail, and it's surprisingly good for what it is.

Not only do I abuse script but I also abuse the output by removing stuff I find "useless", one of those things is codes to clear the terminal and to set cursor at 1,1. The problem is that your version of vim generates code that looks like: clear everything (that I remove) print this print that hej let's jump to X, Y because I know I can and print stuff.

I remove the terminal clearing because I want to cat stuff at the position I am not at the top of the terminal. Remember that vim will position the cursor at X,Y? it does that because it know that the terminal was cleared but I screw that up.

vimkat doesn't print a colored file, it replays a mangles interactive session (because vim doesn't know how to work non interactively we have to do just that).

So what can be done? I can leave the terminal reset, an option as vimkat works good for me as it is, that means that when you cat files captured from vimkat you'll be playing an interactive session (doesn't matter the result is the same) and thus will clear your terminal. That would work fine in application like fzf, ...

The cursor problem is the same, I probably take too much liberty with my removing codes, the good news is that I could add my own and maybe the problem would be fixed.

Where are we?

Sorry for the long explanation, let me know if you want to try the little hack or if you want to wait and see something a bit more solid.

Hammermaniac commented 1 year ago

Don't apologize! That was a pretty fun read! Could you perhaps make a new branch that has the other clear and cursor reset codes? What is the clear code btw? You've sent me the cursor reset code before: "\e[?25l" — I'm certainly willing to test out what you have in mind.

Why would removing those 2 codes be an issue, would the behavior be undesired? Why would you want those codes there?

I don't think forcing a specific terminal emulator is a good idea, I think it's a better idea to figure out how to get it to work across multiple terminal emulators :)

Hammermaniac commented 1 year ago

I do have a Vim version 8.1 installation on an Ubuntu 20.04 LTS machine btw.

nkh commented 1 year ago

Removing the code mangles the output, if the output contains a position reset, all the position codes after assume that the cursor was placed at 1,1 first. Here's an example: goto 1,1 print A print B goto 3,1 print C

that would print (starting at the top of the terminal) A B C

if you remove the goto 1,1 and are at line, say, 25 when you out put the commands:

A (at line 25) B (at line 26) C (at line 3 because of goto 3,1)

not really what you expect!

the same goes for the cursor: hide cursor do lots of stuff show cursor

if I remove the code to show the cursor then it will not be shown

The text casting app I wrote was a good start, I stopped using it because the VT102 emulation it didn't support ECMA-48 sequence for 256 colors, you can run "man console_code" if you are curious, I'll try to add that support so I can use it to implement vimkat properly and I''ll also upgrade my text casting application in the same shot.

I'll try to do the small hack this weekend so you can try it, it's a two lines change; I just need to think how ugly those two lines should be.

The terminal emulator change is going to take longer, but much less than writing my own, the code is large but redable so there's a good chance of success.

Hammermaniac commented 1 year ago

Alright, I'll wait for your reply then, thank you for taking the time! If you don't get to it this weekend it's all good, no rush :)

nkh commented 1 year ago

https://github.com/nkh/vimkat/tree/set_pos_reset_cursor

try this branch and let me know if it works better

Hammermaniac commented 1 year ago

That shows the cursor again, which has fixed that! I see you're simply sending the codes to the terminal for that, that's honestly so cool how the codes do that for a terminal :D

I was only aware of codes that can change text color and boldness.

I'm not sure how the mangling thing will be fixed though :/ Because that hasn't fixed that.

Could you please send me one of the .js files that you have used vimkat on? I want to see the difference in codes. I mentioned before that the terminal does get in "a weird state" and it acts odd, specifically I've noticed that after using vimkat on a .js file:

I think it's not ending the interactive state when specifically running it on those files, if that's the case, is there a way to ensure that either the interactive state/session gets ended or perhaps ensure that it gets printed out to the terminal the way that something like echo prints it out?

It's as if the behavior for those files are different from that of shell files. I think it's due to other syntax, color etc files used by vim.

I'm not nearly as experienced as you with how this works, so I'm sorry if what I'm saying is just babble. :)

nkh commented 1 year ago

Re-send me a file triplet that still gets mangled, that should have been fixed, don't use the vimrc that removes color (use nothing), I think I understand enough to be able to work with colors now.

I've just been using the js files you send me.

I do have an application where my history scrollback gets messed up, maybe it's related, I put the input in raw-mode (IE: I bypass the line discipline), so let's shoot in the dark and try something (with near zero chance of success): add "stty cooked" on a new line after the line which text is "echo -en "\e[?25h"" and let's wee what happens.

hmm, how do you scroll? keyboard or mouse? Try adding 'echo -en "\e[?1000h"' too.

The problem is that I can't reproduce your error here, If I could things would be much easier. I'm working in the dark. So I'm going to ask you to create a Virtual Machine, install whatever terminal and vim you like in, test that it doesn't work as expected.

I'm not nearly as experienced as you think, so I'm saying is just babble. :)

Hammermaniac commented 1 year ago

Hi there, I haven't gotten to doing this, things went side ways the past couple of days.

Today I logged on and ran an update before installing virtualbox, then ran into an issue where VMs did not want to start due to an issue with the kernel. Rebooting (in hopes of seeing a difference) presented me with an error "kernel needs to be loaded first" on Arch so I need to get that fixed first (fallback kernel doesn't work either).

Please give me a few days, I need to sort out things at work and get my OS fixed. Sorry about that!

nkh commented 1 year ago

No worries, I got stuck in another project (https://github.com/nkh/P5-App-Asciio) and reviving https://github.com/nkh/P5-App-Asciio which uses a terminal emulator, the later I hope to be able as backend for Vimkat;.

I'm adding 256 and rgb color support, then I need to find out why vim doesn't work properly in it (of course!), if I get that to work (it will take time) we'll have a much more solid vimkat (but it will be slower till I fix the C version of it)

nkh commented 1 year ago

@Hammermaniac if you're using tmux (if not try), there's a new version tvcat that you may want to try.

Hammermaniac commented 1 year ago

I can't seem to find anything online about tvcat, is that a utility that ships with tmux or something?

Also, I've recently found that my vim color theme configuration file had an error in it and after fixing that vimcat's color profiling is working correctly :)

Hammermaniac commented 1 year ago

I think you can close this issue to be honest, as think I the issues are with my setup's compatibility with your project, rather than any fault of your project.

Thank you so much for your efforts helping out though!

nkh commented 1 year ago

tvcat is part of the vimkat distribution. I started playing a bit with a faster and better solution (would probably be called "rat") but I have too many projects open at the same time so it may take a while.

thank you for your input