Pressing " to filter selected rows does not work. Instead I get the error. No command for CTL_PADENTER

[kalyan860]

Small description Pressing " to filter selected rows does not work. Instead I get the error. No command for CTL_PADENTER

Expected result Opening of selected cells as a filter in a duplicate sheet.

Actual result with screenshot

Steps to reproduce with sample data and a .vd Select a few rows in a sheet with s key and then press " to filter them.

Please attach the commandlog (saved with Ctrl-D) to show the steps that led to the issue.

!vd -p

{"longname": "open-file", "input": "test2.xlsx", "keystrokes": "o"} {"sheet": "test2", "col": "", "row": "\u30adtest2", "longname": "open-row", "input": "", "keystrokes": "Enter", "comment": "open current row with sheet-specific dive"} {"sheet": "test2_test2", "col": "", "row": "1", "longname": "select-row", "input": "", "keystrokes": "s", "comment": "select current row"} {"sheet": "test2_test2", "col": "", "row": "3", "longname": "select-row", "input": "", "keystrokes": "s", "comment": "select current row"}

Additional context Please include the version of VisiData and Python. Python 3.11.1 saul.pw/VisiData v2.11.1

[saulpw]

What platform, what locale, what keyboard are you using? This is an issue with your environment and probably not something we can fix.

[kalyan860]

What platform? Windows11 What locale? LCID Name DisplayName

---

1033 en-US English (United States) What keyboard? I have an ASUS laptop and using the keyboard that comes with it.

Please help how I can fix this, I love visidata but without this functionality it will not be of much use to me.

[kalyan860]

This is the list of all my environment variables on windows. Please see if this can help.

ALLUSERSPROFILE=C:\ProgramData APPDATA=C:\Users\kalya\AppData\Roaming ChocolateyInstall=C:\ProgramData\chocolatey ChocolateyLastPathUpdate=133097932560588838 CommonProgramFiles=C:\Program Files\Common Files CommonProgramFiles(x86)=C:\Program Files (x86)\Common Files CommonProgramW6432=C:\Program Files\Common Files COMPUTERNAME=ASUSLAPTOPKALYA ComSpec=C:\Program Files\PowerShell\7\pwsh.exe CPLUS_INCLUDE_PATH=C:\Users\kalya\scoop\apps\gcc\current\include C_INCLUDE_PATH=C:\Users\kalya\scoop\apps\gcc\current\include DriverData=C:\Windows\System32\Drivers\DriverData EDITOR=nvim GEM_HOME=C:\Users\kalya\scoop\apps\ruby\current\gems GEM_PATH=C:\Users\kalya\scoop\apps\ruby\current\gems GIT_INSTALL_ROOT=C:\Users\kalya\scoop\apps\git\current GOROOT=C:\Users\kalya\scoop\apps\go\current HOME=C:\Users\kalya HOMEDRIVE=C: HOMEPATH=\Users\kalya LOCALAPPDATA=C:\Users\kalya\AppData\Local LOGONSERVER=\ASUSLAPTOPKALYA MAGICK_CODER_MODULE_PATH=C:\Users\kalya\scoop\apps\imagemagick\current\modules\coders MAGICK_CONFIGURE_PATH=C:\Users\kalya\scoop\apps\imagemagick\current MAGICK_HOME=C:\Users\kalya\scoop\apps\imagemagick\current NODE_PATH=C:\Users\kalya\scoop\apps\yarn\current\global\node_modules NUMBER_OF_PROCESSORS=4 OneDrive=C:\Users\kalya\OneDrive OneDriveConsumer=C:\Users\kalya\OneDrive OS=Windows_NT Path=C:\Program Files (x86)\Common Files\Oracle\Java\javapath;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WINDOWS\System32\WindowsPowerShell\v1.0\;C:\WINDOWS\System32\OpenSSH\;C:\ProgramData\chocolatey\bin;C:\Program Files\nu\bin\;C:\Program Files\gs\gs10.00.0\bin;C:\Program Files\PowerShell\7\;C:\Users\kalya\scoop\apps\imagemagick\current;C:\Users\kalya\scoop\apps\llvm\current\bin;C:\Users\kalya\scoop\apps\gpg\current\bin;C:\Users\kalya\scoop\apps\ghostscript\current\lib;C:\Users\kalya\scoop\apps\mpv\current;C:\Users\kalya\scoop\apps\yarn\current\global\node_modules.bin;C:\Users\kalya\scoop\apps\yarn\current\bin;C:\Users\kalya\scoop\apps\ruby\current\bin;C:\Users\kalya\scoop\apps\ruby\current\gems\bin;C:\Users\kalya\go\bin;C:\Users\kalya\scoop\apps\tcc\current\tcc;C:\Users\kalya\AppData\Roaming\local\bin;C:\Users\kalya\scoop\apps\haskell\current\lib\bin;C:\Users\kalya\scoop\apps\nodejs\current\bin;C:\Users\kalya\scoop\apps\nodejs\current;C:\Users\kalya\AppData\Local\Programs\Python\Python311\Scripts\;C:\Users\kalya\AppData\Local\Programs\Python\Python311\;C:\Users\kalya\scoop\apps\gcc\current\bin;C:\Users\kalya.cargo\bin;C:\Users\kalya\Arch_Online;C:\Users\kalya\scripts\whatscli-v1.0.11-windows;C:\Users\kalya\scoop\shims;C:\Users\kalya\AppData\Local\Microsoft\WindowsApps;C:\Program Files\Adobe\Acrobat DC\Acrobat\;C:\Program Files\Emacs\emacs-28.2\bin;C:\Program Files\Microsoft Office\root\Office16;C:\Program Files\Google\Chrome\Application;C:\ProgramData\Microsoft\Windows\Start Menu\Programs\Ghostscript;C:\Users\kalya\scoop\apps\python\3.10.8\Scripts;C:\Users\kalya\scripts;C:\Users\kalya.emacs.d\bin;C:\Users\kalya\AppData\Local\Pandoc\;C:\Users\kalya\AppData\Local\Microsoft\WindowsApps;C:\Users\kalya\AppData\Local\Programs\MiKTeX\miktex\bin\x64\;C:\Program Files\Adobe\Acrobat DC\Acrobat;C:\Program Files\LibreOffice\program\soffice.exe;C:\cygwin64\bin PATHEXT=.COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH;.MSC;.PY PHP_INI_SCAN_DIR=C:\Users\kalya\scoop\apps\php\current\cli;C:\Users\kalya\scoop\apps\php\current\cli\conf.d; POWERSHELL_DISTRIBUTION_CHANNEL=MSI:Windows 10 Home Single Language PROCESSOR_ARCHITECTURE=AMD64 PROCESSOR_IDENTIFIER=AMD64 Family 23 Model 24 Stepping 1, AuthenticAMD PROCESSOR_LEVEL=23 PROCESSOR_REVISION=1801 ProgramData=C:\ProgramData ProgramFiles=C:\Program Files ProgramFiles(x86)=C:\Program Files (x86) ProgramW6432=C:\Program Files PROMPT=$P$G PSModulePath=C:\Program Files\WindowsPowerShell\Modules;C:\WINDOWS\system32\WindowsPowerShell\v1.0\Modules PUBLIC=C:\Users\Public PYTHONPATH=C:\Users\kalya\scripts SBCL_HOME=C:\Users\kalya\scoop\apps\sbcl\current SCROLLVIEW_PATH=C:\Users\kalya\scoop\persist\tesseract\tessdata SESSIONNAME=Console SystemDrive=C: SystemRoot=C:\WINDOWS TEMP=C:\Users\kalya\AppData\Local\Temp TESSDATA_PREFIX=C:\Users\kalya\scoop\persist\tesseract\tessdata TMP=C:\Users\kalya\AppData\Local\Temp USERDOMAIN=ASUSLAPTOPKALYA USERDOMAIN_ROAMINGPROFILE=ASUSLAPTOPKALYA USERNAME=kalya USERPROFILE=C:\Users\kalya windir=C:\WINDOWS WSLENV=WT_SESSION:WT_PROFILE_ID: WT_PROFILE_ID={0caa0dad-35be-5f56-a8ff-afceeeaa6101} WT_SESSION=da707308-8a10-4c47-a8a8-63d5790c672b

[saulpw]

I have no idea what's going wrong with your setup, @kalyan860. I've never seen the CTL_PADENTER key before and I don't think anyone else has this issue on Windows. If you're just pressing " (Shift+' on a US keyboard) it should come up as a " and work fine. I don't know why it thinks a Ctrl key is being pressed here. Sorry.

[kalyan860]

CTL_PADENTER

I googled CTL_PADENTER and nothing comes up. I use a PC at work and same error is occuring there. I am in India. Is it probably because of some Indian locale keyboard setting? But this comes up when I check keyboard locale.

PS C:\Users\kalya\junk> Get-WinSystemLocale

LCID Name DisplayName

---

1033 en-US English (United States)

[isosphere]

I'm having the same issue when working from the development version pulled from github. My result from Get-WinSystemLocale is also 1033 en-US. I'm using Windows Terminal 1.18.2822.0. Python 3.10.4, same behaviour with Python 3.11.6.

Attempting to inspect my locale in Python (with the venv activated), I see:

>>> locale.getlocale()
('English_Canada', '1252')

While I am in Canada, I am using a US keyboard that is configured as such - I can't type accented French characters with my configuration. Windows-1252 is the most common encoding. The "English_Canada" just means my numbers and dates get formatted a little different. My " is a "":

>>> rawr = input()
"
>>> ord(rawr)
34

That's the decimal ASCII coordinate for ".

I think this project uses curses for input. Here's me trying the same in my virtualenv for this project:

>>> import curses
>>> stdscr = curses.initscr()
>>> stdscr.getch()
(nothing)
>>> stdscr.get_wch()
(nothing)
>>> stdscr.getkey()
(nothing)

It doesn't register me pressing ". It does however register me pressing @ just fine.

Windows Terminal does not appear to have any settings that might affect this. Using the default Windows Terminal Host (cmd.exe) does not change the behaviour.

Changing my Windows-level keyboard settings to English (United States) - US rather than English (Canada) - US has no effect.

When I modify my curses setup as recommended:

>>> curses.noecho()
>>> stdscr.keypad(True)
>>> stdscr.getch()
460
>>> stdscr.get_wch()
460
>>> curses.keyname(460)
b'CTL_PADENTER'

It tells me that my " character has an index of 460 (CTL_PADENTER).

The stdscr.keypad(True) call is necessary for my key to be registered at all - if I set it back to false, I get the old behaviour of the key not being noticed.

Checking the locale that curses is using:

>>> stdscr.encoding
'cp437'

Attempting to force en_US:

>>> import locale
>>> locale.setlocale(locale.LC_ALL, 'en_US')
'en_US'
>>> import curses
>>> stdscr = curses.initscr()
>>> stdscr.encoding
'cp437'

Upgrading to Python 3.11.6 so I have locale.getencoding():

>>> import locale
>>> locale.getencoding()
'cp1252'

[saulpw]

Okay, great. We did have a few curses changes recently, so if it was working with an older version and not on develop, then this should be straightforward to figure out. @isosphere do you know how to use git bisect?

Maybe it's related to #192 (https://github.com/saulpw/visidata/commit/3e7b3f3f3f9663ea557eba62439471d4c48e2ef0) though I can't see how.

[isosphere]

Okay, great. We did have a few curses changes recently, so if it was working with an older version and not on develop, then this should be straightforward to figure out. @isosphere do you know how to use git bisect?

Maybe it's related to #192 (3e7b3f3) though I can't see how.

A previous edit of my post indicated that I didn't have issues with a prior version, but I was mistaken. I have a separate, likely unrelated issue with CTRL+H in the development version versus the pipx version[^1].

[^1]: it's bound to hint, rather than the menu, despite the menu saying to press CTRL+H

Given my above results, it looks like curses is reporting that my " character is index 460 which it calls CTL_PADENTER; which is what visidata thinks I pushed; the issue seems to lie with curses thinking that a quote character pressed on a standard US keyboard in cp1252 encoding mode is a combination of CTRL plus the enter key on a number pad.

I should have stated that my _curses provider is the package windows-curses, which might be responsible. I believe https://github.com/zephyrproject-rtos/windows-curses/issues/41 is the same issue.

This user suspects it's a regression between versions 2.3.0 and 2.3.1 of windows-curses: https://github.com/zephyrproject-rtos/windows-curses/issues/41#issuecomment-1537215199

---

Solution

I can confirm that the solution is to downgrade windows-curses to 2.3.0:

pip install windows-curses==2.3.0

This requires that you use a Python version that is 3.10 or lower, as wheels are not available for newer versions. Hopefully the regression gets fixed soon.

[saulpw]

@isosphere How did you install VisiData? This should have been fixed with https://github.com/saulpw/visidata/commit/cb6c36fca6248664120d19b99cc1bde0b4ffba3c in response to #1901.

[isosphere]

@isosphere How did you install VisiData? This should have been fixed with cb6c36f in response to #1901.

In a virtualenv, pip install -r requirements.txt and pip install -r dev/requirements-dev.txt; windows-curses is not installed as a result of either of these, so I installed it manually with pip install windows-curses.

For my pipx installation (2.11.1), I used pipx inject visidata windows-curses==2.3.0 to downgrade.

[kalyan860]

I can confirm that the solution is to downgrade windows-curses to 2.3.0:

However, I get this error when trying to downgrade windows curses. PS C:\Users\kalya\junk> vd .\All_cars_dataset.csv PS C:\Users\kalya\junk> pip install windows-curses==2.3.0 ERROR: Could not find a version that satisfies the requirement windows-curses==2.3.0 (from versions: 2.3.1a2, 2.3.1, 2.3.2a1, 2.3.2) ERROR: No matching distribution found for windows-curses==2.3.0 PS C:\Users\kalya\junk> python --version Python 3.11.1

[saulpw]

@kalyan860 if 2.3.2 is out, try upgrading to that instead. Otherwise I don't know :/

[midichef]

I am copying my comment from a Discussion related to this bug, so it'll show up as mentions in the multiple visidata Issues relating to this problem. That way people can check on the updated status of this bug as of mid-2024. I do not expect windows-curses to fix it any time soon. It is unclear whether PDCurses has fixed it (see their Issues, #​126) and windows-curses does not have any activity this year on bugs that seem related (their Issues, #​41 and #​51):

--

Okay, I now see that this bug has been discussed before in https://github.com/saulpw/visidata/issues/2119 and https://github.com/saulpw/visidata/issues/2062 and #1841.

The last update to those threads was that windows_curses only works with a past version, 2.3.0. It was hoped then that 2.3.2 would fix it.

I can report that 2.3.2 does not fix it, nor does v2.3.3. I tested them on Python 3.12.4, where the versions of windows_curses available to me in pip are 2.3.2a1, 2.3.2, 2.3.3a1, and 2.3.3. None of these work satisfactorily. The first two (.2a1 and .2) cause visidata to exit on startup without showing any data screen; visidata a.tsv exits after saying opening a.tsv as tsv. That last two have the problem of incorrectly reading ' and " and ] and }.

I can report that downgrading windows_curses still works: pip install windows_curses==2.3.0. To use pip for this, I had to use Python 3.10, specifically 3.10.11, which is the last version of 3.10 that has a binary installer for Windows. However, that version is not an acceptable solution as it's missing several security fixes from 3.10.14. If you can figure out how to get Python 3.12.4 to use windows_curses 2.3.0, that may work.

But the solution I recommend is to run visidata inside Windows Subsystem for Linux.

--

Edited to update on 2024-08-06: there is another way to run visidata on Windows: with Python under MSYS2. Thanks to @tahir-hassan for working to find this alternate solution.