please update vim (and xxd)

[avih]

The vim version in w64dk is 9.0, from 2022 judging by the ouitput of vim --version.

I'm not familiar with the vim release process, but I'd think a newer version would be more in-line with the "latest" w64dk policy with gcc, binutils, etc.

At the very least, xxd (from the vim source code) gained colored output sometime in 2023, which works nicely on windows too (tested by compiling latest xxd.c on its own, which compiles cleanly in w64dk).

EDIT: Hmmm... I see the latest official version is 9.0 at http://ftp.vim.org/pub/vim/unix So I guess that's why.

Anyway, while it's trivial for anyone to compile a newer xxd, it would still be nice to include it in w64dk - assuming it's completely stand-alone and not used by other vim sources. I compiled this https://github.com/vim/vim/blob/master/src/xxd/xxd.c

[skeeto]

There are a couple light blockers, and I'm not so personally motivated to figure them out right now. Despite adopting Vim in 2017, I feel completely at home with Vim 7.x+ (2006-ish), so it's all the same to me. (If C/C++ filetype/indent/syntax improved, like being able to parse C99, that would be something…)

The main blocker is that Vim 9.1 breaks Windows XP support, enough that it needs to be patched, and I still need to write that patch. Since Vim's Windows support is solid, this can be worked out natively in w64devkit itself, in case anyone would like to take a crack at it using the i686 release (on any Windows system, as the problems thankfully appear at compile time).

Vim 9.0 was the last release before Bram passed away. My understanding is that despite the size, age, and scope of the project, he didn't delegate much, leaving contributors scrambling to re-organize. They lost access to the original FTP server (that's why it still only lists 9.0), and releases are now distributed differently. In effect it's been like a fork. When 9.1 was announced, the website didn't list a source tarball at all, and it suggested there would be no further source artifacts, which made me hesitate. That has since been corrected.

If going forward requires selecting a fork, then I figure I should also consider the other obvious fork, Neovim. I haven't put any time into this, partly because I speculate serious problems: New dependencies, non-trivial cross-compilation (CMake), and probably hopeless regarding XP support.

While I'm at it, I'll mention a few extras. The recent disorganization is nothing new, and Bram was never particularly organized about the project:

• The sources page was last updated in 2018, listing 8.1 as the latest release: https://www.vim.org/sources.php

• DNS has been busted for as long as I can remember: https://vim.org/

• For as long as I can remember, the official download page has hosted malware ads with fake download buttons (screenshot taken today):

[avih]

Thanks for the info. I know that Bram passed away, but I wasn't aware the releases are in some dissarray - and I now see the latest official release seems to be 9.1 from 2024.

There are a couple light blockers, and I'm not so personally motivated to figure them out right now.

Fair enough. Obviously if that's not a trivial version change at the docker file then someone needs to put in the time to handle it.

EDIT: another option is to build the latest vim for the 64 bit build, and 9.0 for the 32 bit build. I'd think that should be relatively simple to handle?

For what it's worth, one feature which personally I find very useful, and which is available at the latest vim (tested from https://github.com/vim/vim-win32-installer/releases which is linked from the main vim website) - but not in vim from w64dk, is support for mouse wheel scroll on Windows.

This works apparently out of the box on Windows even without a ~/.vimrc config (vim -u NONE myfile.c - then scroll the mouse wheel).

It's not an issue to grab a more recent vim from elsewhere for more vim/xxd features, like the link I posted (which I think is what I'll do, now that I know recent vim has mouse wheel scroll).

So this is just an FYI that there are at least two nice improvements with recent vim compared to the w64dk build, though I guess there are more.

[PNBRQK]

Neovim has a nasty point that it requires new Windows API functions which are even absent on Windows 7 (let alone XP).

[Peter0x44]

In other improvements, I've started using some c23 features (#elifdef, nullptr) and vim 9 doesn't highlight them properly. This is fixed upstream.

Not particularly important, I only mention it as a minor motivation.

Also, RIP to Bram. Vim's flexibility and polishedness is still unmatched, IMO.

[Peter0x44]

I managed to get vim 9.1 compiling using w64devkit i686, it seems some features that are unavailable on windows XP were assumed to be present. There are a few things to patch, but it's not that much. I don't know if I did it all correctly, though.

They are:

os_win32.c: In function 'decode_mouse_wheel':
os_win32.c:1542:49: error: 'MOUSE_HWHEELED' undeclared (first use in this function); did you mean 'MOUSE_WHEELED'?
 1542 |     int     horizontal = (pmer->dwEventFlags == MOUSE_HWHEELED);
      |                                                 ^~~~~~~~~~~~~~
      |                                                 MOUSE_WHEELED

Inside winbase.h, it's defined like this:

#if (_WIN32_WINNT >= 0x0600)
#define MOUSE_HWHEELED 0x8
#endif

I just defined it myself in os_win32.c. There are a few calls to GetFinalPathNameByHandleW in resolve_reparse_point:

os_mswin.c: In function 'resolve_reparse_point':
os_mswin.c:1808:12: error: implicit declaration of function 'GetFinalPathNameByHandleW' [-Wimplicit-function-declaratio ]
 1808 |     size = GetFinalPathNameByHandleW(h, NULL, 0, 0);
      |            ^~~~~~~~~~~~~~~~~~~~~~~~~

I simply stubbed the implementation out, and made it return NULL.

The last error was:

iscygpty.c: In function 'is_cygpty':
iscygpty.c:119:33: error: 'FILE_NAME_INFO' undeclared (first use in this function); did you mean 'CERT_NAME_INFO'?
  119 |         const int size = sizeof(FILE_NAME_INFO) + sizeof(WCHAR) * (MAX_PATH - 1);
      |                                 ^~~~~~~~~~~~~~
      |                                 CERT_NAME_INFO

This function is:

// Check if the fd is a cygwin/msys's pty.
int is_cygpty(int fd)
{
#ifdef STUB_IMPL
    return 0;
#else
    HANDLE h;

Conveniently, there's already a define present to just stub it out, so I just defined it. Judging by the comment, it doesn't matter for w64devkit anyway.

I did not try the resulting build on Windows XP, I hope there are no runtime problems. I don't currently have the means to test, so I will leave that to someone else.

[Peter0x44]

I'd be happy to submit a patch, but unfortuantely I don't understand the hunks are generated, and how to use diff to do that. If @skeeto could tell me some insights on that, I'll do it.

[Metaa4245]

I'd be happy to submit a patch, but unfortuantely I don't understand the hunks are generated, and how to use diff to do that.

WinMerge?, or using diff

# -r recursive compare any subdirs, -N treat absent files as empty, -a treat as text, -u output NUM (default 3) lines of context
diff -Naru file_or_dir_original file_or_dir_new > patch.patch

[skeeto]

Thanks for digging in, @Peter0x44! Quilt shines here, but unfortunately it's Bash and Perl, so porting to native Windows is impractical. (I like it so much I've considered rewriting a more portable implementation.) To generate the patch purely with w64devkit, unpack a fresh Vim 9.1 source tree adjacent to the source tree you modified, then one level up:

$ diff -ruN a/ b/ >vim-9.1-windows-xp.patch

Here I've named them "a" and "b" to match Git's naming style, which is the style I prefer. I usually cut down on context (-U1 or -U2), which overall reduces "merge" conflicts, with trade-offs. I don't like that diff adds file timestamps, which "git diff" doesn't do, and the most practical way to remove them is editing the final patch by hand. "git diff" appends a context line (i.e. what function is being changed) to the hunk headers, which is great when reviewing code, but less important here. I'll probably regenerate your patch after I accept it, so don't worry about fixing it up.

Alternatively, since you probably have Git installed anyway, just check the pristine Vim 9.1 into Git, commit your changes, and "git show" to generate a patch with the commit message. Or even "git format-patch" for something very formal. Alternatively start from the Vim Git repo on the proper tag. Though note that the upstream source artifact is not straight out of a Git commit, but generated through "make unixall". That shouldn't affect this patch, though.

[avih]

"git show" to generate a patch with the commit message. Or even "git format-patch" for something very formal.

Can the output of git show be applied to some git repo? I don't thitnk I'm familiar with a git command for that.

The two methods I know (other than diff/patch) to generate a git diff which can be applied are either git diff [maybe-some-commit] > my.diff which can then be applied using e.g. git apply < my.diff, or git format-patch HEAD~N --stdout > my.patch (where N is the number of commits to inlcude), which can then be applied e.g. using git am < my.patch.

But largely this is probably inconsequential. It seems the required diff is small (nice work!), so any form would be fairly trivial to apply or automate.

[skeeto]

Can the output of git show be applied to some git repo?

"patch" ignores everything before the patch header (+++/---), so patches may begin with free-form prose explaining their purpose. That means piece of text can simultaneously be a mail message and a patch, and this effect is used by old-school open source mailing lists (e.g. the kernel mailing list). Take a close look and notice that patches aren't "attached" (in the MIME sense), but simply dumped into the message body below the prose. The whole message, headers and all, can be applied as a patch.

Following this, the output of "git show" is a valid patch. It doesn't preserve Git metadata the way "git format-patch" and "git am" do, but that doesn't matter here.

[avih]

Following this, the output of "git show" is a valid patch.

Right. Thanks. Indeed the output of git show can be applied using git apply (obviously without the commit message, because git apply applies a diff and not a git commit), but git am refuses to take it, and I think that's what I recalled. Thanks again.

[skeeto]

Digging into some of the details you found, looks like Vim has officially dropped XP support starting in Vim 9.1, with a decision made in September 2022:

https://github.com/vim/vim/commit/27b53be3

From what I see in that commit, even after fixing compile-time issues, there are probably more at run-time. Without upstream support, it will be difficult to maintain XP support here going forward, requiring ever more extensive patching with each Vim release, effectively forking it. It seems I'll need to stick with Vim 9.0 until w64devkit drops XP support — which will happen eventually, after more components in the distribution drop XP support. Vim 9.1+ is just not important enough to do that now.

As an alternative, if there are Vim 9.1+ features you'd like, perhaps we can backport them as patches. That's more practical than patching for XP. In the simple cases the relevant commit is the patch as-is. For instance, upstream C23 changes should somewhere in here:

$ git log -p --reverse v9.0.0000..master -- runtime/syntax/c.vim

I'm not seeing anything about #elifdef nor nullptr, though. But suppose you really want syntax highlighting for __inline:

$ git show 63c39e4e >path/to/w64devkit/src/vim-__inline.patch

Also COPY and "RUN cat vim-*.patch | patch -p1" directives in Dockerfile to pick this up, like other tools already have.

[avih]

Without upstream support, it will be difficult to maintain XP support here going forward

Agreed.

It seems I'll need to stick with Vim 9.0 until w64devkit drops XP support

As an alternative, if there are Vim 9.1+ features you'd like, perhaps we can backport them as patches.

Personally I don't think this is worth the effort. Nevertheless, the features which I find most useful so far in upstream is mouse wheel scroll support, and colors in xxd, but I don't see the value in putting (my) time into bvackporting it when I can use an upstream binary. I did initially compile only xxd, but when I got the whole upstream vim, then I use its xxd as well.

I can think of two alternatives to this:

1. Drop vim completely in w64dk. Windows vim (and other editors) releases are available elsewhere. I've switched to upstream vim binary, and so far (few days) been very happy with it. I also use vscode and Notepad++ frequenctly, depending on context.
2. Keep the 32 bit w64dk vim at version 9.0 without further updates, and track upstream with the 64 bit build. I'd think this shouldn't require meaningful effort, other than "standard" level of effort when updaing to a new version.

[Peter0x44]

As an alternative, if there are Vim 9.1+ features you'd like, perhaps we can backport them as patches.

I don't think this is worth the effort, I didn't realize XP had been dropped deliberately. That's a disappointing outcome, I don't understand the modern obsession of deleting code you don't "personally" need for its theoretical "maintainance cost". The commit really did not simplify much code in a useful way...

Does simply reverting this commit work correctly, as of now? It doesn't seem likely upstream will be convinced to keep this support, but an editor being the "straw breaking the camel's back" on this issue feels too anticlimatic.

[Peter0x44]

Not related to vim or w64devkit, but another case I noticed that dropped windows XP support recently: https://github.com/glfw/glfw/issues/2505