Support for cygwin?

[1o1o1]

checking whether C compiler accepts -Wextra-semi-stmt... no
checking whether C compiler accepts -Wimplicit-int-conversion... yes
checking whether C compiler accepts -Wnull-dereference... no
checking that generated files are newer than configure... done
configure: creating ./config.status
config.status: creating Makefile
config.status: creating htop.1
config.status: creating config.h
config.status: config.h is unchanged
config.status: executing depfiles commands

****************************************************************
WARNING! This platform is not currently supported by htop.

The code will build, but it will produce a dummy version of htop
which shows no processes, using the files from the unsupported/
directory. This is meant to be a skeleton, to be used as a
starting point if you are porting htop to a new platform.
****************************************************************

  htop 3.2.1

  platform:                  unsupported
  os-release file:           /etc/os-release
  (Linux) proc directory:    /proc
  (Linux) openvz:            no
  (Linux) vserver:           no
  (Linux) ancient vserver:   no
  (Linux) delay accounting:  no
  (Linux) sensors:           no
  (Linux) capabilities:      no
  unicode:                   yes
  affinity:                  no
  unwind:                    no
  hwloc:                     no
  debug:                     no
  static:                    no

[22:05][admin@Windows2ooo:~/htop]#

[BenBE]

It would be much better to make this compile native. The necessary code changes required to achieve this are manageable (but not trivial).

[dnabre]

If all you want to should up are Cygwin processes, this is wouldn't take all that much work (ignoring all the Linux stuff like sensors and the like).

If you wanted htop to display Windows process information, that's a whole another story all together. You'd have to operate as a Windows binary to start with, instead of going through Cygwin, and pull all the information about processes from Windows API. That's a sizable amount of work, but the real issues come into play when you compare the Unix/Linux process model to the Windows one. Something stuff just doesn' map. Things like killing processes, process trees, and signals don't translate directly either.

[BenBE]

If all you want to should up are Cygwin processes, this is wouldn't take all that much work (ignoring all the Linux stuff like sensors and the like).

I don't think it would make much sense to have this run with Cygwin. The only sensible way IMHO is actual native support.

If you wanted htop to display Windows process information, that's a whole another story all together.

Yes and no. The main issue is less with getting the information, but getting the current build system to work on Windows (AutoConf doesn't like MSVC as MSVC doesn't run C99 compliant by default, but needs a switch to do so which you can't get passed properly).

You'd have to operate as a Windows binary to start with, instead of going through Cygwin, and pull all the information about processes from Windows API.

The Windows API is quite straight forward on this. Been there, done that.

That's a sizable amount of work, but the real issues come into play when you compare the Unix/Linux process model to the Windows one.

Something stuff just doesn't map. Things like killing processes, process trees, and signals don't translate directly either.

Killing processes does (TerminateProcess/TerminateThread), process trees too (there are parent processes and such).

Signals don'T map directly, but that's no real issue for htop either.

As said: the problem for Windows support is with getting the build system to work …

[dnabre]

My understanding of the Windows side of things doesn't have any practical experience behind it, so I readily concede to anyone that has worked with. Unfortunately when you learn it without actually using it, you end up reading things that focus heavily on the differences. Can definitely see enough translating for it to still be useful. Grabbing process information is easy enough, shoehorning into the small boxes is where my understanding starts to faulter.

As for the build system, I have dealt with that pain. Have you ever considered cross compiling? There is a decent number of OSS projects that cross-compile their Windows binaries from Linux, so they can use the same tooling for building. It lacks elegance, but gets the job done.

[BenBE]

My understanding of the Windows side of things doesn't have any practical experience behind it, so I readily concede to anyone that has worked with. Unfortunately when you learn it without actually using it, you end up reading things that focus heavily on the differences. Can definitely see enough translating for it to still be useful. Grabbing process information is easy enough, shoehorning into the small boxes is where my understanding starts to faulter.

I come from the Windows side of things, so got quite some experience on that part. And yes, there will be some translation work necessary, but nothing that can't be solved with some clever thinking. For an initial step getting the "dummy/unsupported" platform build for Windows would be a huge step forward. Going further from there is manageable.

As for the build system, I have dealt with that pain. Have you ever considered cross compiling? There is a decent number of OSS projects that cross-compile their Windows binaries from Linux, so they can use the same tooling for building. It lacks elegance, but gets the job done.

Would you like to document the steps necessary to get htop to cross-compile (no need to also implement the data collection part; doing the unsupported platform stuff suffices)?

[dnabre]

I'll see what I can put together this weekend .

[dnabre]

Have a write up of the basic process for setting up and doing the cross compile:

https://github.com/dnabre/htop/tree/windows/docs/cross_compiling_for_windows

MingW64 gives all the basic stuff, but one can pull in the entire Windows SDK if needed. That takes a few more steps, but is doable. Don't think it'll be necessary.

Gets me to where parts of the htop code (outside of the platform specific generic/etc folder) will be needed. The COVID booster I got Thursday is hitting me pretty hard, so I didn't get beyond that.

I should be able to add a new platform folder, 'windows' based on 'generic', to the autoconf stuff, going by example of what's there. Then fill in enough of a platform folder to test that I get a solid Windows executable out the end of everything and pull some basic process info from Windows to make sure enough of the Windows API is there.

There is some platform specific stuff in the root folder (Action.c for example uses stuff from pwd.h like getpwnam) that will need to be split out, not sure where to put platform-specific changes to the root folder's code. Though I haven't looked though all of it.

When stuffing this into my fork of the repo, I accidently made a pull request to the main repo. I closed the request, and deleted the branch attached to the request, but not sure if there is anything more to undo it as much as possible. Me and git have never gotten along well.

edit Build errors when built with the cross-compiler. After removing the uts_name.h requirement removed configure.ac so that configure will finish. htop_build_error.log

[BenBE]

Have a write up of the basic process for setting up and doing the cross compile:

https://github.com/dnabre/htop/tree/windows/docs/cross_compiling_for_windows

Thank you for this initial write-up. That's a good starting point.

As I'd suggest to avoid putting binary file formats into the repository, I'd prefer if we could make the markdown version of the build instructions the one canonical source.

MingW64 gives all the basic stuff, but one can pull in the entire Windows SDK if needed. That takes a few more steps, but is doable. Don't think it'll be necessary.

I think for the most basic setup the MingW64 stuff should suffice for a first trial run.

The COVID booster I got Thursday is hitting me pretty hard, so I didn't get beyond that.

Hope you'll recover quickly. :)

Gets me to where parts of the htop code (outside of the platform specific generic/etc folder) will be needed.

The generic folder is actually a somewhat "cross-plattform" folder. It's intended for code that can be used on multiple (similar) platforms, but not all of them. One example is for reading OS version information, which is quite similar on nearly all platforms, except a few ones. So basically, files in the generic folder are – in contrast to the actual platform folders – only compiled on a file-by-file basis as applicable for that target platform. If Windows can't use any of those sources, it should not build any of those files, but instead provide its own implementations in its platform folder instead.

I should be able to add a new platform folder, 'windows' based on 'generic', to the autoconf stuff, going by example of what's there. Then fill in enough of a platform folder to test that I get a solid Windows executable out the end of everything and pull some basic process info from Windows to make sure enough of the Windows API is there.

As the kernel is actually called WinNT for most current Windows releases I'd suggest going with winnt for the platform name/folder. This would align with win16 (Windows 3.1/3.11) and win32 (Windows 95/98/Me).

There is some platform specific stuff in the root folder (Action.c for example uses stuff from pwd.h like getpwnam) that will need to be split out, not sure where to put platform-specific changes to the root folder's code. Though I haven't looked though all of it.

This should likely be split to the generic folder and included from there for all platforms that need it.

When stuffing this into my fork of the repo, I accidently made a pull request to the main repo. I closed the request, and deleted the branch attached to the request, but not sure if there is anything more to undo it as much as possible. Me and git have never gotten along well.

I personally don't mind a PR for this issue existing. Probably marking it as draft for now would be warranted.

From a cleanup point of view:

• In your Markdown version of the build instructions you might want to format code blocks properly. They are hard to read right now.
• The platform folder itself should not include an configure.ac; all the changes from there should ideally be integrated in the main file.
• The build error log should not be part of the (final) source tree.
• The issues in Compat.c should be resolved by providing fitting implementations with the Windows API; testing for the HTOP_WINNT define for includes and so on.
• The stuff that references pwd.h should likely be moved to generic; with the winnt platform providing an alternate implementation for it.

An initial working draft with EnumProcesses/EnumThreads should be fine. Doesn't need to support all of the htop features in the first demo.

[dnabre]

Hadn't meant to sent that PR to the main repo, just push the commit to my fork (github.com/dnabre/htop). So files/format was my working versions. On the same page as you in terms of binary formats, and files that should be other places. Just meant to dump all my working files to my fork of the repo so that a copy existed outside of my computer. Didn't mean for a PR to go to the main repo , especially raw like that.

I might be clumsy with git, but understand what sensibly should be in the repo. Only thing that I'd intended stick in the main repo's docs would be a the markdown version (which needs a clean/formatting pass minimally) and the scripts documenting how to do the cross-compile/build.

In terms of the rest and source layout, my understanding:

• I'll do a clean of that markdown, and resubmit a PR with just what should be in docs folder. Error log might a useful reference, but that is best attached to something in the issue tracker.
• I hadn't seen Compat.c yet, need to go through make make sure I'm familiar what the main folder files are. I'll likely look through and write short sentence of what each does for my own reference. Don't know if those notes (cleaned up as a text file) would be a useful reference to repo map for other.
• Compat.c looks like where most of the ifdef(platform_X) stuff should go. Other root files should be only be ifdefs on features.
• Stuff like pwd.h that conflicts with windows way of doing something should be moved down to the generic folder.
• Windows specific stuff should go in platform subfolder, winnt .

That's the level of complexity I was thinking in terms of a working draft/proof of concept.

[dnabre]

In terms of Cygwin support specifically. Processes in Cygwin are just normal Windows process with extra stuff that Cygwin tracks (not familiar with those details). With a native Windows htop, you'll see all processes. Some of them will just happen to be Cygwin processes. If you look at Task Manager in Windows you can see the individual Cygwin processes and information about them. Cygwin has extra Linux-type information about them, but they are still running on Windows.

If you want htop just for Cygwin processes, that should be doable, I'd think. htop alread builds on Cygwin with place-holder stuff. You just need to put in the code so that htop pulls in the process information through Cygwin. Check the Cygwin code for ps/top to see exactly how to get that information.

Personally, that limited Cygwin-only version of htop isn't very interesting to me. Especially as more and more OSS projects run natively (not through Cygwin). I doubt anyone would be opposed to someone scratching that itch if they were so inclined.

[ghost]

sorry, any update?

yesterday i have started my attempt adding the cygwin support just for fun ( forked repo is here )

so far i've got some meters working but not yet about process listing:

it builds and runs on both cygwin (old ver, host: win vista x64) and msys2 (host: win11 x64)

i borrowed many code from existing bsd/linux support

(looks like the procfs scanning code can be largely borrowed from the existing linux support as well, since cygwin has the similar procfs structure like linux's

tomorrow i may try to get some progress on the process listing functions

and maybe i won't try adding any native windows or windows process support other than showing winpid for cygwin processes due to the complexity

also i think adding cygwin support won't block anyone adding native windows support (or the ability to work with windows processes) in the future, so why not?

top is already working on cygwin, but i am more used to htop. it may not be as useful as on other platforms, but i think it's not meaningless to me.

cygwin may not have that many users and use cases nowadays, but i think msys2 has. some POSIX components it provides rely on cygwin, and it's actively maintained as a platform for porting software to native windows.

(msys2 with mingw is probably suitable for compiling native build of this project with the existing automake build system directly on windows.)

by the way, i did found a htop alternative for windows which is probably useful: https://github.com/gsass1/NTop

[ghost]

good news, i've got the process listing working as expected (showing cygwin processes):

[ghost]

also tested on the original cygwin:

[BenBE]

Unfortunately not much progress here, as WinNT support is more of a side project. With that said, there's some initial documentation for cross-compiling using MinGW at #1107. This is also preferable over Cygwin, as Cygwin requires a separate runtime, which MinGW does not. Furthermore there's a severe performance penalty associated with Cygwin, which will bite you when a lot of files need to be read (there are issues even on Linux regarding htop requiring too much CPU, cf. e.g. #1211).

The downside with the MinGW approach (and partially why it hasn't been put forward much yet) is that some functions in the platform-neutral code assume a POSIX environment with e.g. unistd.h being present. Patches to fix this are welcome.

P.S.: Last time I tried to build things on Cygwin I ran into issues with autotools (in particular the generated configure) tried to assess C99 compatibility of MSVC* for a truely native build. How did you work around this?

*MSVC is C99 compatible, but doesn't advertise so unless a special flag is used, which was interpreted as a path by Cygwin when trying to start the compiler, thus screwing things up.

[ghost]

This is also preferable over Cygwin

i think this port is meaningful to actual cygwin and msys2 users, since people will need a tool to manage those cygwin (or msys2) processes

and it will be harder to access cygwin internal information if using win32 apis, i think we need a seperate platform support for windows processes, using native apis so won't need cygwin or msys runtime. in that case, just consider cygwin processes as any other normal windows processes, however, my attempt was just a simple cygwin port without thinking about too much about the host processes

Cygwin requires a separate runtime, which MinGW does not

i think it's reasonable, cygwin need to maintain its global and internal states which are shared between all running cygwin processes in the same install

after all, my attempt was a cygwin port, unfortunately not mingw or native port...

Furthermore there's a severe performance penalty associated with Cygwin, which will bite you when a lot of files need to be read (there are issues even on Linux regarding htop requiring too much CPU, cf. e.g. #1211).

unfortunately cygwin prefers compatibility and not performance first...

actually the performance is not that bad, here is a test on windows vista with xfce running in bg (so there a few background processes in the list to poll)

it still runs quite fast, although with periodically lags, but it's also the case on linux, just more obvious on cygwin due to its compatibility layer (the environment variables list is working in later builds):

https://github.com/htop-dev/htop/assets/103345643/7eac1906-b243-4a40-a722-4d9cf261a24e

it somehow runs much slower on my main pc running win11 with msys2, maybe it's caused by the extra handling in msys2, or maybe it's my pc or os issue (like the anti-virus software or some other quirks of windows...

[ghost]

P.S.: Last time I tried to build things on Cygwin I ran into issues with autotools (in particular the generated configure) tried to assess C99 compatibility of MSVC* for a truely native build. How did you work around this?

i didn't make a native build, but it should work fine with mingw toolchain. and sorry, i am not familiar with msvc so i dont quite understand the problem you mentioned..

[BenBE]

P.S.: Last time I tried to build things on Cygwin I ran into issues with autotools (in particular the generated configure) tried to assess C99 compatibility of MSVC* for a truely native build. How did you work around this?

i didn't make a native build, but it should work fine with mingw toolchain. and sorry, i am not familiar with msvc so i dont quite understand the problem you mentioned..

MSVC (Microsoft Visual C compiler) is the native toolchain from Microsoft for development on Windows. Problem is that MSVC is (nearly) C99 compliant by default, but does not report support/being compliant unless you explicitly ask for it. Autotools (configure) has a check for if the compiler needs some extra flags for "compliant C99 support" (-std=c99 for GCC BTW, because default is ANSI C89). You can try to hack the configure script to include this flag in the check and if you were to execute the check properly on cmd.exe you'd receive the correct output of compliance from MSVC, BUT if running the configure script from Cygwin or MSys with the proper environment variables set up for MSVC the configure script runs into a problem that the necessary flag looks like /C99, which both Cygwin and MSys translate to C:\C99, thus breaking the configure script. And there's also no way to override this one exact parameter to be left alone, thus you can't run configure on Windows inside Cygwin/MSys to create the proper build files. And running configure from cmd.exe just plain fails due to the missing POSIX environment.

If you somehow would convince configure to run (some hacking stuff involved) you'd currently still end up with some compile issues in the platform-neutral code, as that code uses POSIX features, which aren't part of the main WinAPI (but Windows has a POSIX subsystem itself).

Re MinGW: That's what the PR I linked to also uses for cross-compilation. That solves the issue with configure, but fails at the POSIX API issue in the code, as mentioned.

[ghost]

BUT if running the configure script from Cygwin or MSys with the proper environment variables set up for MSVC the configure script runs into a problem that the necessary flag looks like /C99, which both Cygwin and MSys translate to C:\C99, thus breaking the configure script.

maybe this is caused by the path conversion of msys2 (but i am not sure about cygwin), you can bypass it by specifying //C99 (double slashes) instead, or setting a special environment variable like MSYS2_ARG_CONV_EXCL="/C99" to bypass it without the build system being aware of it

further reading: https://www.msys2.org/docs/filesystem-paths/#automatic-unix-windows-path-conversion

[ghost]

sorry for the confusions i caused...

[ghost]

maybe this is caused by the path conversion of msys2 (but i am not sure about cygwin)

by the way are you using msvc for the build or sth? if not, maybe it's a common issue when building native mingw project using automake, so i think try reading their document or building some known working packages for testing may help..?

also you shouldn't use cygwin if you want to start a native port, the old -mno-cygwin flag of gcc was deprecated and removed since a long time ago, and their mingw cross toolchain is unmaintain so it's outdated

for that you should prefer msys2, it also has more development library packages for native mingw shipped so you can install them via pacman and use them more easily

[BenBE]

Your build already looks impressive from what I've also seen in the PR.

And while there sure is some need for Cygwin/MSys support I'm still a bit hesitant compared to having a fully native build. As Cygwin/MSys try to emulate a POSIX-like environment it begs the question where exactly they fall short from being compatible with the current Linux implementation (asking re code duplication). In particular because if they were to emulate procfs faithfully, you normally should be able to run a feature-reduced version of the Linux platform code (that's actually what NetBSD users have been doing for quite a while until native support for NetBSD was added to htop).

To be frank, what I most dislike about both Cygwin/MSys is their added overhead due to emulating a full environment on top of the usual C runtime. I'd accept if you could go ahead and compile code in the Cygwin/MSys environment, but have it still run native without extra dependencies on the translation layer of that environment unless I explicitly request it to (or for those kinds of tools more like an opt-out). Every translation layer takes up processing, and particularly for procfs there's a whole bunch of code in the Linux kernel that just aims at speeding up the generation of those synthetic files. For Cygwin/MSys you intercept every I/O operation and need to check for all those special cases, which doesn't exactly benefit doing this fast; especially with all the translation of paths/references/mountpoints in between. If there'S a chance for avoiding that translation in between altogether I'm all for it.

I'm also not entirely against having a Windows version for htop – and mentioning the Cygwin/MSys process emulation as a target is a valid usecase –, but I think it'd be nice if the focus could be on MinGW support as that is much faster (from experience) and closer to a real native experience. And if that entails cross-compiling htop from Linux for now due to build system issues, I can fully live with that.

FWIW: @ookiineko With other platforms there is somewhat an arrangement of a person responsible for maintenance that the core team can consult if questions with the code arise (quite rare, but somewhat long-term commitment). To add cygwin support I'd ask if you'd like to take up that role for cygwin/msys if we were to go ahead with adding this platform support.

[ghost]

thank you for replying

I'd ask if you'd like to take up that role for cygwin/msys if we were to go ahead with adding this platform support.

i'm not a cygwin or msys contributor, so i'm probably not good enough for it, and i only plan to use this port when i'm on windows (i may go inactive when i'm switched to linux).

i was just sharing my progress and code patches, as well as adding some basic support as i could for this platform if possible

[BenBE]

maybe this is caused by the path conversion of msys2 (but i am not sure about cygwin)

by the way are you using msvc for the build or sth? if not, maybe it's a common issue when building native mingw project using automake, so i think try reading their document or building some known working packages for testing may help..?

also you shouldn't use cygwin if you want to start a native port, the old -mno-cygwin flag of gcc was deprecated and removed since a long time ago, and their mingw cross toolchain is unmaintain so it's outdated

for that you should prefer msys2, it also has more development library packages for native mingw shipped so you can install them via pacman and use them more easily

I was testing with MSys2 mainly due to native cmd.exe unable to run autotools (automake, autoconf, …). From there the goal was to perform actual compilation with MSVC (AKA the native development tools for Windows). GCC/MinGW was not involved in my trials I did on Windows.

For cross-compilation I ran a MinGW-Win32 cross-compiler toolchain on Linux (Ubuntu), cf. #1107/#1114.

The results were:

• On Windows (MSys2): Could't get configure to run properly and accept MSVC for compilation
• When Cross-compiling: configure ran, but issues in the htop platform-independent code cause build failures.

I think, with the information provided above the compilation issues on Windows should be able to be addressed/worked around.

The issues mentioned regarding the cross-compilation (unistd.h being used) are likely to be unaffected by this and need to be addressed separately. Those issues will also affect the MSVC build regardless of the build system issues.

[ghost]

The results were:

• On Windows (MSys2): Could't get configure to run properly and accept MSVC for compilation
• When Cross-compiling: configure ran, but issues in the htop platform-independent code cause build failures.

i still recommend giving it a try with msys2/mingw and gcc instead of msvc, or they also have a clang toolchain if you prefer that. i guess msys2 should provide working autotools if you are in the correct terminal (MSYS2 MINGW64) and have correct packages installed (like mingw-w64-x86_64-toolchain and mingw-w64-x86_64-autotools)

but some effort may be needed for the POSIX api parts on mingw, like adding lightweight stubs or wrappers for those stuffs (this method is used by many projects for keeping the code clean for other platforms) and add alternate implementation for mingw where that strategy isn't optimal

just my thoughts

[ghost]

i was just sharing my progress and code patches, as well as adding some basic support as i could for this platform if possible

as i said before it was just for fun, but i saw this on stackoverflow:

https://stackoverflow.com/questions/34118879/building-htop-from-source-in-cygwin

actually i had this idea and found that answer since long ago but didn't really try to make a port

recently i found it's useful to me because personally i'm using msys2 more and more frequently

[dnabre]

Cygwin binaries are native Win32 binaries but they require its runtime library cygwin1.dll to run. This is what provides the translation code give POSIX features (actually Linux features are specifically their goal). Note that using the POSIX/Linux style process information functions here will only give you processes which are running through that DLL. This would be a Cygwin-only version, which may be useful to some people, but is pretty limited.

MSYS2 provides several environments from its base installation, which can be rather confusing to navigate. The 'MSYS2 - MSYS' environment does the same sort of things as Cygwin, with its runtime msys-2.0.dll, same limitation as Cygwin. Note that neither of these two environments see the others processes. It also provides 'MSYS2 - MINGW64' environment which lets you build completely native Windows executable which use its runtime msvcrt.dll.

A good illustration is looking at the dynamic links of a basic "Hello World\n" program build using each of the above:

$ ldd *.exe
x86_64-pc-cygwin-gcc_hello.exe:
        ntdll.dll => /c/WINDOWS/SYSTEM32/ntdll.dll (0x7ffbeabb0000)
        KERNEL32.DLL => /c/WINDOWS/System32/KERNEL32.DLL (0x7ffbe94b0000)
        KERNELBASE.dll => /c/WINDOWS/System32/KERNELBASE.dll (0x7ffbe8210000)
        cygwin1.dll => not found
x86_64-pc-msys-gcc_hello.exe:
        ntdll.dll => /c/WINDOWS/SYSTEM32/ntdll.dll (0x7ffbeabb0000)
        KERNEL32.DLL => /c/WINDOWS/System32/KERNEL32.DLL (0x7ffbe94b0000)
        KERNELBASE.dll => /c/WINDOWS/System32/KERNELBASE.dll (0x7ffbe8210000)
        msys-2.0.dll => /usr/bin/msys-2.0.dll (0x180040000)
x86_64-w64-mingw32-gcc_hello.exe:
        ntdll.dll => /c/WINDOWS/SYSTEM32/ntdll.dll (0x7ffbeabb0000)
        KERNEL32.DLL => /c/WINDOWS/System32/KERNEL32.DLL (0x7ffbe94b0000)
        KERNELBASE.dll => /c/WINDOWS/System32/KERNELBASE.dll (0x7ffbe8210000)
        msvcrt.dll => /c/WINDOWS/System32/msvcrt.dll (0x7ffbeaac0000)

The last, which just relies on standard Windows libraries is unquestionably the best for the platform. Beyond distribution of libraries and stuff, since everything is going through the Windows base system, you see all processes, even those running under Cygwin (a filter specific for these would be a useful feature to add though).

This all aside from the building/testing/distribution issues. This actually is more of an issue than actually modifying the code and making a binary on Windows that works. Being able to automate building the Windows binary preferably if not necessarily on Linux, is really a requirement. If the Windows version can't be rolled into automated testing workflow, it will break as changes get made elsewhere in the code.

My proposed solution was to cross-compile the native Windows binary from Linux (See #1114 ) . I started some instructions on setting up the build environment. It ended up having a roadmap of changes, with input from @BenBE about where/how to layout source files and hook into the autoconf setup.

Real life, COVID, then post-infection Long Covid, killed my ability to actually go anywhere with that, but it's a decent starting point and worth a read if you want to go in that direction. I think the document was finished and all issues (identified at the time) were wrapped up, but I haven't gotten around to a going back and doing a final review on it.

[ghost]

Cygwin binaries are native Win32 binaries but they require its runtime library cygwin1.dll to run. This is what provides the translation code give POSIX features (actually Linux features are specifically their goal). Note that using the POSIX/Linux style process information functions here will only give you processes which are running through that DLL. This would be a Cygwin-only version, which may be useful to some people, but is pretty limited.

i am aware of that... i believe every msys or cygwin user is aware or at least should be aware of that, you can even find that statement on cygwin official site, right on the home page you will see before even downloading cygwin.

it is useful for all msys and cygwin users, since they are already using the unix-like environment.

i understand htop need a native windows port, but i am unfortunately not familiar with those stuffs so i cannot help either...

MSYS2 provides several environments from its base installation, which can be rather confusing to navigate.

indeed. but in my case, msys2 refers to the cygwin fork. i use word mingw to refer to it's other environments. in other cases msys2 may also refer to the whole distribution but i dont use it in that way because i think it causes the confusion

The 'MSYS2 - MSYS' environment does the same sort of things as Cygwin, with its runtime msys-2.0.dll, same limitation as Cygwin. Note that neither of these two environments see the others processes. It also provides 'MSYS2 - MINGW64' environment which lets you build completely native Windows executable which use its runtime msvcrt.dll.

because msys2-runtime is a fork of cygwin, with minimal modifications for working better with native programs such as auto path conversions, changing symlink behavior and making chmod/chown no-op, etc.

Real life, COVID, then post-infection Long Covid, killed my ability to actually go anywhere with that

i'm sorry to hear that :(

please take care of yourself

[ghost]

i believe this issue is mostly about adding some initial support for cygwin (this is probably also any actual cygwin user including OP will care first), and the ability to list/operate native windows proccesses should be a different topic.

and a native windows port doesn't have that much to do with cygwin, neither with this issue.

(my english is not so good, sorry for any confusion i may have caused

[BenBE]

Hi @dnabre, thank you for chiming in!

The last, which just relies on standard Windows libraries is unquestionably the best for the platform.

And I'd somewhat also prefer htop to go in that direction (caveat incoming).

Beyond distribution of libraries and stuff, since everything is going through the Windows base system, you see all processes, even those running under Cygwin

ACK. Having a check to see if processes use some translation/mapping could be handled similar to how Rosetta on Darwin is handled. Thus you could have a translation column with values "native", "WoW64", "cygwin", "msys", … Taking a look at SysInternal's Process Explorer should give even more ideas on that subject.

(a filter specific for these would be a useful feature to add though).

ACK. Might even be one of those platform-specific configuration options …

This all aside from the building/testing/distribution issues. This actually is more of an issue than actually modifying the code and making a binary on Windows that works. Being able to automate building the Windows binary preferably if not necessarily on Linux, is really a requirement. If the Windows version can't be rolled into automated testing workflow, it will break as changes get made elsewhere in the code.

ACK. @ookiineko You could include this in your CI configuration of your branch. AFAICS, Github Actions supports Windows runners. Thus both native builds as well as cross-builds could be viable configurations (I'd suggest implementing both). You could start with Cygwin for now, but switching to MinGW or MSVC would be nice to aim for later.

My proposed solution was to cross-compile the native Windows binary from Linux (See #1114 ) . I started some instructions on setting up the build environment. It ended up having a roadmap of changes, with input from @BenBE about where/how to layout source files and hook into the autoconf setup.

IIRC the instructions where easy to follow and there were few issues I noticed. Though at times you should have a good grasp of your system to understand how to continue in case of issues. I think, one reason they were not yet really merged directly into upstream was some finalization work as well as having actual work based on that.

Real life, COVID, then post-infection Long Covid, killed my ability to actually go anywhere with that, […], but I haven't gotten around to a going back and doing a final review on it.

I'm sorry to hear that. Please take care of yourself.

RE @ookiineko:

i believe this issue is mostly about adding some initial support for cygwin (this is probably also any actual cygwin user including OP will care first), and the ability to list/operate native windows processes should be a different topic.

I partially agree on one part, but disagree on some other aspects of this.

Let me explain: The question posed as part of this issue is whether or not htop should get support for a new platform. While technically this is mostly solved (we have a PR), but not the full story how this works out with further maintenance, as (cf. note by @dnabre) this has to receive some care once it's part of the code base.

Thus for any large feature, the dev team is somewhat reluctant to jump ahead and just merge it in. Instead a roadmap should be made for how that feature will develop over time and where things should go once the initial support has been merged. That's why I added the "needs discussion" tag for this issue (if the overall feature was about to be rejected you would have a clear note on that by now). BUT, because once merged the majority of the work will likely be with the core dev team, they ought to be allowed to have some word on how they intent the roadmap to look like and what needs to be done beforehand.

and a native windows port doesn't have that much to do with cygwin, neither with this issue.

Yes and no.

There's the option to go half the way with Cygwin and MSys2, or go all the way and use the WinAPI without any wrappers. Doing both is duplicated work and unlikely to happen.

Once a Cygwin implementation exists, it's not too complicated to replace those bits depending on the translation layer to use native code. Thus targeting fully native and using a wrapper in the meantime could be an interim option.

As both Cygwin and MSys2 are additional components not part of Windows itself IMHO they best should be treated as such: optional. Ideally something like how SystemD is treated on Linux would be ideal (i.e. htop doesn't require systemd (headers) for building, but if present it can interact with it and gather additional information).

NB: I've developed Windows applications for years, including stuff a bit closer to the kernel. Thus if necessary I can somewhat support in that regard. But since I haven't had a working dev system on Windows for years, it's not likely I will be adding tons of code …

(my english is not so good, sorry for any confusion i may have caused

NP. Feel free to ask if you are unsure about anything or if you need something explained differently.

Based on the comments in today's discussion I got some new information that I had been missing before. At least from that PoV it was worthwhile.

[ghost]

@ookiineko You could include this in your CI configuration of your branch.

tomorrow i will try to add both msys and cygwin CI (had done that for some other personal cygwin ports before

RE @ookiineko:

Thus for any large feature, the dev team is somewhat reluctant to jump ahead and just merge it in. Instead a roadmap should be made for how that feature will develop over time and where things should go once the initial support has been merged. That's why I added the "needs discussion" tag for this issue.

thanks for replying, i am totally agree with and understand your concern

There's the option to go half the way with Cygwin and MSys2, or go all the way and use the WinAPI without any wrappers. Doing both is duplicated work and unlikely to happen.

Once a Cygwin implementation exists, it's not too complicated to replace those bits depending on the translation layer to use native code. Thus targeting fully native and using a wrapper in the meantime could be an interim option.

cygwin port is different from a native port, because cygwin executable will always depend on cygwin1.dll or msys-x.0.dll, you cannot remove this dependency even if you have replaced all the usages of cygwin APIs with native ones from the code. although it's not ideal for windows, it's still a thing for cygwin and msys users. this is why this issue and my attempt exist.

however, the ability to identify and interact with cygwin process can still be added to a native port, but may be limited due to the incompatibilites between each others

As both Cygwin and MSys2 are additional components not part of Windows itself IMHO they best should be treated as such: optional. Ideally something like how SystemD is treated on Linux would be ideal (i.e. htop doesn't require systemd (headers) for building, but if present it can interact with it and gather additional information).

in my opinion, cygwin "should" be considered as another unix-like platform, not windows in this case, even if it's running on top of windows (that's why msys uses it to run certain POSIX stuffs to assist the build process for native programs) as a compatibility layer and able to mix with windows apis.

because it has its own compiler target x86_64-pc-cygwin (or x86_64-pc-msys) and provides its own set of POSIX compliant APIs.

you see in my previous screen recording, cygwin itself is already good enough to run X11 desktop environments like Xfce4 (or even Gnome, KDE, etc., see: https://x.cygwin.com/docs/ug/using.html, although the versions are old and packages are unmaintained).

in conclusion, cygwin users use cygwin because they prefer or need it, and when adding support for cygwin, we should treat cygwin as another platform, not windows, but adding native windows process support to it is fine.

NP. Feel free to ask if you are unsure about anything or if you need something explained differently.

Based on the comments in today's discussion I got some new information that I had been missing before. At least from that PoV it was worthwhile.

thank you, and glad i could provide some info that may be useful

[BenBE]

@ookiineko You could include this in your CI configuration of your branch.

tomorrow i will try to add both msys and cygwin CI (had done that for some other personal cygwin ports before

Looking forward. :)

There's the option to go half the way with Cygwin and MSys2, or go all the way and use the WinAPI without any wrappers. Doing both is duplicated work and unlikely to happen. Once a Cygwin implementation exists, it's not too complicated to replace those bits depending on the translation layer to use native code. Thus targeting fully native and using a wrapper in the meantime could be an interim option.

cygwin port is different from a native port, because cygwin executable will always depend on cygwin1.dll or msys-x.0.dll, you cannot remove this dependency even if you have replaced all the usages of cygwin APIs with native ones from the code. although it's not ideal for windows, it's still a thing for cygwin and msys users. this is why this issue and my attempt exist.

ACK. That's the nature of such wrappers.

I had a quick conversation in the team, and based on that there are a few ways to proceed:

1. Keep it strictly Cygwin/MSys2 (basically the current PR)
2. Based on the current PR go ahead and step by step reduce the dependencies on Cygwin/MSys2 until a version without calls to the Cygwin/MSys2 layer is reached. At that point switching over to MinGW should become possible. The effect would be that the resulting code could be used both on Cygwin and a native version (partial differences might be covered with conditional compilation).
3. Directly aim for full native support with MinGW.
4. Like option two, but split development into two target platforms.

Based on the feedback here and the other discussion the second option looks viable as it can be based on option 1 which we have. The difference between options 2 and 4 would need some investigation. But at least options 1 and 2 seem feasible.

however, the ability to identify and interact with cygwin process can still be added to a native port, but may be limited due to the incompatibilities between each others

ACK.

As both Cygwin and MSys2 are additional components not part of Windows itself IMHO they best should be treated as such: optional. Ideally something like how SystemD is treated on Linux would be ideal (i.e. htop doesn't require systemd (headers) for building, but if present it can interact with it and gather additional information).

in my opinion, cygwin "should" be considered as another unix-like platform, not windows in this case, even if it's running on top of windows (that's why msys uses it to run certain POSIX stuffs to assist the build process for native programs) as a compatibility layer and able to mix with windows apis.

Okay, let's do a small thought experiment for a moment and assume we build cygwin as it's own platform. It absolutely is possible to retrieve all necessary objects (processes, threads, system resources) that Cygwin emulates. To do so it calls some API of the OS. If we minimize calls to these functions to when they refer to derived resources (Cygwin processes vs. OS processes) we would technically still be runing on Cygwin (still with the dependencies, but possibly avoiding much of the slowdown that the emulation would cause). In this experiment we even get help from Cygwin itself as the underlying resources are exposed (e.g. WINPID for processes). Thus once we can efficiently ask the OS for it's resources, we only need to map the Cygwin specific resources onto what we already know from the OS. Based on this mapping we could skip all information gathering on Cygwin, where we know how to gather that datum from the OS layer. Basically how option 2 from above could be realized starting from a fully Cygwin implementation (also a reason I remark about the native APIs in the PR: Not necessarily directly, but for follow-up work).

What do you think of this as a mid-term roadmap?

The alternative most likely looks like implementing both options 1 and 3, which would possibly duplicate quite a bit of work.

because it has its own compiler target x86_64-pc-cygwin (or x86_64-pc-msys) and provides its own set of POSIX compliant APIs.

Windows has a POSIX layer. If we were to treat every subsystem as its own, we would have to split Windows into several platforms: OS/2, POSIX, Cygwin, MSys2, and Win32/WinNT. That's not how you should approach it if you still want to maintain the code base. Instead you should look if you can consolidate the differences and reduce the surface. Also access from one subsystem to another should be embraced: Even when starting from the point of Cygwin you could still include the data the base system provides. Especially if doing so offers significantly more performance.

you see in my previous screen recording, cygwin itself is already good enough to run X11 desktop environments like Xfce4 (or even Gnome, KDE, etc., see: https://x.cygwin.com/docs/ug/using.html, although the versions are old and packages are unmaintained).

in conclusion, cygwin users use cygwin because they prefer or need it, and when adding support for cygwin, we should treat cygwin as another platform, not windows, but adding native windows process support to it is fine.

Fighting hard to not be cynic right now.

Let's conclude (neutrally): There are valid use cases. There are users (maybe even more than some other platforms we already support). It's doable. And there are no pressing reasons not to support Cygwin.

Personally, I'd quite happily see this initial support for Cygwin to evolve such that most of its information are gathered without Cygwin/MSys2 being involved beyond where its necessary, but given where we stand this looks rather unlikely.

TL;DR: Start with some Cygwin implementation and step by step reduce Cygwin API usage by replacing it with direct WinAPI calls where ever possible, especially if this increases performance. If fully eradicating Cygwin dependencies is possible, needs to be seen.

[ghost]

1. Keep it strictly Cygwin/MSys2 (basically the current PR)
2. Based on the current PR go ahead and step by step reduce the dependencies on Cygwin/MSys2 until a version without calls to the Cygwin/MSys2 layer is reached. At that point switching over to MinGW should become possible. The effect would be that the resulting code could be used both on Cygwin and a native version (partial differences might be covered with conditional compilation).

for best posix and cygwin compatibility, cygwin apis should be preferred in the cygwin port

however, if you plan to use cygwin as starting point and adapt it to a native port, better separate it into two different variants

3. Directly aim for full native support with MinGW.
4. Like option two, but split development into two target platforms.

i am not sure about these because personally i don't have enough experience with windows apis, but with some efforts, i think a pure native port may be possible...

It absolutely is possible to retrieve all necessary objects (processes, threads, system resources) that Cygwin emulates

i didn't find any way to retrieve the thread info, cygwin is using a different thread model from linux where threads are not implement by lightweight processes. this is probably due to the windows thread model where thread is a separated concept from process, i.e. even in the main thread, TID has nothing todo with PID (on linux, TID == PID when in main thread

What do you think of this as a mid-term roadmap?

please read on

If we were to treat every subsystem as its own, we would have to split Windows into several platforms: OS/2, POSIX, Cygwin, MSys2, and Win32/WinNT. That's not how you should approach it if you still want to maintain the code base.

OS/2 and POSIX subsystems were already deprecated and dropped... and i am not sure how many of them are able to mix with windows api like cygwin in your listed platforms. Cygwin and Msys are not a subsystem, they are both running on top of the Win32 subsystem and depending on both Win32 api and some NT kernel apis

Cygwin itself is a special compatibility layer like wine, but require recompiling the POSIX programs from source to use them. it has its own set of system headers and unix-like APIs exposed in them, just like any other UNIX platform. by this design, i think it's more reasonable to treat cygwin as a standalone platform, if you see people do not, they should always have some specific reason (like only utilizing the shell utilities it provides while still calling native windows program through it), however, this is not the case when porting program like htop.

cygwin was invented to reduce the duplicated effort when mapping posix calls to win32 ones manually, so to keep this port simple and meaningful we should prefer cygwin api when possible, so cygwin's efforts are not wasted, and it fits best in the cygwin spirit.

to summarize my thoughs, i think adding native windows process listing function to the cygwin port as an enhancement is totally good (in fact Cygwin already provides this function in the ps utility it offers), but not required since Cygwin's goal is only to be as much as possible POSIX compliant.

i think replacing cygwin api calls with native windows api where not necessary (like when cygwin also has a wrapper for it) looks a bit off to me, because the ability to mix with windows apis is not required for a UNIX-like platform like cygwin. i understand the performance concerns, however, cygwin is used where performance is not a problem. (for example, msys uses it when building other programs, but doesn't use it when the native program is compiled and running)

maybe it's just me, sorry.

by the way, thanks for taking time replying and reviewing the pr. honestly i didn't expect to talk so much, at first i was just making htop work on cygwin for fun and for my personal use, sharing the progress here and hoping it's useful to some else with the same intention.

[ghost]

Cygwin CI added on the main branch

test run

[ghost]

Msys2 CI also added, test run here

[BenBE]

Maybe add pkgconf explicitly for MSys2 too, as configure tries to read some information from it when present. It's in the indirect dependencies, but I think it's warranted to have pkgconf added explicitly. Also, can you check for the dependencies to avoid installing "recommended" packages to be as explicit about dependencies as possible?

[ghost]

Maybe add pkgconf explicitly for MSys2 too, as configure tries to read some information from it when present. It's in the indirect dependencies, but I think it's warranted to have pkgconf added explicitly. Also, can you check for the dependencies to avoid installing "recommended" packages to be as explicit about dependencies as possible?

OK, i will check that later

[ghost]

how about this

diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index a927fa2f..2a6f65ab 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -239,7 +239,7 @@ jobs:
       with:
         msystem: MSYS
         update: true
-        install: autotools gcc base-devel ncurses-devel
+        install: autoconf-wrapper automake-wrapper gcc make ncurses-devel pkgconf
     - name: Bootstrap
       shell: msys2 {0}
       run: ./autogen.sh

[ghost]

how about this

landed

[ghost]

it somehow runs much slower on my main pc running win11 with msys2, maybe it's caused by the extra handling in msys2, or maybe it's my pc or os issue (like the anti-virus software or some other quirks of windows...

i have rebooted my pc and now it runs with the expected performance on my pc with msys2 too

(i haven't rebooted my pc for almost 2 weeks

so maybe it's not a big deal for now

[ghost]

Support for Cygwin/Msys is currently available in my fork and I will try to maintain them in the future (you have to build from source yourself though)

[lm8]

Tried building htop natively on Windows using MinGW (without Cygwin). So far, I have the memory, swap and uptime displaying. I need to add code to support getting information about the processes running and more information on the CPUs (beyond just their count).