search

EddyRivasLab / hmmer

HMMER: biological sequence analysis using profile HMMs
http://hmmer.org
Other
307 stars 69 forks source link

Mingw build #263

Open jones-gareth opened 2 years ago

jones-gareth commented 2 years ago

Hi,

I have created Windows executables for hmmer programs using mingw.

This PR contains the changes I made to get all tests (except hmmpgmd_ga and hmmpgmd_shard_ga) passing using the mingw compiler and the msys2 shell. Details are in Windows.md

If you're interested it would be nice to include these changes in the main repository. However, I could certainly understand that you might not want Windows specific tweaks in your code base.

I also had to make changes to easel/develop and will create a PR for that repository if you want to move forward.

Thanks for making all the source for this great program freely available.

Gareth

npcarter commented 2 years ago

Hello Gareth, and thanks for your work. Windows is sufficiently different from the Unix and Unix-like operating systems that we support that we don't feel we have the staff to support it officially, as that would mean signing up to keep a Windows version functional as Windows changes over time.

However, if you're interested in setting your code up on your own Github or other repository, we'd be happy to link to it from the hmmer.org web page as a suggestion for people running on Windows. Alternatively, we've had good luck running HMMER under the Windows Subsystem for Linux. Again, we don't support it officially, but in our tests, it's worked right out of the box once you install the right set of build packages on WSL.

-Nick

On Thu, Nov 11, 2021 at 1:12 PM Gareth Jones @.***> wrote:

Hi,

I have created Windows executables for hmmer programs using mingw.

This PR contains the changes I made to get all tests (except hmmpgmd_ga and hmmpgmd_shard_ga) passing using the mingw compiler and the msys2 shell. Details are in Windows.md

If you're interested it would be nice to include these changes in the main repository. However, I could certainly understand that you might not want Windows specific tweaks in your code base.

I also had to make changes to easel/develop and will create a PR for that repository if you want to move forward.

Thanks for making all the source for this great program freely available.

Gareth

You can view, comment on, or merge this pull request online at:

https://github.com/EddyRivasLab/hmmer/pull/263 Commit Summary

File Changes

(28 files https://github.com/EddyRivasLab/hmmer/pull/263/files)

Patch Links:

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/EddyRivasLab/hmmer/pull/263, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABDJBZCXTF2OYIGM7IW6NCLULQBRZANCNFSM5H3BT3LQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

jones-gareth commented 2 years ago

Nick, I understand where you're coming from. I will package things up in such a way that others can reproduce the build steps. WSL is great for many things but not (yet) a practical solution to distributing hmmer to many users in corporate environment. Thanks, Gareth

npcarter commented 2 years ago

Sounds good. I look forward to taking a look at the package.

-Nick

On Thu, Nov 18, 2021 at 8:54 PM Gareth Jones @.***> wrote:

Nick, I understand where you're coming from. I will package things up in such a way that others can reproduce the build steps. WSL is great for many things but not (yet) a practical solution to distributing hmmer to many users in corporate environment. Thanks, Gareth

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/EddyRivasLab/hmmer/pull/263#issuecomment-973655165, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABDJBZFDMILZ7S73WIJ2V2TUMWU47ANCNFSM5H3BT3LQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

jones-gareth commented 2 years ago

I have packaged up my changes into master branches of my forks of hmmer and easel. In addition to build instructions I've included the resulting execubles.

I would be great if you could link to the https://github.com/jones-gareth/hmmer repository. Since you don't want to to merge in my changes I'm planning to ask github to remove the fork status from these 2 projects (unless you have any objections). Thanks again.

npcarter commented 2 years ago

Thanks! My Windows machine is currently being reinstalled, but I'll give this a whirl and we'll link to it if it works well.

-Nick

On Tue, Nov 30, 2021 at 10:34 AM Gareth Jones @.***> wrote:

I have packaged up my changes into master branches of my forks of hmmer https://github.com/jones-gareth/hmmer and easel https://github.com/jones-gareth/easel. In addition to build instructions I've included the resulting execubles.

I would be great if you could link to the https://github.com/jones-gareth/hmmer repository. Since you don't want to to merge in my changes I'm planning to ask github to remove the fork status from these 2 projects (unless you have any objections). Thanks again.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/EddyRivasLab/hmmer/pull/263#issuecomment-982750812, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABDJBZH2BC66BSZ7Q4VALW3UOTVJDANCNFSM5H3BT3LQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

npcarter commented 2 years ago

Gareth, Martin,

I wanted to put the two of you in touch with each other because you've both contributed pull requests to help make HMMER compatible with MinGW.

Martin,

We don't have the available people to take on responsibility for maintaining any sort of Windows version of HMMER at this point. What we've suggested to Gareth, and what he's working on, is that he put together a Github repo of his work, and once we've had a chance to test it, we can put a link to it on the HMMER web page for people who are interested. Maybe you and he could collaborate on this to avoid replicating effort?

-Nick

On Fri, Dec 3, 2021 at 4:38 PM Nick Carter @.***> wrote:

Thanks! My Windows machine is currently being reinstalled, but I'll give this a whirl and we'll link to it if it works well.

-Nick

On Tue, Nov 30, 2021 at 10:34 AM Gareth Jones @.***> wrote:

I have packaged up my changes into master branches of my forks of hmmer https://github.com/jones-gareth/hmmer and easel https://github.com/jones-gareth/easel. In addition to build instructions I've included the resulting execubles.

I would be great if you could link to the https://github.com/jones-gareth/hmmer repository. Since you don't want to to merge in my changes I'm planning to ask github to remove the fork status from these 2 projects (unless you have any objections). Thanks again.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/EddyRivasLab/hmmer/pull/263#issuecomment-982750812, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABDJBZH2BC66BSZ7Q4VALW3UOTVJDANCNFSM5H3BT3LQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

althonos commented 2 years ago

@npcarter : thanks for the heads-up!

I was trying to update MinGW compatibility to be able to build PyHMMER for Windows, which would require less effort than actually building all the binaries like Gareth is doing. I'll see if I can get his fork to build, in which case I would be happy to use while compiling for Windows, while using the official sources for UNIX.

Of course, I'd be happier if the patches were merged, but I also understand why it's not practical for you now.

jones-gareth commented 2 years ago

@althonos and @npcarter - is there anything else I can do to move this forward?

althonos commented 2 years ago

@jones-gareth : I'll have some opportunity to work on this next month hopefully. At the moment I couldn't get a working MinGW / Cython build because of Windows memory mapping which does weird things with memory allocated in a DLL being used in another DLL. This is why there has been no real progress on my side.

npcarter commented 2 years ago

Hello. I've finally gotten some time to give this a try, and noticed a few issues. The first is pretty minor: your instructions for installing MinGW leave out the "git" package, so trying to check out your hmmer and easel packages fails. That part worked fine for me once I did "pacman -S git" however.

More seriously, Easel tests 73-84 (except 83) fail with crash error messages on my machine. Are they working for you guys?

The final thing is that the configure command you provided tells gcc to generate unoptimized code, which is going to hurt performance a lot. When I tried compiling with optimizations on, the big change was that the hmmpgmd test hung instead of failed. Have you looked into what's going on with optimization there?

jones-gareth commented 2 years ago

Hello. I've finally gotten some time to give this a try, and noticed a few issues. The first is pretty minor: your instructions for installing MinGW leave out the "git" package, so trying to check out your hmmer and easel packages fails. That part worked fine for me once I did "pacman -S git" however.

Sorry about that- I have updated the README.

More seriously, Easel tests 73-84 (except 83) fail with crash error messages on my machine. Are they working for you guys?

Those tests are all working for me- I checked by pulling the repositories to a temporary directory and following the README instructions.

The final thing is that the configure command you provided tells gcc to generate unoptimized code, which is going to hurt performance a lot. When I tried compiling with optimizations on, the big change was that the hmmpgmd test hung instead of failed. Have you looked into what's going on with optimization there?

I'm also not seeing that. I did a full optimization using

CPPFLAGS="-Ic:/local/include" LDFLAGS=" -static -Lc:/local/lib" CFLAGS="-O3" ./configure --prefix=c:/local

All tests completed successfully (except the 2 that normally fail). I did see 2 compile warnings about inlined macros which I should probably investigate.

Unfortunately, your issues are a mystery to me right now. What error messages are you seeing? I wonder if it is worth creating an Azure pipeline to validate the build. I haven't worked with Azure, but it looks like it should be possible to create a reference build.