windows build with msys2

[msarahan]

This works locally - VS2008 and 2015 tested so far. Fingers crossed on Appveyor. I'm pretty sure it will fail on the first try, because it requires the msys2 channel to be added.

ping @pelson @ocefpaf - how should I go about adding that channel? @jakirkham - check this out when you get back

[pelson]

There is no long term solution to adding extra channels yet, but it can be fudged here for testing.

[conda-forge-linter]

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

[jakirkham]

We probably should switch from binstar.org to anaconda.org. :wink:

[pelson]

We probably should switch from binstar.org to anaconda.org. :wink:

👍 https://github.com/conda-forge/conda-smithy/blob/41e300e6a44be315e3070bf744f2c120ee5d1e8f/conda_smithy/templates/appveyor.yml.tmpl#L60

[mingwandroid]

I uploaded new m2-ca-certificates packages with bumped build numbers. Is there a way to retrigger the AppVeyor build?

[msarahan]

I usually just close/reopen the PR. Doing that now. Fingers crossed...

[mingwandroid]

The new packages may not have synced across yet since I only did it about an hour ago.

[ccordoba12]

@mingwandroid, md5 checksums keep failing.

[mingwandroid]

@ccordoba12 it's getting the right ca-ceritifcates (build 102) now:

    m2-ca-certificates:     20150426-102```

[mingwandroid]

Can someone please close and reopen this? I want to see if it uses m2-coreutils 8.25-101 from anaconda.org/msys2 instead. I suspect the failure was due to link.exe being from coreutils and not the MSVC linker. Build 101 fixes this.

[mingwandroid]

Seems like an unrelated mirroring/connectivity problem, my new build was being used which is good. Can we spin it once more please?

[mingwandroid]

OK, drat. What would be helpful here is to be able to setup build artefacts on AppVeyor so config.log could be inspected. Otherwise, @msarahan, can you change:

bash runConfigureICU MSYS/MSVC

to:

bash -x runConfigureICU MSYS/MSVC

it might give us some more clues.

[jakirkham]

Should we restart this again or have there been any changes since the last build failure?

[msarahan]

I don't think just restarting it is going to work. I think we need to make sure it's working locally first.

[mingwandroid]

Locally was fine, let's see what bash -x gives us. @jakirkham any clues about adding artefacts to AppVeyor builds? IMHO, it'd be good if we could recursively glob a series of patterns and add those.

[jakirkham]

I think @jjhelmus added something like that for staged-recipes. I think this is what you need, @mingwandroid. Though I don't know if there is more to it or not.

[msarahan]

This is the issue: https://ci.appveyor.com/project/conda-forge/icu-feedstock/build/1.0.37/job/i8v51c2tm1hyosp1#L523

I have no idea what is adding that, or why, but it's breaking the configure checks.

[msarahan]

It's a metadata path: https://msdn.microsoft.com/en-us/library/x1x72k9t.aspx

seems like the win sdk sets this, but VS does not. I think I'm going to add something to just unset it.

[msarahan]

Did I break appveyor? seems like it is not getting triggered any more...

[jakirkham]

Maybe. 😆 It has happened to me before too. Let's have it sit for a bit.

One thing you could try is to create a project under your user for icu and try testing this branch there for the interim.

[jakirkham]

Looks like you are getting somewhere, @msarahan. The Python 2.7 builds have passed. The rest are waiting to run.

[msarahan]

I won't start celebrating until 3.4 x64 passes.

[mingwandroid]

Good news.

I got the vs2010 / Win7.1 SDK build to work locally (up to a make check segfault). It seems that link.exe reads the CL environment variable, and we're converting it from POSIX to Windows form. Just after where you set MSYS2_ARG_CONV_EXCL, add:

set "MSYS2_ENV_CONV_EXCL=CL"

Also, while you're at it, I'd add "/out" to MSYS2_ARG_CONV_EXCL since I saw that case used in the logs (it didn't matter in this instance since I think a Windows program was calling another one, but it can't hurt).

Does anyone know why my conda-build console text has gone green now that I've installed Win7.1 SDK? Is there a way around this?

[msarahan]

@mingwandroid, green text is just windows envying your superior MSYS2 technology. I don't know how to help it get over it.

Seriously, I have ignored it, as it did not seem to affect functionality. Does setenv.cmd called by itself (outside the icu build) do this?

Note: I have not succeeded in getting the win 7.1 SDK installed. If you have tips on that, it would help me debug these things faster. I suspect I might need to remove vs 2010 pro?

[jakirkham]

So, this is at least building through on VS 2010 64-bit. However, the make check step has an error. Definitely an improvement, but worth investigation.

[jakirkham]

Sorry, I think I caused conflicts for you by merging the re-rendering PR ( https://github.com/conda-forge/icu-feedstock/pull/7 ). I'm hoping they should be pretty easy to fix as they include things like whitespace changes and shorter channel names.

[ocefpaf]

@msarahan is this PR still relevant?

(I lost track of the status of MSYS2 but I want to work on it soon to get some R packages going.)

[msarahan]

Sorry, I don't know what the state of our ICU is. If it's working on Windows just fine, then yes, I'd say this PR is irrelevant.

[ocefpaf]

If it's working on Windows just fine

@gillins is probably more qualified to answer that. (Sorry for the noise I am just looking into some old stuff and doing some house cleaning.)

[gillins]

The Windows packages appear to be empty (no binaries inside). Not sure if this is intended, but I suspect something has gone wrong in the build...

[mingwandroid]

IMHO this PR needs to be merged. Perhaps squashed a lot first (but not into a single commit) if we want a clean history. What is the conda-forge rules regarding history?

[ocefpaf]

IMHO this PR needs to be merged.

:+1:

Perhaps squashed a lot first (but not into a single commit) if we want a clean history. What is the conda-forge rules regarding history?

@mingwandroid history is not the problem here. This PR needs a rebase and an strategy to either use the msys2 or get msys2 into conda-forge

[mingwandroid]

msys2 is now in defaults. You do not need to add channels to install any m2- or m2w64- prefixed packages anymore.

I'm happier with it being in defaults too since due to ABI and dependency incompatibilities (conda-forge's libxml2 being linked to libicu for example - https://github.com/conda-forge/matplotlib-feedstock/pull/42) I cannot add -c conda-forge during builds of software intended for the defaults channel.

[ocefpaf]

msys2 is now in defaults. You do not need to add channels to install any m2- or m2w64- prefixed packages anymore.

I think I already knew that but lost the information in the noise. Thanks!

I know @msarahan is busy so I will take this from his hands, rebase and see how it hows.

I'm happier with it being in defaults too since due to ABI incompatibilities I cannot add -c conda-forge in general during builds of software I make for defaults.

I am interested in knowing what ABI incompatibilities you are finding.It is always nice to get a heads up in future problems :smile:

[mingwandroid]

@ocefpaf, I updated my comment, there are two incompatibilities, ABI and dependency choices, both are fatal to building packages intended for defaults with -c conda-forge on the conda-build command-line since packages in defaults can't depend on pakages from conda-forge.

[ocefpaf]

@ocefpaf, I updated my comment, there are two incompatibilities,

Thanks @mingwandroid!

[mingwandroid]

We should share recipes as much as possible though. I do wonder if the amount and intrusive nature of some of the patching I'm having to do the retain backwards compatibility with CentOS 5.11 and OS X 10.7 will fly with conda-forge, and if not, what we can do about it. For reference, this qt5 recipe should be a good talking point (I think this is atypical and as bad as things will get due to the C++-ness and the size of Qt5):

https://github.com/mingwandroid/staged-recipes/tree/CentOS5-Windows-no-side-by-side/recipes/qt5

[ocefpaf]

We should share recipes as much as possible though.

:+1:

We need you in our meetings :grin:

[ocefpaf]

@mingwandroid I am having success building this in AppVeyor with msys2. Awesome!

@msarahan what is the policy on $PREFIX/sbin/{genccode,gencmn,gennorm2,gensprep,icupkg} files? Remove? move to $PREFIX/bin? I am on the fence here.

[jakirkham]

We install them on Linux and Mac. Would like if we do the same with Windows. Will let others weigh in on where they go.

[ocefpaf]

We install them on Linux and Mac. Would like if we do the same with Windows. Will let others weigh in on where they go.

I guess I need some context here. Are you referring to $PREFIX/sbin/{genccode,gencmn,gennorm2,gensprep,icupkg} files? If so take a look at https://github.com/conda-forge/icu-feedstock/blob/master/recipe/build.sh#L33

[jakirkham]

You're right. I'm confused. Was thinking of the bin that we already provide.

So are you thinking we should start providing sbin too? I'm ok with it. 👍

[jakirkham]

So went for a trip down memory lane. It seems @msarahan, @ccordoba12, and I decided to drop these sbin executables as they didn't work properly and were pretty bulky. Would be ok with keeping them if we could get them to work. Though I suppose there is a question of including these bulky files. Not sure to what extent people using conda expect to use these.

[ocefpaf]

Closing in favor of #11