search

conda-forge / vtk-feedstock

A conda-smithy repository for vtk.
BSD 3-Clause "New" or "Revised" License
13 stars 64 forks source link

MNT: Re-rendered with conda-smithy 2.3.0 #27

Closed nicoddemus closed 7 years ago

nicoddemus commented 7 years ago

Re-rendering to generate py36 packages.

conda-forge-linter commented 7 years ago

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

Korijn commented 7 years ago

There are other PRs attempting to do this. Build timeouts are a problem.

nicoddemus commented 7 years ago

@Korijn

There are other PRs attempting to do this. Build timeouts are a problem.

I noticed after it failed, sorry for not taking a look at the current PRs first.

How was the current version built? Manually and then uploaded? Anything I can do to help in this regard? Qt seems to have the same issue, and they have been making local builds to address this.

jakirkham commented 7 years ago

We have had some luck with scipy by splitting branches out for different versions of numpy. Could do the same thing here with python. Given python 3.5 is probably the next one to go, would recommend making that a separate branch. Alternatively could just drop python 3.5 support and go to python 2.7 and 3.6 (only adding back python 3.5 if there is demand).

nicoddemus commented 7 years ago

@jakirkham that's a good idea. Unfortunately looking at the logs it seems it is timing out while building the very first package (py27), so that wouldn't work.

It timed out around 21m, I'm surprised CircleCI has such a low limit; AppVeyor and Travis have 1h timeouts.

jakirkham commented 7 years ago

So if you scroll down (IOW download the log), you will see there was some sort of build failure. Given how simple this change was, it might be worth restarting.

nicoddemus commented 7 years ago

Good idea. I don't have write access though, could any of the maintainers try that?

patricksnape commented 7 years ago 
FAILED: Filters/Core/CMakeFiles/vtkFiltersCore.dir/vtkGridSynchronizedTemplates3D.cxx.o 
/opt/rh/devtoolset-2/root/usr/bin/g++  -DVTK_IN_VTK -DvtkFiltersCore_EXPORTS -IFilters/Core -I../Filters/Core -ICommon/Core -I../Common/Core -IUtilities/KWIML -I../Utilities/KWIML -IUtilities/KWSys -I../Utilities/KWSys -ICommon/DataModel -I../Common/DataModel -ICommon/Math -I../Common/Math -ICommon/Misc -I../Common/Misc -ICommon/System -I../Common/System -ICommon/Transforms -I../Common/Transforms -ICommon/ExecutionModel -I../Common/ExecutionModel -DBOOST_MATH_DISABLE_FLOAT128 -m64  -O3 -DNDEBUG -fPIC -fvisibility=hidden -MD -MT Filters/Core/CMakeFiles/vtkFiltersCore.dir/vtkGridSynchronizedTemplates3D.cxx.o -MF Filters/Core/CMakeFiles/vtkFiltersCore.dir/vtkGridSynchronizedTemplates3D.cxx.o.d -o Filters/Core/CMakeFiles/vtkFiltersCore.dir/vtkGridSynchronizedTemplates3D.cxx.o -c ../Filters/Core/vtkGridSynchronizedTemplates3D.cxx
g++: internal compiler error: Killed (program cc1plus)
Please submit a full bug report,
with preprocessed source if appropriate.
See <http://bugzilla.redhat.com/bugzilla> for instructions.
grlee77 commented 7 years ago

I do not get any compiler error when running this PR from a local linux machine.

googling that "g++: internal compiler error" message a bit, I found the following issue on another project where it seems to be a memory issue and they recommend increasing swap space (or running with fewer cores during build, but then we would likely see timeouts). https://github.com/Valloric/ycmd/issues/224

grlee77 commented 7 years ago

@jakirkham. Re: my previous comment and swap space. I am not too knowledgable on docker. Do you think it is the docker container itself that is encountering the limitation in memory or is it whatever underlying system CircleCI is running the container on?

grlee77 commented 7 years ago

At this point it is probably worth re-rendering to 2.3.1 so that we are up to date. May also want to cherry-pick the freetype 2.7 pinning update from the first commit in #23 as well. The libpng pinning should also be updated to >=1.6.28,<1.7.

jakirkham commented 7 years ago

The container can have a separate memory limit. Not sure if CircleCI configures it under the hood somehow or not. Normally the default would be to use all available memory that the system provides. So you may very well already be running up against the CircleCI limit (FWICT it is 4GB). A common cause for these sorts of issues is using to many threads. Would note that even one thread per core could be too many if each thread needs more memory than that configuration would allow.

grlee77 commented 7 years ago

Thanks. I restarted the CircleCI build to see if this memory problem is repeatable or just a fluke.

grlee77 commented 7 years ago

The memory error did not occur again upon restart, but the build did time out.

Korijn commented 7 years ago

Unfortunately, adding a configuration also means dropping one to stay within time limits.

nicoddemus commented 7 years ago

@jakirkham gave an idea of using a different branch for each Python version; would that be

  1. open 3 PRs targeting master where each PR only builds a single python version, and they are merged one after the other;
  2. have 3 branches in conda-forge/vtk-feedstock for each Python version, and then open 3 PRs targeting the appropriate branch;

For 2) to work, it would mean that packages are published from any branch in conda-forge/vtk-feedstock, but I'm not sure if packages are only published from master builds.

grlee77 commented 7 years ago

I'm pretty sure packages are published from all branches. For example, we have two separate branches for 4.x and 5.x versions in the following feedstock: https://github.com/conda-forge/suitesparse-feedstock

If we can reliably complete two builds then perhaps just a 2.7 branch and then a 3.x branch that builds the two most recent Python 3 versions. I think the only real downside to duplicate branches is that whenever versions are updated, etc, we need to push similar PRs to each branch. Slightly more work, but I don't see another good way around it at the moment.

grlee77 commented 7 years ago

To be clear, what I am suggesting above is: 1.) only build for the two most recent python 3 releases in master 2.) build Python 2.7 in a separate branch

Are there objections to creating a separate branch for 2.7? We can always go back to a single master branch in the future if we find a way around the CI time limits.

nicoddemus commented 7 years ago

@grlee77 your proposal sounds good to me. I'm closing this so we can focus the discussion in #23.

jakirkham commented 7 years ago

So just barely got a working implementation up for CircleCI matrix builds in conda-smithy. Have tested a couple of cases and seems to work ok. Though will require moving to CircleCI 2.0, which is still technically in Beta. In any event, here is the PR ( https://github.com/conda-forge/conda-smithy/pull/532 ) would appreciate it if some people take a look.