search

conda-forge / vtk-feedstock

A conda-smithy repository for vtk.
BSD 3-Clause "New" or "Revised" License
13 stars 64 forks source link

Use system libraries #4

Closed Korijn closed 8 years ago

Korijn commented 8 years ago

This PR can facilitate the work for https://github.com/conda-forge/vtk-feedstock/issues/2

I dropped the old branch because it had merge problems.

conda-forge-linter commented 8 years ago

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

Korijn commented 8 years ago

Looks like the build fails on missing system libraries now, which is actually quite nice: we can fill them in one by one until the build passes to determine the minimum set.

Korijn commented 8 years ago

NETCDF requires some files that aren't in our package, not sure who's fault that is. Leaving it out of the recipe for now. See here: https://github.com/Kitware/VTK/blob/v7.0.0/CMake/FindNetCDF.cmake#L105

msarahan commented 8 years ago

Sorry for my confusion here, but when you say "system libraries" - you seem to mean libraries provided by Conda? I only mention this because it seems like adding dependencies to particular versions of low-level libraries tied to the OS is more fragile than depending on Conda's libraries, which we have more control over. It seems from your conda recipe that you are using Conda's versions, not arbitrary system versions, but I just wanted to make sure.

Korijn commented 8 years ago

It also looks like GLUT is only used in a test once, so I'm not too sure if that's really a requirement here... https://github.com/Kitware/VTK/search?utf8=%E2%9C%93&q=glut

Korijn commented 8 years ago

@msarahan Yeah I meant using conda packages. VTK refers to third party libs as system libraries, so I was using their terminology here. Sorry for the confusion.

ocefpaf commented 8 years ago

NETCDF requires some files that aren't in our package, not sure who's fault that is. Leaving it out of the recipe for now. See here: https://github.com/Kitware/VTK/blob/v7.0.0/CMake/FindNetCDF.cmake#L105

It seems that VTK is looking for netcdfcpp.h netcdf_c++, netcdf.inc, and netcdf.mod. I believe you will get those from the netcdf-cxx4 and netcdf-fortran packages (both in conda-forge)

Korijn commented 8 years ago

@ocefpaf Right, they're Linux only packages, which explains why I couldn't install them on my Windows system. :) I'll add them under a conditional. Thanks!

ocefpaf commented 8 years ago

I couldn't install them on my Windows system.

Right. We are already using cmake to build both libraries there, so Windows should be possible. I will take a look at that soon.

Korijn commented 8 years ago

I just remembered that we have mesa-libGLU-devel in the yum_requirements.txt here... does that mean we can try building against mesa for offscreen rendering?

Korijn commented 8 years ago

@ocefpaf Unfortunately the file and directory names don't match, and there are no options to tell CMake where to look.

ocefpaf commented 8 years ago

@ocefpaf Unfortunately the file and directory names don't match, and there are no options to tell CMake where to look.

That is a bummer :unamused:

Maybe you'll have to patch the cmake files to make it work. However, I would not do that now and leave netcdf out for a bit to reduce the scope of your PR. You can always try that later.

Korijn commented 8 years ago

Offscreen rendering is pretty important when you want to do rendering on headless servers (or CI, for testing!), and VTK requires that you link to mesa for this. However, if I understand correctly, this means you can no longer do _on_screen rendering, unless you produce a "mangled Mesa" library and link to that. I don't think that is available here though, is it...? I'll see what I can do on the mesa recipe, unless anyone else has a brilliant suggestion. :)

Korijn commented 8 years ago

Snags hit so far:

Korijn commented 8 years ago

@jakirkham Looks like the system libraries are working now, but still hitting the build timeouts here...

ccordoba12 commented 8 years ago

@Korijn, great job!! Could you remove Python 3.4 on Linux? I think it's better to have two builds for it instead of none :-)

Korijn commented 8 years ago

@ccordoba12 Sure, I'll also disable OS X again. Too bad...

Korijn commented 8 years ago

@jakirkham I have a couple of questions before merging this:

demarle commented 8 years ago

Regarding onscreen+offscreen via mesa, this was fixed for the first time since the mangled mesa days (~10 years ago) in VTK Yesterday (https://gitlab.kitware.com/vtk/vtk/merge_requests/1642 For hardware based OpenGL, it looks like we have to wait for EGL to mature a bit for this to work. Until those are ready for prime time the two build option is your best terrible option in my opinion.

Korijn commented 8 years ago

@demarle Thanks for elaborating on that! Me and @Maxyme are trying to get a build of VTK put together that'll do offline rendering, but I must admit, it's tough! Both of us are new to the field, and most of the documentation appears to be outdated. Getting a conda package together is a real challenge so far. :( If you could point out any up-to-date documentation out to us, that would be a real help!

demarle commented 8 years ago

I've used this article (http://www.paraview.org/Wiki/ParaView_And_Mesa_3D) for hints for years on years on supercomputers without X to get offscreen VTK through OpenGL1 back end. Both mesa classic and mesa llvmpipe have worked as recently as 7.0.0.

Korijn commented 8 years ago

Maybe @ocefpaf can answer some of my questions before we merge this?

ocefpaf commented 8 years ago

Maybe @ocefpaf can answer some of my questions before we merge this?

I can try :smile:

  • What is your policy on pinning the versions of the dependencies added here?

Not sure if you know about our guide here. Try to pin to those versions and let's see if that works.

  • I would like to have a second version of the VTK package that is suitable for offscreen rendering on headless servers (e.g. without X server), but it entails different CMake parameters, effectively producing a different build. Can I make a recipe that "depends" on this one, so that they can evolve together?

I cannot see why not, but I am not sure what exactly that means. Is the "offscreen" version just a component/part of VTK that would depend on this? Or is it a full new version with a different flag for example? If the former your plan sounds fine, if the latter maybe we can try a metapackage instead.

The metapackage can install both, but would still allow people to install them separately if they want to.

Not sure I was helpful...

Korijn commented 8 years ago

This used to be an entirely different build. However, @demarle pointed out that it could be within the same build from 7.1 so I think it would be worthwhile to wait for that release.

Korijn commented 8 years ago

Thanks for the version numbers! Might be a nice feature to add a check to the linter?

jakirkham commented 8 years ago

Thanks for the version numbers! Might be a nice feature to add a check to the linter?

Feel free to raise an issue at the conda-smithy repo. 😄

jakirkham commented 8 years ago

Sounds like @ocefpaf has answered the first point quite nicely. Also it sounds like the second point does not need to be addressed as the next version of VTK will solve the problem, correct?