search

conda-forge / vtk-feedstock

A conda-smithy repository for vtk.
BSD 3-Clause "New" or "Revised" License
13 stars 64 forks source link

v8.2.0 #76

Closed grlee77 closed 5 years ago

grlee77 commented 5 years ago

Checklist

conda-forge-linter commented 5 years ago

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

grlee77 commented 5 years ago

The Python modules were getting installed to the wrong location in some cases. Apprently this is due to some changes to the CMAKE variables supported.

There were some warnings about unused flags

for example, on linux:

CMake Warning:
  Manually-specified variables were not used by the project:

    VTK_EXTERNAL_HDF5_IS_SHARED
    VTK_INSTALL_PYTHON_MODULE_DIR
    VTK_USE_SYSTEM_OGGTHEORA

Here is an issue related to the Python module install dir changes: https://gitlab.kitware.com/vtk/vtk/commit/b94049f69f6113b7817071369f1b22cfdca59d80

I just pushed a new commit attempting to update these, but haven't done things quite as recommended in the linked issue. It seems easier to use $SP_DIR instead of a relative path and the variable VTK_INSTALL_PYTHON_MODULES_DIR does seem to exist (note MODULES vs MODULE in the variable name).

grlee77 commented 5 years ago

The python libraries seem to be getting installed and OSX and Windows now passed CI, but there are two issues remaining. The first linux-specific, but the second applies to either linux or OS X.

1.) I don't know what is the issue with the linking problems being seen on linux. See for example the tons of warnings about "Needed DSO / not found". An example for one of the compiled libs:

WARNING (vtk,lib64/libvtkIOLegacyPython36D-8.2.so.1): Needed DSO / not found in any CDT/compiler package, nor the whitelist?!
WARNING (vtk,lib64/libvtkIOLegacyPython36D-8.2.so.1): Needed DSO / not found in any CDT/compiler package, nor the whitelist?!
WARNING (vtk,lib64/libvtkIOLegacyPython36D-8.2.so.1): Needed DSO / not found in any CDT/compiler package, nor the whitelist?!
WARNING (vtk,lib64/libvtkIOLegacyPython36D-8.2.so.1): Needed DSO / not found in any CDT/compiler package, nor the whitelist?!
   INFO (vtk,lib64/libvtkIOLegacyPython36D-8.2.so.1): Needed DSO x86_64-conda_cos6-linux-gnu/sysroot/lib/libc.so.6 found in CDT/compiler package conda-forge::gcc_impl_linux-64-7.3.0-habb00fd_1
WARNING (vtk,lib64/libvtkIOLegacyPython36D-8.2.so.1): Needed DSO / not found in any CDT/compiler package, nor the whitelist?!
   INFO (vtk,lib64/libvtkIOLegacyPython36D-8.2.so.1): Needed DSO lib/libstdc++.so.6 found in conda-forge::libstdcxx-ng-7.3.0-hdf63c60_0
WARNING (vtk,lib64/libvtkIOLegacyPython36D-8.2.so.1): Needed DSO / not found in any CDT/compiler package, nor the whitelist?!
   INFO (vtk,lib64/libvtkParallelCorePython36D-8.2.so.1): Needed DSO lib/libgcc_s.so.1 found in conda-forge::libgcc-ng-7.3.0-hdf63c60_0

2.) VTK_USE_SYSTEM_OGGTHEORA is being ignored so that VTK's own bundled Theora is getting built. I tried changing the name of the flag to VTK_USE_SYSTEM_THEORA which seems to try to use the external libs as desired, but that caused errors during compilation of VTK. See for example at: https://circleci.com/gh/conda-forge/vtk-feedstock/1229?utm_campaign=vcs-integration-link&utm_medium=referral&utm_source=github-build-link

grlee77 commented 5 years ago

Thanks for your suggestion on gitter @mingwandroid!

Setting CMAKE_INSTALL_LIBDIR did fix the linux builds (issue 1 in the previous comment).

Two fix the second issue: 1.) Currently, setting VTK_USE_SYSTEM_OGGTHEORA is being ignored so that flag could be remove and libogg and libtheora could probably also be removed from the build/run requirements.

2.) I think VTK_USE_SYSTEM_THEORA applies only to THEORA and not OGG so probably mixing conda-forge's libtheora with VTK's bundled ogg is somehow leading to errors like the one pasted below. I found another variable VTK_MODULE_USE_EXTERNAL_vtkogg referenced in ThirdParty/ogg/vtk_ogg.h.in that may need to be set as well, but I'm not sure.

[3028/8296] : && $BUILD_PREFIX/bin/x86_64-apple-darwin13.4.0-clang++ -march=core2 -mtune=haswell -mssse3 -ftree-vectorize -fPIC -fPIE -fstack-protector-strong -O2 -pipe -stdlib=libc++ -fvisibility-inlines-hidden -std=c++14 -fmessage-length=0 -I$PREFIX/include -fdebug-prefix-map==/usr/local/src/conda/- -fdebug-prefix-map==/usr/local/src/conda-prefix  -O3 -DNDEBUG -isysroot /Applications/Xcode-9.0.1.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk -mmacosx-version-min=10.9 -dynamiclib -Wl,-headerpad_max_install_names -Wl,-pie -Wl,-headerpad_max_install_names -Wl,-dead_strip_dylibs -Wl,-rpath,$PREFIX/lib -L$PREFIX/lib    -compatibility_version 1.0.0 -current_version 1.0.0 -o lib/libvtkIOMovie-8.2.1.dylib -install_name @rpath/libvtkIOMovie-8.2.1.dylib IO/Movie/CMakeFiles/vtkIOMovie.dir/vtkGenericMovieWriter.cxx.o IO/Movie/CMakeFiles/vtkIOMovie.dir/vtkOggTheoraWriter.cxx.o  -Wl,-rpath,$SRC_DIR/build/lib lib/libvtkCommonExecutionModel-8.2.1.dylib $PREFIX/lib/libtheoraenc.dylib $PREFIX/lib/libtheoradec.dylib lib/libvtkCommonDataModel-8.2.1.dylib lib/libvtkCommonMisc-8.2.1.dylib lib/libvtkCommonSystem-8.2.1.dylib lib/libvtkCommonTransforms-8.2.1.dylib lib/libvtkCommonMath-8.2.1.dylib lib/libvtkCommonCore-8.2.1.dylib $PREFIX/lib/libtbb.dylib && :
FAILED: lib/libvtkIOMovie-8.2.1.dylib 
: && $BUILD_PREFIX/bin/x86_64-apple-darwin13.4.0-clang++ -march=core2 -mtune=haswell -mssse3 -ftree-vectorize -fPIC -fPIE -fstack-protector-strong -O2 -pipe -stdlib=libc++ -fvisibility-inlines-hidden -std=c++14 -fmessage-length=0 -I$PREFIX/include -fdebug-prefix-map==/usr/local/src/conda/- -fdebug-prefix-map==/usr/local/src/conda-prefix  -O3 -DNDEBUG -isysroot /Applications/Xcode-9.0.1.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk -mmacosx-version-min=10.9 -dynamiclib -Wl,-headerpad_max_install_names -Wl,-pie -Wl,-headerpad_max_install_names -Wl,-dead_strip_dylibs -Wl,-rpath,$PREFIX/lib -L$PREFIX/lib    -compatibility_version 1.0.0 -current_version 1.0.0 -o lib/libvtkIOMovie-8.2.1.dylib -install_name @rpath/libvtkIOMovie-8.2.1.dylib IO/Movie/CMakeFiles/vtkIOMovie.dir/vtkGenericMovieWriter.cxx.o IO/Movie/CMakeFiles/vtkIOMovie.dir/vtkOggTheoraWriter.cxx.o  -Wl,-rpath,$SRC_DIR/build/lib lib/libvtkCommonExecutionModel-8.2.1.dylib $PREFIX/lib/libtheoraenc.dylib $PREFIX/lib/libtheoradec.dylib lib/libvtkCommonDataModel-8.2.1.dylib lib/libvtkCommonMisc-8.2.1.dylib lib/libvtkCommonSystem-8.2.1.dylib lib/libvtkCommonTransforms-8.2.1.dylib lib/libvtkCommonMath-8.2.1.dylib lib/libvtkCommonCore-8.2.1.dylib $PREFIX/lib/libtbb.dylib && :
ld: warning: -pie being ignored. It is only used when linking a main executable
Undefined symbols for architecture x86_64:
  "_ogg_stream_clear", referenced from:
      vtkOggTheoraWriterInternal::End() in vtkOggTheoraWriter.cxx.o
  "_ogg_stream_flush", referenced from:
      vtkOggTheoraWriterInternal::WriteHeader() in vtkOggTheoraWriter.cxx.o
  "_ogg_stream_init", referenced from:
      vtkOggTheoraWriterInternal::Start() in vtkOggTheoraWriter.cxx.o
  "_ogg_stream_packetin", referenced from:
      vtkOggTheoraWriterInternal::WriteHeader() in vtkOggTheoraWriter.cxx.o
      vtkOggTheoraWriterInternal::EncodeFrame(th_img_plane*, int) in vtkOggTheoraWriter.cxx.o
  "_ogg_stream_pageout", referenced from:
      vtkOggTheoraWriterInternal::WriteHeader() in vtkOggTheoraWriter.cxx.o
      vtkOggTheoraWriterInternal::EncodeFrame(th_img_plane*, int) in vtkOggTheoraWriter.cxx.o
ld: symbol(s) not found for architecture x86_64
clang-4.0: error: linker command failed with exit code 1 (use -v to see invocation)
looooo commented 5 years ago

is this one ready for merge?

grlee77 commented 5 years ago

Aside from the OGG/THEORA issue where it is using the internally bundled version rather than the conda-forge one, it seems to be good.

Do any of the other maintainers have an opinion on just removing the dependency on libtheora and libogg? I won't have time to work on this recipe in terms of trying to fix the linking this week, but I think this should probably not block merging if there is a need for 8.2 in downstream packages.

tadeu commented 5 years ago

Support for external libtheora and libogg were added by @jakirkham here, so maybe it's up to him if they could be temporarily removed.

I particularly see no problem in removing them for 8.2.0, as long as the ones that come bundled are statically linked (or, if that is not the case, perhaps disable ogg/theora writers?). If they are needed, they could be added again for a new build later.

tadeu commented 5 years ago

By the way, try passing -LAH to cmake so that it show all available options. Sometimes it's hard to get these options right, specially when they keep changing them.

grlee77 commented 5 years ago

I see that anaconda updated their recipe a few weeks ago as well:

https://github.com/AnacondaRecipes/vtk-feedstock/commit/29bc989f756a01c80ff8b2869818f17538736fd6

and do not seem to have changed anything related to ogg/theora.

They moved to checking all Python imports via vtkmodules while I had left most of them in the old-style imports from vtk and just checked that vtkmodules also imports.

It looks like the also enabled the following optional module that we do not currently have enabled. Module_vtkIOXdmf2:INTERNAL=ON

grlee77 commented 5 years ago

I particularly see no problem in removing them for 8.2.0, as long as the ones that come bundled are statically linked

The libraries that get built have vtk in the name, so they will not conflict with libogg or libtheora from elsewhere. Here is an excerpt from a Travis log:

   INFO (vtk,lib/libvtktheora-8.2.so.1): lib/libvtkogg-8.2.so.1 found in this package

   INFO (vtk,lib/libvtkIOMovie-8.2.so.1): Needed DSO lib/libgcc_s.so.1 found in conda-forge::libgcc-ng-7.3.0-hdf63c60_0
   INFO (vtk,lib/libvtkIOMovie-8.2.so.1): Needed DSO lib/libstdc++.so.6 found in conda-forge::libstdcxx-ng-7.3.0-hdf63c60_0
   INFO (vtk,lib/libvtkIOMovie-8.2.so.1): lib/libvtktheora-8.2.so.1 found in this package
   INFO (vtk,lib/libvtkIOMovie-8.2.so.1): lib/libvtkogg-8.2.so.1 found in this package
   INFO (vtk,lib/libvtkIOMovie-8.2.so.1): lib/libvtkCommonCore-8.2.so.1 found in this package
   INFO (vtk,lib/libvtkIOMovie-8.2.so.1): lib/libvtkCommonMisc-8.2.so.1 found in this package
   INFO (vtk,lib/libvtkIOMovie-8.2.so.1): lib/libvtkCommonExecutionModel-8.2.so.1 found in this package
   INFO (vtk,lib/libvtkIOMovie-8.2.so.1): Needed DSO x86_64-conda_cos6-linux-gnu/sysroot/lib/libc.so.6 found in CDT/compiler package conda-forge::gcc_impl_linux-64-7.3.0-habb00fd_1

Also, the following warning confirms that the ogg/theora included in the meta.yaml are not being used:

WARNING (vtk): dso library package conda-forge::libtheora-1.1.1-h14c3975_1001 in requirements/run but it is not used (i.e. it is overdepending or perhaps statically linked? If that is what you want then add it to `build/ignore_run_exports`)
WARNING (vtk): dso library package conda-forge::libogg-1.3.2-h14c3975_1001 in requirements/run but it is not used (i.e. it is overdepending or perhaps statically linked? If that is what you want then add it to `build/ignore_run_exports`)
grlee77 commented 5 years ago

I did not hear any objections to dropping libogg/libtheora from meta.yaml, so I am trying it out now. The bundled libvtkogg and libvtktheora are being built, so no functionality should be lost.

If there are any objections please let me know. Otherwise, let's merge if the CI passes