search

enthought / mayavi

3D visualization of scientific data in Python
http://docs.enthought.com/mayavi/mayavi/
Other
1.31k stars 285 forks source link

MAINT: Update for latest VTK #1050

Closed larsoner closed 3 years ago

larsoner commented 3 years ago

Closes #1049

@opoplawski can you see if this fixes your build hang on Fedora 34? It at least "fixes" the hang for me on Windows. Plotting still doesn't work there (mlab.test_plot3d()) despite pyvistaqt.BackgroundPlotter working okay, so there is more to fix. But this might already be good enough progress in that it will prevent total pip install hangs for people using the VTK pre wheels. (They'll have a problem later when they try to actually plot, but that's at least easier to diagnose than an empty build log that took an hour of CI time).

larsoner commented 3 years ago

It would also be good to add a CI job that tests with pip install --upgrade --pre vtk, but that should be another PR since I'm not really familiar with the CI setup here

opoplawski commented 3 years ago

@larsoner It seems to fix the hang I was seeing, thanks.

mjg commented 3 years ago

On Fedora 34 with Mayavi-4.7.3 including this fix here, vtk-9.0.1 and python 3.9.5, Mayavi2 (the app) now starts and plots but throws some error (see below). from mayavi import mlab still core dumps (even with ETS_TOOLKIT=wx).

2021-06-29 12:39:44.250 (  40,375s) [        FD47C740]     vtkOpenGLState.cxx:505   WARN| Error glBindFramebuffer1 OpenGL errors detected
  0 : (1282) Invalid operation

 with stack trace of
0x7f8aee7ae67e : ??? [(???) ???:-1]
0x7f8aee7af3a2 : vtksys::SystemInformation::GetProgramStack[abi:cxx11](int, int) [(libvtksys.so.1) ???:-1]
0x7f8aec605089 : ??? [(???) ???:-1]
0x7f8aec606b0e : vtkOpenGLState::vtkBindFramebuffer(unsigned int, vtkOpenGLFramebufferObject*) [(libvtkRenderingOpenGL2.so.1) ???:-1]
0x7f8aee2e97fd : vtkRenderWindow::Render() [(libvtkRenderingCore.so.1) ???:-1]
0x7f8aec5e7582 : vtkOpenGLRenderWindow::Render() [(libvtkRenderingOpenGL2.so.1) ???:-1]
0x7f8aec6a6e47 : vtkXOpenGLRenderWindow::Render() [(libvtkRenderingOpenGL2.so.1) ???:-1]
0x7f8aea7667ad : ??? [(???) ???:-1]
0x7f8afd8e0f68 : ??? [(???) ???:-1]
0x7f8afd8d2de7 : _PyObject_MakeTpCall [(libpython3.9.so.1.0) ???:-1]
0x7f8afd8cfcee : _PyEval_EvalFrameDefault [(libpython3.9.so.1.0) ???:-1]
0x7f8afd8d7753 : ??? [(???) ???:-1]
0x7f8afd8cf73f : _PyEval_EvalFrameDefault [(libpython3.9.so.1.0) ???:-1]
0x7f8afd8d7753 : ??? [(???) ???:-1]
0x7f8afd8dfa92 : ??? [(???) ???:-1]
0x7f8ab8b742fa : ??? [(???) ???:-1]
0x7f8ab8b743e3 : ??? [(???) ???:-1]
0x7f8ab3d6f649 : ??? [(???) ???:-1]
0x7f8ae865885e : QWidget::event(QEvent*) [(libQt5Widgets.so.5) ???:-1]
0x7f8ab3d7095b : ??? [(???) ???:-1]
0x7f8ae8617e73 : QApplicationPrivate::notify_helper(QObject*, QEvent*) [(libQt5Widgets.so.5) ???:-1]
0x7f8ab3b385b6 : ??? [(???) ???:-1]
0x7f8ae7b89f48 : QCoreApplication::notifyInternal2(QObject*, QEvent*) [(libQt5Core.so.5) ???:-1]
0x7f8ae865093a : QWidgetPrivate::sendPaintEvent(QRegion const&) [(libQt5Widgets.so.5) ???:-1]
0x7f8ae86514df : QWidgetPrivate::drawWidget(QPaintDevice*, QRegion const&, QPoint const&, QFlags<QWidgetPrivate::DrawWidgetFlag>, QPainter*, QWidgetRepaintManager*) [(libQt5W
idgets.so.5) ???:-1]
0x7f8ae8651d52 : QWidgetPrivate::paintOnScreen(QRegion const&) [(libQt5Widgets.so.5) ???:-1]
0x7f8ae8651e88 : QWidgetPrivate::syncBackingStore() [(libQt5Widgets.so.5) ???:-1]
0x7f8ae8658f8d : QWidget::event(QEvent*) [(libQt5Widgets.so.5) ???:-1]
0x7f8ab3d7095b : ??? [(???) ???:-1]
0x7f8ae8617e73 : QApplicationPrivate::notify_helper(QObject*, QEvent*) [(libQt5Widgets.so.5) ???:-1]
0x7f8ab3b385b6 : ??? [(???) ???:-1]
0x7f8ae7b89f48 : QCoreApplication::notifyInternal2(QObject*, QEvent*) [(libQt5Core.so.5) ???:-1]
0x7f8ae7b8cc76 : QCoreApplicationPrivate::sendPostedEvents(QObject*, int, QThreadData*) [(libQt5Core.so.5) ???:-1]
0x7f8ae7bd6c57 : ??? [(???) ???:-1]
0x7f8aec0374cf : g_main_context_dispatch [(libglib-2.0.so.0) ???:-1]
0x7f8aec08b4e8 : ??? [(???) ???:-1]
0x7f8aec034c03 : g_main_context_iteration [(libglib-2.0.so.0) ???:-1]
0x7f8ae7bd66f8 : QEventDispatcherGlib::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) [(libQt5Core.so.5) ???:-1]
0x7f8ae7b889b2 : QEventLoop::exec(QFlags<QEventLoop::ProcessEventsFlag>) [(libQt5Core.so.5) ???:-1]
0x7f8ae7b90544 : QCoreApplication::exec() [(libQt5Core.so.5) ???:-1]
0x7f8ab3b39b2e : ??? [(???) ???:-1]
0x7f8afd8e0f68 : ??? [(???) ???:-1]
0x7f8afd8d2de7 : _PyObject_MakeTpCall [(libpython3.9.so.1.0) ???:-1]
0x7f8afd8cfcee : _PyEval_EvalFrameDefault [(libpython3.9.so.1.0) ???:-1]
0x7f8afd8d7753 : ??? [(???) ???:-1]
0x7f8afd8caa5a : _PyEval_EvalFrameDefault [(libpython3.9.so.1.0) ???:-1]
0x7f8afd8d7753 : ??? [(???) ???:-1]
0x7f8afd8df9f1 : ??? [(???) ???:-1]
0x7f8afd8cf73f : _PyEval_EvalFrameDefault [(libpython3.9.so.1.0) ???:-1]
0x7f8afd8d7753 : ??? [(???) ???:-1]
0x7f8afd8df9f1 : ??? [(???) ???:-1]
0x7f8afd8cf73f : _PyEval_EvalFrameDefault [(libpython3.9.so.1.0) ???:-1]
0x7f8afd8c96dd : ??? [(???) ???:-1]
0x7f8afd8d745e : _PyFunction_Vectorcall [(libpython3.9.so.1.0) ???:-1]
0x7f8afd8df9f1 : ??? [(???) ???:-1]
0x7f8afd8cf73f : _PyEval_EvalFrameDefault [(libpython3.9.so.1.0) ???:-1]
0x7f8afd8d7753 : ??? [(???) ???:-1]
0x7f8afd8caa5a : _PyEval_EvalFrameDefault [(libpython3.9.so.1.0) ???:-1]
0x7f8afd8c96dd : ??? [(???) ???:-1]
0x7f8afd9460e5 : _PyEval_EvalCodeWithName [(libpython3.9.so.1.0) ???:-1]
0x7f8afd94607d : PyEval_EvalCodeEx [(libpython3.9.so.1.0) ???:-1]
0x7f8afd94602f : PyEval_EvalCode [(libpython3.9.so.1.0) ???:-1]
0x7f8afd9722e4 : ??? [(???) ???:-1]
0x7f8afd96e566 : ??? [(???) ???:-1]
0x7f8afd84621a : ??? [(???) ???:-1]
0x7f8afd968ac3 : PyRun_SimpleFileExFlags [(libpython3.9.so.1.0) ???:-1]
0x7f8afd965ec8 : Py_RunMain [(libpython3.9.so.1.0) ???:-1]
0x7f8afd938add : Py_BytesMain [(libpython3.9.so.1.0) ???:-1]
0x7f8afd619b75 : __libc_start_main [(libc.so.6) ???:-1]
0x55f2fbd2509e : _start [(python3) ???:-1]
larsoner commented 3 years ago

VTK just released 9.0.2, and all my pip install commands are stalled on Windows. I'm going to work around it by using my branch, but I expect this to be a (somewhat hard to diagnose) timeout problem for a lot of folks!

larsoner commented 3 years ago

@mjg for your error I would open a bug with the VTK folks (of course after some searching), it's possible they will recognize something from the traceback:

https://gitlab.kitware.com/vtk/vtk/-/issues

Alternatively you could engage in a time consuming git bisect + wheel build + test to figure out the VTK commit that caused the problem, e.g. what pyvista folks did here:

https://github.com/pyvista/pyvista/pull/1394#issuecomment-865308257

bcalvr commented 3 years ago

What's the latest on this please? I can't install mayavi, it just seems to hit a wall and get stuck!

prabhuramachandran commented 3 years ago

@larsoner -- I apologize for the delay in getting this done. I am going to try to get a bit more regular on this going forward.

larsoner commented 3 years ago

Even on this branch on 9.1.0-rc2 wheels on macOS at least I get:

Failed on Logger
(#3 of 11 nodes, #18 of 32 subnodes)

Traceback (most recent call last):
...
  File "/Users/larsoner/python/mayavi/tvtk/wrapper_gen.py", line 749, in _gen_get_set_methods
    if not meths[vtk_attr_name] and get_sig[0][1]:
IndexError: list index out of range

and then after fixing that:

Failed on InteractorObserver
(#4 of 11 nodes, #187 of 367 subnodes)
...
  File "/Users/larsoner/python/mayavi/tvtk/indenter.py", line 238, in get_method_doc
    name = camel2enthought(orig_name)
  File "/Users/larsoner/python/mayavi/tvtk/common.py", line 175, in __call__
    if ret[0] == '_':
IndexError: string index out of range

I can push a commit to work around (fix?) those here or in a separate PR @prabhuramachandran , let me know which is better

larsoner commented 3 years ago

... and then after fixing those:

../mayavi/tvtk/pyface/tvtk_scene.py:28: in <module>
    VTK_VER = tvtk.Version().vtk_version
E   AttributeError: 'Version' object has no attribute 'vtk_version'
larsoner commented 3 years ago

Clearly my workarounds are not good, or something else is wrong :(

>>> tvtk.Version()._vtk_obj.GetVTKVersion()
'9.1.0'
larsoner commented 3 years ago

And this exists:

(Pdb) p tvtk.Version().get_vtk_version()
'9.1.0'

So I'm not sure where/why the traits-based wrapper is failing :(

prabhuramachandran commented 3 years ago

@larsoner -- I will merge this for now to make life easier for everyone, I will look at the additional failures for 9.1.0 tomorrow and will not be able to look at this today. Looking at the errors, it looks like 9.1.0 changed the docstring signature which is going to break many things for sure. Thanks for bringing this up. My first goal is to get 9.0.3 all set and the tests going there and then I will look at 9.1.0-rc2. Am hoping that by next weekend things will be in better shape.

prabhuramachandran commented 3 years ago

@rahulporuri -- I am not able to merge this since the tests are required and don't have the time to sort this out today, so if you are able to merge this, please do so so I can build on top of this to fix additional issues. Thanks.

codecov-commenter commented 3 years ago

Codecov Report

Merging #1050 (e00c6de) into master (e360db9) will decrease coverage by 0.00%. The diff coverage is 0.00%.

:exclamation: Current head e00c6de differs from pull request most recent head 4e7f3b3. Consider uploading reports for the commit 4e7f3b3 to get more accurate results Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1050      +/-   ##
==========================================
- Coverage   49.83%   49.83%   -0.01%     
==========================================
  Files         263      263              
  Lines       23885    23887       +2     
  Branches     3248     3249       +1     
==========================================
  Hits        11903    11903              
- Misses      11210    11211       +1     
- Partials      772      773       +1
Impacted Files Coverage Δ
tvtk/vtk_parser.py 90.00% <0.00%> (-0.54%) :arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 7b007d1...4e7f3b3. Read the comment docs.

mdickinson commented 3 years ago

I've merged the master branch into this one; that should sort out the test requirements.

prabhuramachandran commented 3 years ago

@mdickinson -- who am I to talk to about the admin privileges? Until this point I was merging some of the PRs but I am not sure if it is something on my end or in the setup of the repo.

mdickinson commented 3 years ago

@prabhuramachandran The inability to merge was purely related to the PR, not to you. I wasn't being allowed to merge either. :-) Everyone in the "enthought" team in the "enthought" org currently has write privileges on this repository, which should allow merging PRs.

The merge was blocked because the expected GitHub Actions test runs hadn't passed, and the branch protections that we recently put in place require those test runs to pass. And the test runs didn't pass because they weren't running, which in turn was because the connected GitHub Actions workflow was present in the master branch but not in this branch. That's why the merge from master into this branch fixed things. We're likely to see the same effect with other open PRs that haven't had a merge from master recently. The fix should be the same for those PRs: merge master into the branch.

mdickinson commented 3 years ago

@prabhuramachandran Re: admin privileges, @rahulporuri should be able to push that through on our side.