search

Kitware / visualizer

The ParaViewWeb Visualizer application which can be used standalone or as a component within another web context.
https://kitware.github.io/visualizer/
BSD 3-Clause "New" or "Revised" License
115 stars 28 forks source link

exit signal Segmentation fault #74

Closed Shaileshbg1 closed 5 years ago

Shaileshbg1 commented 5 years ago

Hello,

I am trying to build a flask application which calls paraview, loads VTK files from an OpenFOAM simulation, applies a few filters on it and then saves a statefile. I am running this on AWS EC2 micro instance which does not have gpu support, hence I am using a docker image of pv-osmesa-v5.6.0 to build my paraview.

Upon building my image and running a container I got many "file not found errors", to solve that I downloaded "ParaView-5.6.0-osmesa-MPI-Linux-64bit binaries" and copied that onto my container. I set my PYTHONPATH and LD_LIBRARY_PATHs to point to site-packages modules. That solved the file not found errors, but it gives me a new "exit signal Segmentation fault" :

Wed Apr 10 11:14:32.024814 2019] [mpm_event:notice] [pid 7:tid 139777627867008] AH00489: Apache/2.4.18 (Ubuntu) mod_wsgi/4.6.5 Python/3.6 configured -- resuming normal operations [Wed Apr 10 11:14:32.024898 2019] [core:notice] [pid 7:tid 139777627867008] AH00094: Command line: 'apache2 (mod_wsgi-express) -f /tmp/mod_wsgi-localhost:9090:999/httpd.conf -D MOD_WSGI_MPM_ENABLE_EVENT_MODULE -D MOD_WSGI_MPM_EXISTS_EVENT_MODULE -D MOD_WSGI_MPM_EXISTS_WORKER_MODULE -D MOD_WSGI_MPM_EXISTS_PREFORK_MODULE -D FOREGROUND' [Wed Apr 10 11:14:33.026052 2019] [core:notice] [pid 7:tid 139777627867008] AH00051: child pid 9 exit signal Segmentation fault (11), possible coredump in /tmp/mod_wsgi-localhost:9090:999 [Wed Apr 10 11:14:34.027550 2019] [core:notice] [pid 7:tid 139777627867008] AH00051: child pid 24 exit signal Segmentation fault (11), possible coredump in /tmp/mod_wsgi-localhost:9090:999 [Wed Apr 10 11:14:35.029068 2019] [core:notice] [pid 7:tid 139777627867008] AH00051: child pid 26 exit signal Segmentation fault (11), possible coredump in /tmp/mod_wsgi-localhost:9090:999 [Wed Apr 10 11:14:36.030557 2019] [core:notice] [pid 7:tid 139777627867008] AH00051: child pid 28 exit signal Segmentation fault (11), possible coredump in /tmp/mod_wsgi-localhost:9090:999

I am using python3.6 to build my flask/mod_wsgi application, but I think the paraview build using super build on the osmesa docker image uses python2.7.. I am not sure if this is the source of my error.

This is my Dockerfile: `ARG BASE_IMAGE=kitware/paraviewweb:pv-osmesa-v5.6.0 FROM ${BASE_IMAGE} ENV DEBIAN_FRONTEND=noninteractive USER root

Initial update and upgrade

RUN apt-get update RUN apt-get upgrade -y RUN apt-get install -y software-properties-common

Install python3.6 through PPA

RUN add-apt-repository ppa:deadsnakes/ppa RUN apt-get update RUN apt-get install -y \ python3.6 \ python3.6-dev \ python3.6-venv

Install all necessary packages

RUN apt-get install -y \ lsof \ apache2 \ apache2-dev \ vim \ libapache2-mod-wsgi \ curl \ wget

Install pip

RUN wget https://bootstrap.pypa.io/get-pip.py RUN python3.6 get-pip.py RUN ln -s /usr/bin/python3.6 /usr/local/bin/python3 RUN ln -s /usr/bin/python3.6 /usr/local/bin/python

pip install necesary python packages

RUN pip install \ Flask \ flask-restful \ boto3 \ requests \ mod_wsgi

###################################################################################

Start dumb_init as pid 1, to prevent any weird behaviour.

ADD https://github.com/Yelp/dumb-init/releases/download/v1.1.1/dumb-init_1.1.1_amd64 /usr/local/bin/dumb-init RUN chmod +x /usr/local/bin/dumb-init

Default user is root, which could cause issues in case of a breakout.

RUN groupadd -r niramai && useradd -m -r -g niramai niramai

RUN mkdir -p /home/niramai/postProcess_application

WORKDIR /home/niramai/postProcess_application

RUN ln -fs /usr/lib/python2.7/plat-x86_64-linux-gnu/_sysconfigdata_nd.py /usr/lib/python2.7/

COPY --chown=niramai:niramai . .

RUN tar -xvzf ParaView-5.6.0-osmesa-MPI-Linux-64bit.tar.gz

ENV LD_LIBRARY_PATH="/home/niramai/postProcess_application/ParaView-5.6.0-osmesa-MPI-Linux-64bit/lib"

ENV PYTHONPATH="/home/niramai/postProcess_application/ParaView-5.6.0-osmesa-MPI-Linux-64bit/lib:/opt/paraview/install/bin:/home/niramai/postProcess_application/ParaView-5.6.0-osmesa-MPI-Linux-64bit/lib/python2.7/site-packages:/home/niramai/postProcess_application/ParaView-5.6.0-osmesa-MPI-Linux-64bit/lib/python2.7/site-packages/vtkmodules" USER niramai

EXPOSE 9090

CMD ["dumb-init", "mod_wsgi-express", "start-server", "postProcessing_flask.wsgi", "--port", "9090", "--socket-timeout", "600" ]`

I have tried many options as setting permissions to the library directory, setting different paths in PYTHONPATH and LD_LIBRARY_PATH, copying the site-packages and pasting it in /usr/local/lib/ python modules, setting PYTHONPATH explicitly in mod_wsgi command, but none of them seem to work.

I tried having the whole thing use only python2.7, but that gave me a "Time not found error", only on upgrading to python3.6 did it get the necessary .so files and solved that problem.

I am not sure on how to proceed and any suggestion would be greatly appreciated. Looking forward to hearing back from you.

Regards, Shailesh

jourdain commented 5 years ago

Just use the downloaded version of osmesa paraview inside your own container without starting with the pv base one.

Shaileshbg1 commented 5 years ago

Hello Jourdain,

Thank you for your response, I removed the line ARG BASE_IMAGE=kitware/paraviewweb:pv-osmesa-v5.6.0 and replaced it with ARG BASE_IMAGE=ubuntu:16.04, so that use the downloaded version of osmesa and neglect the built pv base image.

But I still get the same error. [Wed Apr 10 15:12:18.201926 2019] [mpm_event:notice] [pid 6:tid 140012321695616] AH00489: Apache/2.4.18 (Ubuntu) mod_wsgi/4.6.5 Python/3.6 configured -- resuming normal operations [Wed Apr 10 15:12:18.201997 2019] [core:notice] [pid 6:tid 140012321695616] AH00094: Command line: 'apache2 (mod_wsgi-express) -f /tmp/mod_wsgi-localhost:9090:999/httpd.conf -D MOD_WSGI_MPM_ENABLE_EVENT_MODULE -D MOD_WSGI_MPM_EXISTS_EVENT_MODULE -D MOD_WSGI_MPM_EXISTS_WORKER_MODULE -D MOD_WSGI_MPM_EXISTS_PREFORK_MODULE -D FOREGROUND' [Wed Apr 10 15:12:19.203142 2019] [core:notice] [pid 6:tid 140012321695616] AH00051: child pid 8 exit signal Segmentation fault (11), possible coredump in /tmp/mod_wsgi-localhost:9090:999 [Wed Apr 10 15:12:20.204622 2019] [core:notice] [pid 6:tid 140012321695616] AH00051: child pid 23 exit signal Segmentation fault (11), possible coredump in /tmp/mod_wsgi-localhost:9090:999 Regards, Shailesh

jourdain commented 5 years ago

What is the command that trigger that segfault. That's not clear to me.

Shaileshbg1 commented 5 years ago

That error is triggered by the mod_wsgi-express start server command, which provides an apache module for hosting the flask application. The command in the docker file is: CMD ["dumb-init", "mod_wsgi-express", "start-server", "postProcessing_flask.wsgi", "--port", "9090", "--socket-timeout", "600" ]

which starts the server listening to port 9090.

Thanks and regards, Shailesh

jourdain commented 5 years ago

How is that related to ParaView or Visualizer?

Shaileshbg1 commented 5 years ago

Initially, when I opened this issue, I thought the issue was paraview not being able to find the modules, maybe because of different versions of python. Then by your suggestion, I understood that I can directly use only the downloaded osmesa binary, which simplified my work. After which now I realize that the issue is with mod_wsgi.

Very sorry to have wasted your time and thank you for your patience. Please let me know of how to delete this issue as it might not be of use to anyone. I am closing the issue.

Shaileshbg1 commented 5 years ago

Hello Jourdain,

Sorry to disturb you again but I am stuck with a dilemma here and could really use some suggestions from you.

For the application which I am trying to build I need to compile ParaView with python 3.

The official binaries and the docker images all have ParaView compiled with python 2. Compiling ParaView on my own against Python 3 is a very scary option for me. Is there any other way?

Looking forward to hearing back from you.

Regards, Shailesh

jourdain commented 5 years ago

No, someone need to build ParaView with Python 3. Feel free to post on the ParaView discourse to request binaries for Python 3.

aronhelser commented 5 years ago

Building ParaView with python 3 is not that bad - the instructions https://github.com/Kitware/ParaView/blob/master/Documentation/dev/build.md work well. The external dependencies are mostly just the compiler (and I recommend Ninja build), CMake and QT5.

We do need official Python 3 binaries, though!

On Mon, Apr 22, 2019 at 11:11 AM Sebastien Jourdain < notifications@github.com> wrote:

No, someone need to build ParaView with Python 3. Feel free to post on the ParaView discourse to request binaries for Python 3.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/Kitware/visualizer/issues/74#issuecomment-485445172, or mute the thread https://github.com/notifications/unsubscribe-auth/AEX6THPVO763OHOCKLHRSPTPRXIR5ANCNFSM4HE34O3A .

jourdain commented 5 years ago

Aslo since it will be for a ParaViewWeb application, you don't need Qt but either EGL or OSMesa.

Shaileshbg1 commented 5 years ago

Very sorry for such a late reply. Thank you for the documentation, following that I obtained the source and got all the prerequisites ready. I did not want to deviate much from the default build settings, so I only changed these variables:

cmake ../ -DPARAVIEW_BUILD_QT_GUI=OFF -DPARAVIEW_USE_MPI=ON -DPARAVIEW_ENABLE_PYTHON=ON -DPARAVIEW_ENABLE_WEB=ON

I could successfully build this using Ninja, but I get the following error: `[Tue Apr 30 11:25:27.120804 2019] [mpm_event:notice] [pid 1:tid 140375490279296] AH00489: Apache/2.4.18 (Ubuntu) mod_wsgi/4.6.5 Python/3.6 configured -- resuming normal operations [Tue Apr 30 11:25:27.120872 2019] [core:notice] [pid 1:tid 140375490279296] AH00094: Command line: 'apache2 (mod_wsgi-express) -f /tmp/mod_wsgi-localhost:9090:999/httpd.conf -D MOD_WSGI_MPM_ENABLE_EVENT_MODULE -D MOD_WSGI_MPM_EXISTS_EVENT_MODULE -D MOD_WSGI_MPM_EXISTS_WORKER_MODULE -D MOD_WSGI_MPM_EXISTS_PREFORK_MODULE -D FOREGROUND' [Tue Apr 30 11:25:27.752476 2019] [wsgi:error] [pid 164:tid 140375490279296] Error: Cannot import vtkPVAnimation

[Tue Apr 30 11:26:08.563378 2019] [wsgi:error] [pid 164:tid 140374181582592] [remote 106.51.23.138:12312] [2019-04-30 11:26:08,561] ERROR in app: Exception on /postProcess [POST] [Tue Apr 30 11:26:08.563405 2019] [wsgi:error] [pid 164:tid 140374181582592] [remote 106.51.23.138:12312] Traceback (most recent call last): [Tue Apr 30 11:26:08.563410 2019] [wsgi:error] [pid 164:tid 140374181582592] [remote 106.51.23.138:12312] File "/usr/local/lib/python3.6/dist-packages/flask/app.py", line 1813, in full_dispatch_request [Tue Apr 30 11:26:08.563414 2019] [wsgi:error] [pid 164:tid 140374181582592] [remote 106.51.23.138:12312] rv = self.dispatch_request() [Tue Apr 30 11:26:08.563417 2019] [wsgi:error] [pid 164:tid 140374181582592] [remote 106.51.23.138:12312] File "/usr/local/lib/python3.6/dist-packages/flask/app.py", line 1799, in dispatch_request [Tue Apr 30 11:26:08.563414 2019] [wsgi:error] [pid 164:tid 140374181582592] [remote 106.51.23.138:12312] rv = self.dispatch_request() [Tue Apr 30 11:26:08.563417 2019] [wsgi:error] [pid 164:tid 140374181582592] [remote 106.51.23.138:12312] File "/usr/local/lib/python3.6/dist-packages/flask/app.py", line 1799, in dispatch_request [Tue Apr 30 11:26:08.563421 2019] [wsgi:error] [pid 164:tid 140374181582592] [remote 106.51.23.138:12312] return self.view_functionsrule.endpoint [Tue Apr 30 11:26:08.563425 2019] [wsgi:error] [pid 164:tid 140374181582592] [remote 106.51.23.138:12312] File "/usr/local/lib/python3.6/dist-packages/flask_restful/init.py", line 458, in wrapper [Tue Apr 30 11:26:08.563429 2019] [wsgi:error] [pid 164:tid 140374181582592] [remote 106.51.23.138:12312] resp = resource(*args, kwargs) [Tue Apr 30 11:26:08.563432 2019] [wsgi:error] [pid 164:tid 140374181582592] [remote 106.51.23.138:12312] File "/usr/local/lib/python3.6/dist-packages/flask/views.py", line 88, in view [Tue Apr 30 11:26:08.563436 2019] [wsgi:error] [pid 164:tid 140374181582592] [remote 106.51.23.138:12312] return self.dispatch_request(*args, *kwargs) [Tue Apr 30 11:26:08.563439 2019] [wsgi:error] [pid 164:tid 140374181582592] [remote 106.51.23.138:12312] File "/usr/local/lib/python3.6/dist-packages/flask_restful/init.py", line 573, in dispatch_request [Tue Apr 30 11:26:08.563443 2019] [wsgi:error] [pid 164:tid 140374181582592] [remote 106.51.23.138:12312] resp = meth(args, kwargs) [Tue Apr 30 11:26:08.563446 2019] [wsgi:error] [pid 164:tid 140374181582592] [remote 106.51.23.138:12312] File "/postProcess_app/postProcessing_flask.py", line 15, in post [Tue Apr 30 11:26:08.563450 2019] [wsgi:error] [pid 164:tid 140374181582592] [remote 106.51.23.138:12312] call_paraview() [Tue Apr 30 11:26:08.563453 2019] [wsgi:error] [pid 164:tid 140374181582592] [remote 106.51.23.138:12312] File "/postProcess_app/postProcessing.py", line 26, in call_paraview [Tue Apr 30 11:26:08.563457 2019] [wsgi:error] [pid 164:tid 140374181582592] [remote 106.51.23.138:12312] animationScene1.UpdateAnimationUsingDataTimeSteps() [Tue Apr 30 11:26:08.563462 2019] [wsgi:error] [pid 164:tid 140374181582592] [remote 106.51.23.138:12312] AttributeError: 'NoneType' object has no attribute 'UpdateAnimationUsingDataTimeSteps' [Tue Apr 30 11:26:08.563489 2019] [wsgi:error] [pid 164:tid 140374181582592] [remote 106.51.23.138:12312]`

I tried to manually copy vtkPVAnimation.py from the official python2.7 paraview binaries, but that did not work. I tried to search which build setting relates to PVAnimation so that I can switch it off (I do not require animation at this moment), but I could not find it.

As per this https://public.kitware.com/pipermail/paraview/2015-May/034180.html It was suggested to add the path to PYTHONPATH, which I have already done.

Could you please let me know which build setting I have to modify and is there documentation giving us a list of all build parameters so that I can keep that as a reference for future errors.

Thanks and regards, Shailesh