search

underworldcode / underworld2

underworld2: A parallel, particle-in-cell, finite element code for Geodynamics.
http://www.underworldcode.org/
Other
168 stars 58 forks source link

Parallel computing and An installation problem (MpiRun） #682

Closed tyszwh closed 9 months ago

tyszwh commented 10 months ago

I'm trying to install a new version of UW2, but I'm having problems with both installation methods I've tried (1), it can be installed normally but the parallel run error (attached file is the installation log) Run error XXX (mpirun -np XXX)

Non linear solver - iteration 0
Linear solver (T5U8X3ZT__system-execute) 
[0]PETSC ERROR: ------------------------------------------------------------------------
[0]PETSC ERROR: Caught signal number 11 SEGV: Segmentation Violation, probably memory access out of range
[0]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger
[0]PETSC ERROR: or see https://petsc.org/release/faq/#valgrind
[0]PETSC ERROR: or try http://valgrind.org on GNU/linux and Apple Mac OS X to find memory corruption errors
[0]PETSC ERROR: likely location of problem given in stack below
[0]PETSC ERROR: ---------------------  Stack Frames ------------------------------------
[0]PETSC ERROR: The EXACT line numbers in the error traceback are not available.
[0]PETSC ERROR: instead the line number of the start of the function is given.
[0]PETSC ERROR: #1 KSPSetFromOptions() at /home/tysz/software/petsc/src/ksp/ksp/interface/itcl.c:349
[0]PETSC ERROR: #2 _BlockSolve() at /tmp/pip-req-build-ax_lx83y/underworld/libUnderworld/Solvers/KSPSolvers/src/StokesBlockKSPInterface.c:293
[0]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------
[0]PETSC ERROR: Signal received
[0]PETSC ERROR: See https://petsc.org/release/faq/ for trouble shooting.
[0]PETSC ERROR: Petsc Release Version 3.16.1, unknown 
[0]PETSC ERROR: HimalayasReconstructionModel.py on a  named ubuntu by tysz Thu Nov 30 02:18:42 2023
[0]PETSC ERROR: Configure options --prefix=/home/tysz/miniconda3/envs/uw2 --with-debugging=yes --COPTFLAGS=-O3 --CXXOPTFLAGS=-O3 --FOPTFLAGS=-O3 --with-shared-libraries --with-cxx-dialect=C++11 --with-make-np=8 --download-mpich=yes --download-hdf5=yes --download-mumps=yes --download-parmetis=yes --download-metis=yes --download-superlu=yes --download-hypre=yes --download-superlu_dist=yes --download-scalapack=yes --download-cmake=yes
[0]PETSC ERROR: #1 User provided function() at unknown file:0
[0]PETSC ERROR: Checking the memory for corruption.
The EXACT line numbers in the error traceback are not available.
Instead the line number of the start of the function is given.
[0] #1 PetscAbortFindSourceFile_Private() at /home/tysz/software/petsc/src/sys/error/err.c:35
[0] #2 _BlockSolve() at /tmp/pip-req-build-ax_lx83y/underworld/libUnderworld/Solvers/KSPSolvers/src/StokesBlockKSPInterface.c:293
application called MPI_Abort(MPI_COMM_WORLD, 59) - process 0

(2), it doesn't install properly （local install with command pip install -vvv -e . ）(Now. is there a way to compile manually as it used to be? cmake,swig....) Error message 

sing pip 23.3.1 from /home/tysz/miniconda3/envs/uw2/lib/python3.9/site-packages/pip (python 3.9)
Non-user install because site-packages writeable
Created temporary directory: /tmp/pip-build-tracker-d25wdizc
Initialized build tracking at /tmp/pip-build-tracker-d25wdizc
Created build tracker: /tmp/pip-build-tracker-d25wdizc
Entered build tracker: /tmp/pip-build-tracker-d25wdizc
Created temporary directory: /tmp/pip-install-lys6d2eg
Created temporary directory: /tmp/pip-ephem-wheel-cache-cisiue0l
Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
Obtaining file:///home/tysz/software/underworld2
  Added file:///home/tysz/software/underworld2 to build tracker '/tmp/pip-build-tracker-d25wdizc'
  Created temporary directory: /tmp/pip-build-env-1fx9o5hk
  Running command pip subprocess to install build dependencies
  Using pip 23.3.1 from /home/tysz/miniconda3/envs/uw2/lib/python3.9/site-packages/pip (python 3.9)
  Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
INFO[77212] [TCP] 127.0.0.1:36680 --> 101.6.15.130:443 match GeoIP(CN) using Domestic[DIRECT]
  Collecting setuptools
    Using cached https://pypi.tuna.tsinghua.edu.cn/packages/bb/e1/ed2dd0850446b8697ad28d118df885ad04140c64ace06c4bd559f7c8a94f/setuptools-69.0.2-py3-none-any.whl (819 kB)
  Collecting numpy>=1.20.3
    Using cached https://pypi.tuna.tsinghua.edu.cn/packages/2f/75/f007cc0e6a373207818bef17f463d3305e9dd380a70db0e523e7660bf21f/numpy-1.26.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (18.2 MB)
  Collecting swig>=4.0.0
    Using cached https://pypi.tuna.tsinghua.edu.cn/packages/50/ce/d683fd61491983749484105a752291fe094d6247e336de1b76eab5600e14/swig-4.1.1.post0-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl (1.8 MB)
  Collecting cmake>=0.29.24
    Using cached https://pypi.tuna.tsinghua.edu.cn/packages/5a/e1/001da8b79b5d336d42aee95aae4cb934348ffa8925a6280fcd81859d8734/cmake-3.27.7-py2.py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (26.0 MB)
  Installing collected packages: swig, cmake, setuptools, numpy
    Creating /tmp/pip-build-env-1fx9o5hk/overlay/bin
    changing mode of /tmp/pip-build-env-1fx9o5hk/overlay/bin/swig to 775
    changing mode of /tmp/pip-build-env-1fx9o5hk/overlay/bin/cmake to 775
    changing mode of /tmp/pip-build-env-1fx9o5hk/overlay/bin/cpack to 775
    changing mode of /tmp/pip-build-env-1fx9o5hk/overlay/bin/ctest to 775
    changing mode of /tmp/pip-build-env-1fx9o5hk/overlay/bin/f2py to 775
  Successfully installed cmake-3.27.7 numpy-1.26.2 setuptools-69.0.2 swig-4.1.1.post0
  Installing build dependencies ... done
  Running command Checking if build backend supports build_editable
  Checking if build backend supports build_editable ... done
  Running command Getting requirements to build editable
  Traceback (most recent call last):
    File "/home/tysz/miniconda3/envs/uw2/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
      main()
    File "/home/tysz/miniconda3/envs/uw2/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
    File "/home/tysz/miniconda3/envs/uw2/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 132, in get_requires_for_build_editable
      return hook(config_settings)
    File "/tmp/pip-build-env-1fx9o5hk/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 441, in get_requires_for_build_editable
      return self.get_requires_for_build_wheel(config_settings)
    File "/tmp/pip-build-env-1fx9o5hk/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 325, in get_requires_for_build_wheel
      return self._get_build_requires(config_settings, requirements=['wheel'])
    File "/tmp/pip-build-env-1fx9o5hk/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 295, in _get_build_requires
      self.run_setup()
    File "/tmp/pip-build-env-1fx9o5hk/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 311, in run_setup
      exec(code, locals())
    File "<string>", line 294, in <module>
    File "/tmp/pip-build-env-1fx9o5hk/overlay/lib/python3.9/site-packages/setuptools/__init__.py", line 103, in setup
      return distutils.core.setup(**attrs)
    File "/tmp/pip-build-env-1fx9o5hk/overlay/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 147, in setup
      _setup_distribution = dist = klass(attrs)
    File "/tmp/pip-build-env-1fx9o5hk/overlay/lib/python3.9/site-packages/setuptools/dist.py", line 314, in __init__
      self.metadata.version = self._normalize_version(self.metadata.version)
    File "/tmp/pip-build-env-1fx9o5hk/overlay/lib/python3.9/site-packages/setuptools/dist.py", line 350, in _normalize_version
      normalized = str(Version(version))
    File "/tmp/pip-build-env-1fx9o5hk/overlay/lib/python3.9/site-packages/setuptools/_vendor/packaging/version.py", line 198, in __init__
      raise InvalidVersion(f"Invalid version: '{version}'")
  setuptools.extern.packaging.version.InvalidVersion: Invalid version: '2.16.x'
  error: subprocess-exited-with-error

  × Getting requirements to build editable did not run successfully.
  │ exit code: 1
  ╰─> See above for output.

  note: This error originates from a subprocess, and is likely not a problem with pip.
  full command: /home/tysz/miniconda3/envs/uw2/bin/python3.9 /home/tysz/miniconda3/envs/uw2/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py get_requires_for_build_editable /tmp/tmppsgw86if
  cwd: /home/tysz/software/underworld2
  Getting requirements to build editable ... error
error: subprocess-exited-with-error

× Getting requirements to build editable did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.
Exception information:
Traceback (most recent call last):
  File "/home/tysz/miniconda3/envs/uw2/lib/python3.9/site-packages/pip/_internal/cli/base_command.py", line 180, in exc_logging_wrapper
    status = run_func(*args)
  File "/home/tysz/miniconda3/envs/uw2/lib/python3.9/site-packages/pip/_internal/cli/req_command.py", line 245, in wrapper
    return func(self, options, args)
  File "/home/tysz/miniconda3/envs/uw2/lib/python3.9/site-packages/pip/_internal/commands/install.py", line 377, in run
    requirement_set = resolver.resolve(
  File "/home/tysz/miniconda3/envs/uw2/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 76, in resolve
    collected = self.factory.collect_root_requirements(root_reqs)
  File "/home/tysz/miniconda3/envs/uw2/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 513, in collect_root_requirements
    reqs = list(
  File "/home/tysz/miniconda3/envs/uw2/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 474, in _make_requirements_from_install_req
    cand = self._make_candidate_from_link(
  File "/home/tysz/miniconda3/envs/uw2/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 190, in _make_candidate_from_link
    self._editable_candidate_cache[link] = EditableCandidate(
  File "/home/tysz/miniconda3/envs/uw2/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 318, in __init__
    super().__init__(
  File "/home/tysz/miniconda3/envs/uw2/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 156, in __init__
    self.dist = self._prepare()
  File "/home/tysz/miniconda3/envs/uw2/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 225, in _prepare
    dist = self._prepare_distribution()
  File "/home/tysz/miniconda3/envs/uw2/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 328, in _prepare_distribution
    return self._factory.preparer.prepare_editable_requirement(self._ireq)
  File "/home/tysz/miniconda3/envs/uw2/lib/python3.9/site-packages/pip/_internal/operations/prepare.py", line 696, in prepare_editable_requirement
    dist = _get_prepared_distribution(
  File "/home/tysz/miniconda3/envs/uw2/lib/python3.9/site-packages/pip/_internal/operations/prepare.py", line 71, in _get_prepared_distribution
    abstract_dist.prepare_distribution_metadata(
  File "/home/tysz/miniconda3/envs/uw2/lib/python3.9/site-packages/pip/_internal/distributions/sdist.py", line 54, in prepare_distribution_metadata
    self._install_build_reqs(finder)
  File "/home/tysz/miniconda3/envs/uw2/lib/python3.9/site-packages/pip/_internal/distributions/sdist.py", line 122, in _install_build_reqs
    build_reqs = self._get_build_requires_editable()
  File "/home/tysz/miniconda3/envs/uw2/lib/python3.9/site-packages/pip/_internal/distributions/sdist.py", line 111, in _get_build_requires_editable
    return backend.get_requires_for_build_editable()
  File "/home/tysz/miniconda3/envs/uw2/lib/python3.9/site-packages/pip/_internal/utils/misc.py", line 763, in get_requires_for_build_editable
    return super().get_requires_for_build_editable(config_settings=cs)
  File "/home/tysz/miniconda3/envs/uw2/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_impl.py", line 226, in get_requires_for_build_editable
    return self._call_hook('get_requires_for_build_editable', {
  File "/home/tysz/miniconda3/envs/uw2/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_impl.py", line 311, in _call_hook
    self._subprocess_runner(
  File "/home/tysz/miniconda3/envs/uw2/lib/python3.9/site-packages/pip/_internal/utils/subprocess.py", line 252, in runner
    call_subprocess(
  File "/home/tysz/miniconda3/envs/uw2/lib/python3.9/site-packages/pip/_internal/utils/subprocess.py", line 224, in call_subprocess
    raise error
pip._internal.exceptions.InstallationSubprocessError: Getting requirements to build editable exited with 1
Remote version of pip: 23.3.1
Local version of pip:  23.3.1
Was pip installed by pip? False
Removed file:///home/tysz/software/underworld2 from build tra

LOG_FILE.txt

tyszwh commented 10 months ago

I have recently configured a new environment that includes Python 3.10.13 and Petsc 3.19.4. However, I encountered an issue while trying to install it using the recommended method, which is the command pip install -v git+https://github.com/underworldcode/underworld2. I have attached the installation log for reference. Errors are different from the previous ones. LOG_FILE_1.log

julesghub commented 9 months ago

Hi @tyszwh, Yes a newer version of petsc is required, compared to the previous attempt you made. 3.16 is too old for the current version of UW2.

I see this in the error log. 

2023-11-30T19:57:39,393   Running command git clone --filter=blob:none https://github.com/underworldcode/underworld2 /tmp/pip-req-build-1ltcspyg
2023-11-30T19:57:39,396   Cloning into '/tmp/pip-req-build-1ltcspyg'...
2023-11-30T19:57:43,693 ERROR: Operation cancelled by user

Your download from git appears to have been cancelled. Any ideas why?

tyszwh commented 9 months ago

Hi, julesghub Sorry, there was a problem with the previous file. LOG_FILE_1.txt

However, I think I have located the issue. Please ensure Petsc is using version 3.19.4 and the Setuptools version is less than 65. GCC 13 results in errors during compilation. （Although, I don't know which error is preventing pip from installing.）

Hopefully, this will help anyone experiencing the same problem

Cheers

julesghub commented 9 months ago

Thanks for the points @tyszwh. Glad you got to the bottom of it. I'll close the ticket for now.

PS: The compile error in the log file is 

2023-12-10T23:36:53,280   /home/tysz/miniconda3/envs/geo3/include/petscstring.h:688:249: error: expected ')' before 'PRIxPTR'
2023-12-10T23:36:53,280     688 |   PetscAssert(!(((al > bl) && (al - bl) < n) || (bl - al) < n), PETSC_COMM_SELF, PETSC_ERR_ARG_INCOMP, "Memory regions overlap: either use PetscMemmove()\nor make sure your copy regions and lengths are correct.\nLength (bytes) %zu first address %" PRIxPTR " second address %" PRIxPTR, n, al, bl);

I guessing this is a gcc version related issue.

Feel free to reopen the ticket if you like.