search

LLNL / echemfem

Finite Element Method for Electrochemical Transport (EchemFEM)
MIT License
20 stars 5 forks source link

Errors with examples #10

Closed TomTranter closed 3 weeks ago

TomTranter commented 6 months ago

I am running on linux Ubuntu 22.04.4 LTS I installed firedrake successfully using the install script here https://www.firedrakeproject.org/download.html I activated the firedrake virtualenv and tried importing it into a python terminal but got an h5py import error So I uninstalled it and reinstalled the latest version and import error went away I installed the latest echemfem code and tried this example but got another error possibly related to H5py

(firedrake) tom@tom:~/code/echemfem/examples$ python simple_flow_battery.py firedrake:WARNING OMP_NUM_THREADS is not set or is set to a value greater than 1, we suggest setting OMP_NUM_THREADS=1 to improve performance Using dimensional Navier-Stokes-Brinkman 0 SNES Function norm 3.719318934070e-09 1 SNES Function norm 2.221647403080e-23 /home/tom/code/firedrake/src/firedrake/firedrake/_deprecation.py:65: UserWarning: The use ofFilefor output is deprecated, please update your code to useVTKFilefromfiredrake.output. warn( Traceback (most recent call last): File "simple_flow_battery.py", line 109, in <module> solver = FlowBatterySolver() File "simple_flow_battery.py", line 75, in __init__ super().__init__( File "/home/tom/code/echemfem/echemfem/solver.py", line 383, in __init__ self.set_velocity() File "simple_flow_battery.py", line 105, in set_velocity NSB_solver.solve() File "/home/tom/code/echemfem/echemfem/flow_solver.py", line 109, in solve with CheckpointFile('Velocity_field'+'.h5', 'w') as afile: File "/home/tom/code/firedrake/src/firedrake/firedrake/checkpointing.py", line 538, in __init__ self.set_attr_byte_string("/", "dmplex_storage_version", version) File "/home/tom/code/firedrake/src/firedrake/firedrake/checkpointing.py", line 1692, in set_attr_byte_string grp = self.require_group(path) File "/home/tom/code/firedrake/src/firedrake/firedrake/checkpointing.py", line 1772, in require_group _self = self.h5pyfile File "/home/tom/code/firedrake/src/firedrake/firedrake/checkpointing.py", line 1645, in h5pyfile self._h5pyfile = h5i.get_h5py_file(self.viewer) File "firedrake/cython/hdf5interface.pyx", line 35, in firedrake.cython.hdf5interface.get_h5py_file File "h5py/h5i.pyx", line 33, in h5py.h5i.wrap_identifier File "h5py/h5i.pyx", line 38, in h5py.h5i.wrap_identifier File "h5py/defs.pyx", line 1240, in h5py.defs.H5Iget_type RuntimeError: Unspecified error in H5Iget_type (return value <0)

running another example gives slightly different errors

(firedrake) tom@tom:~/code/echemfem/examples$ python paper_test.py 
firedrake:WARNING OMP_NUM_THREADS is not set or is set to a value greater than 1, we suggest setting OMP_NUM_THREADS=1 to improve performance
Eliminating species: C2
Creating function spaces for an extruded mesh
*** WARNING: Using custom solver options
*** ECHEM
Activated flow components
  advection      order = 1
  diffusion      order = 1
  migration      order = 1
  electroneutrality      order = 1
Species information
> C1
  z = 2.0
> C2
  z = -2.0
Solver information
variable     #dofs
C1       512
C2       512
Potential    512
Total        1536
#ranks       1
#dofs/rank   1536
Writing solutions. This may take a while...
/home/tom/code/firedrake/src/firedrake/firedrake/_deprecation.py:65: UserWarning: The use of `File` for output is deprecated, please update your code to use `VTKFile` from `firedrake.output`.
  warn(
/home/tom/code/firedrake/src/FInAT/finat/ufl/elementlist.py:441: UserWarning: Discontinuous Lagrange element requested on quadrilateral * interval, creating DQ element.
  warnings.warn("Discontinuous Lagrange element requested on %s, creating DQ element." % cell.cellname())
tsfc:WARNING Estimated quadrature degree 11 more than tenfold greater than any argument/coefficient degree (max (1, 1))
tsfc:WARNING Estimated quadrature degree 11 more than tenfold greater than any argument/coefficient degree (max (1, 1))
cells = 16 L2err = 6.08029E-03 C1err = 3.43798E-03 Uerr = 2.64230E-03
Eliminating species: C2
Creating function spaces for an extruded mesh
*** WARNING: Using custom solver options
*** ECHEM
Activated flow components
  advection      order = 1
  diffusion      order = 1
  migration      order = 1
  electroneutrality      order = 1
Species information
> C1
  z = 2.0
> C2
  z = -2.0
Solver information
variable     #dofs
C1       4096
C2       4096
Potential    4096
Total        12288
#ranks       1
#dofs/rank   12288
Writing solutions. This may take a while...
/home/tom/code/firedrake/src/firedrake/firedrake/_deprecation.py:65: UserWarning: The use of `File` for output is deprecated, please update your code to use `VTKFile` from `firedrake.output`.
  warn(
/home/tom/code/firedrake/src/FInAT/finat/ufl/elementlist.py:441: UserWarning: Discontinuous Lagrange element requested on quadrilateral * interval, creating DQ element.
  warnings.warn("Discontinuous Lagrange element requested on %s, creating DQ element." % cell.cellname())
cells = 64 L2err = 1.49294E-03 C1err = 8.63412E-04 Uerr = 6.29528E-04
Eliminating species: C2
Creating function spaces for an extruded mesh
*** WARNING: Using custom solver options
*** ECHEM
Activated flow components
  advection      order = 1
  diffusion      order = 1
  migration      order = 1
  electroneutrality      order = 1
Species information
> C1
  z = 2.0
> C2
  z = -2.0
Solver information
variable     #dofs
C1       32768
C2       32768
Potential    32768
Total        98304
#ranks       1
#dofs/rank   98304
Writing solutions. This may take a while...
/home/tom/code/firedrake/src/firedrake/firedrake/_deprecation.py:65: UserWarning: The use of `File` for output is deprecated, please update your code to use `VTKFile` from `firedrake.output`.
  warn(
/home/tom/code/firedrake/src/FInAT/finat/ufl/elementlist.py:441: UserWarning: Discontinuous Lagrange element requested on quadrilateral * interval, creating DQ element.
  warnings.warn("Discontinuous Lagrange element requested on %s, creating DQ element." % cell.cellname())
cells = 256 L2err = 3.79596E-04 C1err = 2.27132E-04 Uerr = 1.52464E-04
Eliminating species: C2
Creating function spaces for an extruded mesh
*** WARNING: Using custom solver options
*** ECHEM
Activated flow components
  advection      order = 1
  diffusion      order = 1
  migration      order = 1
  electroneutrality      order = 1
Species information
> C1
  z = 2.0
> C2
  z = -2.0
Solver information
variable     #dofs
C1       262144
C2       262144
Potential    262144
Total        786432
#ranks       1
#dofs/rank   786432
Writing solutions. This may take a while...
/home/tom/code/firedrake/src/firedrake/firedrake/_deprecation.py:65: UserWarning: The use of `File` for output is deprecated, please update your code to use `VTKFile` from `firedrake.output`.
  warn(
/home/tom/code/firedrake/src/FInAT/finat/ufl/elementlist.py:441: UserWarning: Discontinuous Lagrange element requested on quadrilateral * interval, creating DQ element.
  warnings.warn("Discontinuous Lagrange element requested on %s, creating DQ element." % cell.cellname())
cells = 1024 L2err = 1.11179E-04 C1err = 7.37598E-05 Uerr = 3.74192E-05
Eliminating species: C2
Creating function spaces for an extruded mesh
*** WARNING: Using custom solver options
*** ECHEM
Activated flow components
  advection      order = 1
  diffusion      order = 1
  migration      order = 1
  electroneutrality      order = 1
Species information
> C1
  z = 2.0
> C2
  z = -2.0
Solver information
variable     #dofs
C1       2097152
C2       2097152
Potential    2097152
Total        6291456
#ranks       1
#dofs/rank   6291456
Killed
TomTranter commented 6 months ago

In general it would be nice to have more information about how to run the examples and how to view the output

tlroy commented 6 months ago

I will keep the conversation about improving the examples in Issue #11 . I will focus here on these specific errors.

tlroy commented 6 months ago

(closed by mistake)

I am not sure about the first error. It is probably due to h5py installation. Let me look into it with a fresh firedrake install.

The paper_test.py example will probably need to be run in parallel on most machines to get to the finest mesh. It looks like you ran out of memory. As you mentioned, the examples should be better documented so I will add something about the memory requirements of this example.

In terms of output, do you mean what is being printed? In terms of visualization, the solution fields are being stored in results/collection.pvd (and vtu files). Some 1D examples also have custom matplotlib plotting. Again, this will be better documented.

tlroy commented 5 months ago

I added something about memory requirements related to default docker allocation as well as information about visualization in the QuickStart guide.

alizma commented 5 months ago

I came across issues with some examples as well that seem to be issues with the scripts themselves rather than installation or configuration. Using the Docker linked in the documentation, the simple_flow_battery example results in the following error: 

Traceback (most recent call last):
  File "/home/firedrake/echemfem/examples/simple_flow_battery.py", line 117, in <module>
    solver = FlowBatterySolver()
  File "/home/firedrake/echemfem/examples/simple_flow_battery.py", line 83, in __init__
    super().__init__(
  File "/home/firedrake/echemfem/echemfem/solver.py", line 388, in __init__
    self.setup_forms(us, v)
  File "/home/firedrake/echemfem/echemfem/solver.py", line 397, in setup_forms
    self.Form, self.bcs = self.steady_forms(us, v)
  File "/home/firedrake/echemfem/echemfem/solver.py", line 460, in steady_forms
    a, bc = self.potential_poisson_form(us,
  File "/home/firedrake/echemfem/echemfem/solver.py", line 2191, in potential_poisson_form
    U_0 = self.U_app
AttributeError: 'FlowBatterySolver' object has no attribute 'U_app'

Additionally, when running catalyst_layer_Cu_full.py, I receive the following error: 

Nonlinear echem_system_ solve did not converge due to DIVERGED_DTOL iterations 1
Traceback (most recent call last):
  File "/home/firedrake/echemfem/examples/catalyst_layer_Cu_full.py", line 247, in <module>
    solver.solve()
  File "/home/firedrake/echemfem/echemfem/solver.py", line 717, in solve
    self.echem_solver.solve()
  File "petsc4py/PETSc/Log.pyx", line 188, in petsc4py.PETSc.Log.EventDecorator.decorator.wrapped_func
  File "petsc4py/PETSc/Log.pyx", line 189, in petsc4py.PETSc.Log.EventDecorator.decorator.wrapped_func
  File "/home/firedrake/firedrake/src/firedrake/firedrake/adjoint_utils/variational_solver.py", line 89, in wrapper
    out = solve(self, **kwargs)
  File "/home/firedrake/firedrake/src/firedrake/firedrake/variational_solver.py", line 321, in solve
    solving_utils.check_snes_convergence(self.snes)
  File "/home/firedrake/firedrake/src/firedrake/firedrake/solving_utils.py", line 138, in check_snes_convergence
    raise ConvergenceError(r"""Nonlinear solve failed to converge after %d nonlinear iterations.
firedrake.exceptions.ConvergenceError: Nonlinear solve failed to converge after 1 nonlinear iterations.
Reason:
   DIVERGED_DTOL
tlroy commented 5 months ago

@alizma Thanks for bringing this to my attention. I just pushed a commit to main to fix the U_app bug with simple_flow_battery.py.

For catalyst_layer_Cu_full.py, it looks like a difference was introduced in a recent firedrake or petsc update. On a new install (not docker), I am getting these values for the nonlinear residuals:

  0 SNES Function norm 2.883957200450e-03
  1 SNES Function norm 1.572419903653e+01
  2 SNES Function norm 5.269680158967e+01
Nonlinear echem_system_ solve did not converge due to DIVERGED_DTOL iterations 2

whereas in a docker container from March I am getting this:

  0 SNES Function norm 2.883957200450e-03
  1 SNES Function norm 4.878061152760e-03
  2 SNES Function norm 5.171149830053e-01
  3 SNES Function norm 1.308949961681e+01
  4 SNES Function norm 7.758895362718e+00
  5 SNES Function norm 3.502470383063e+00
  6 SNES Function norm 2.603075365207e-03
  7 SNES Function norm 1.474182974622e-03
  8 SNES Function norm 1.890120789850e-03
Nonlinear echem_system_ solve converged due to CONVERGED_SNORM_RELATIVE iterations

I'm asking the firedrake devs about this. In the meantime, you can get convergence by adding this line solver.solver_parameters["snes_divergence_tolerance"] = -1 on line 238 of the example, before the solver setup.

tlroy commented 4 months ago

@alizma So I ended up changing the values for the bulk concentrations used in catalyst_layer_Cu_full.py. The ones from the paper are approximations that don't solve the reaction system exactly. This means the initial guess does not solve the bulk reactions exactly and there is a mild discontinuity at the boundary, both of which could affect the convergence. So now I simply solve the system to get actual equilibrium values for the bulk concentrations. It improves the convergence.