Magnet Volumetric Meshing

[connoramoreno]

Current implementation of volumetrically meshing magnets results in MOAB error (fails to interpret EXODUS file). Perhaps fix by having Cubit directly export a H5M file, if possible.

[pshriwise]

I lack context here so ignore me if I'm off base, but in the interest of potentially saving you some time: MOAB can be sensitive to the extension of Exodus files when selecting a file reader. I can't recall if it accepts .e or .exo but perhaps try the other?

[connoramoreno]

Trying both extensions seems to lead to the same error, which I'll provide below. How it looks to me is that MOAB tries several file readers but all fail to load the Exodus format. It might be simpler just to have Cubit export the H5M rather than have MOAB load an alternative format from Cubit and convert it to H5M.

MOAB ERROR: --------------------- Error Message ------------------------------------                               
MOAB ERROR: /home/camoreno/test/LBDQH5_coils/coil_mesh.e: File is not HDF5!
MOAB ERROR: set_up_read() line 449 in ReadHDF5.cpp
MOAB ERROR: --------------------- Error Message ------------------------------------
MOAB ERROR: NULL file handle.!
MOAB ERROR: is_error() line 134 in ReadHDF5.hpp
MOAB ERROR: --------------------- Error Message ------------------------------------
MOAB ERROR: Expected Keyword!
MOAB ERROR: load_file() line 178 in ReadABAQUS.cpp
MOAB ERROR: --------------------- Error Message ------------------------------------
MOAB ERROR: Invalid/unrecognized line!
MOAB ERROR: load_file() line 253 in ReadOBJ.cpp
MOAB ERROR: --------------------- Error Message ------------------------------------
MOAB ERROR: This doesn't appear to be a .cub file!
MOAB ERROR: load_file() line 327 in Tqdcfr.cpp
MOAB ERROR: --------------------- Error Message ------------------------------------
MOAB ERROR: Parsing error at line 1: expected one of { $NOD $MeshFormat } got "CDF"!
MOAB ERROR: match_token() line 367 in FileTokenizer.cpp
MOAB ERROR: --------------------- Error Message ------------------------------------
MOAB ERROR: /home/camoreno/test/LBDQH5_coils/coil_mesh.e.node: cannot read file!
MOAB ERROR: open_file() line 60 in ReadTetGen.cpp
MOAB ERROR: --------------------- Error Message ------------------------------------
MOAB ERROR: /home/camoreno/test/LBDQH5_coils/coil_mesh.e: Trouble reading vertices!
MOAB ERROR: read_vertices() line 139 in ReadTemplate.cpp
MOAB ERROR: --------------------- Error Message ------------------------------------
MOAB ERROR: Failed to load file after trying all possible readers!
MOAB ERROR: serial_load_file() line 588 in Core.cpp
MOAB ERROR: load_file() line 471 in Core.cpp
Traceback (most recent call last):
  * Stuff I've excluded for brevity *
  mb.load_file(exo_path, exodus_set)
  File "pymoab/core.pyx", line 92, in pymoab.core.Core.load_file
  File "pymoab/types.pyx", line 109, in pymoab.types.check_error
IndexError: MOAB ErrorCode: MB_INDEX_OUT_OF_RANGE

[makeclean]

Just checking, you've looked at in Paraview or something to check its a valid Exodus file?

[connoramoreno]

Yes, the Exodus file loads into ParaView without issue.

[connoramoreno]

I've tested using Cubit to export the tetmesh in H5M and it works well.

[pshriwise]

Ah, I see. From the error message above it seems that MOAB isn't trying the Exodus reader. Sounds like you've got a path forward, but building MOAB with NetCDF enabled should allow you to read Exodus files directly.

[connoramoreno]

I see. I pulled MOAB from conda-forge (presumably without NetCDF enabled) rather than build from source

[gonuke]

I think @Edgar-21 and I may have discussed this at some point. Not all versions of moab in conda-forge are built with NetCDF but I think some are. I think you need a version built for tempest/MPI (even if you don't use tempest/MPI). I could probably tweak the moab feedstock to always include NetCDF

[connoramoreno]

Trying with a MOAB version that is built with tempest (but no MPI) results in the same error. Does MPI also need to be enabled?

[gonuke]

Assuming your conda-installing MOAB from conda-forge.... it should have netcdf as a requirement anytime it includes tempest.

can you see if conda installed libnetcdf?

[connoramoreno]

Looks like libnetcdf is installed. For reference, my specific builds are

Name          Version     Build                                 Channel
moab          5.4.1       mpi_mpich_tempest_py311h09b0d74_3     conda-forge
libnetcdf     4.9.2       mpi_mpich_h6e9ae28_5                  conda-forge

I started from a new conda environment that seems to have used versions with MPI enabled. Still results in the same error, however.

[gonuke]

Maybe Cubit exports a version of Exodus that MOAB cant read?

[connoramoreno]

Could be... MOAB still doesn't appear to be trying to use an Exodus reader, though. We could try exporting from Cubit using a different file format. Looks like MOAB can read .cub files, though I feel like @Edgar-21 tried this once and it didn't work (perhaps I'm misremembering).

[gonuke]

CUB files probably won't work

[connoramoreno]

Interestingly, mbconvert seems to work using the same MOAB build. It complains about file readers not working until it (presumably) finds one that does but the file conversion looks good in ParaView.

MOAB ERROR: --------------------- Error Message ------------------------------------
MOAB ERROR: test.e: File is not HDF5!
MOAB ERROR: set_up_read() line 449 in src/io/ReadHDF5.cpp
MOAB ERROR: --------------------- Error Message ------------------------------------
MOAB ERROR: NULL file handle.!
MOAB ERROR: is_error() line 134 in src/io/ReadHDF5.hpp
Read "test.e"
Wrote "test.h5m"

Here's an image of a test tet mesh (the H5M from the above code block converted to VTK), cut in half to ensure it is actually volumetric.

[gonuke]

Interesting - mbconvert should be using the same MOAB installation as the python module, so it's odd that one works and the other doesn't

[connoramoreno]

Right, and I am pretty sure that mbconvert is using the same MOAB installation as the python module. On my machine, the path to mbconvert is

/home/camoreno/miniforge3/envs/ps_env/bin/mbconvert

while the path to pymoab is

/home/camoreno/miniforge3/envs/ps_env/lib/python3.11/site-packages/pymoab

[gonuke]

Have you tried it in an interactive python session? Maybe you can inspect the pymoab modulee to make sure it is the same libMOAB.so under the hood?

[connoramoreno]

It's looking like the issue is stemming from Cubit. Because the Cubit python module is imported into ParaStell before PyMOAB, I believe Cubit's version of PyMOAB is being imported rather than my Conda environment's version. When using PyMOAB to do the file conversion in a separate script without Cubit, it does so without issue. Importing Cubit into the same Python script before PyMOAB then causes the error seen above. However, if I import Cubit after PyMOAB, a different error occurs that seems to be related to HDF5:

Traceback (most recent call last):
  File "/home/camoreno/test/moab_test.py", line 3, in <module>
    import cubit
  File "/opt/Coreform-Cubit-2023.11/bin/cubit.py", line 44, in <module>
    from cubit3 import *
  File "/opt/Coreform-Cubit-2023.11/bin/cubit3.py", line 12, in <module>
    import _cubit3
ImportError: /opt/Coreform-Cubit-2023.11/bin/libcubiti19.so: undefined symbol: H5Pset_dxpl_mpio

[connoramoreno]

Would there be a way to ensure Cubit only accesses its own libraries while the Conda modules only access their corresponding libraries?

[gonuke]

Good catch! I think you've nailed it. It's not Cubit's pymoab but Cubit's libMOAB.so that is being engaged which is probably NOT compiled with the NetCDF library.

While there may be a long term solution that involves coordinating our MOAB better with Cubit - the short term solution is probably to run mbconvert as a subprocess.

[connoramoreno]

Closed via #38