search

nschloe / meshio

:spider_web: input/output for many mesh formats
MIT License
1.93k stars 397 forks source link

[BUG] `AssertionError` sometimes raised in Nastran float -> string conversion #1428

Open whophil opened 1 year ago

whophil commented 1 year ago

Describe the bug AssertionError is sometimes raised in Nastran float -> string conversion

To Reproduce

from meshio.nastran._nastran import _float_to_nastran_string

f = -167.99725341796875 # a float that fails; originally it was the result of some upstream operation
_float_to_nastran_string(f)  # raises `AssertionError`

Diagnose meshio=5.3.4

adtzlr commented 9 months ago

Yes, I encountered that too. This is related to https://github.com/nschloe/meshio/commit/afcdfe0d033ff53f3bbabf44f5800775a18137b5. It happens because the negative sign produces a string which does not meet the requirements.

Possible Solution

Change the output precision from 11 to 10 to take the possible negative sign into account. https://github.com/nschloe/meshio/blob/0138cc8692b806b44b32d344f7961e8370121ff7/src/meshio/nastran/_nastran.py#L412

or an "adaptive" solution as proposed in #1350.