search

SteveDoyle2 / pyNastran

A Python-based interface tool for Nastran's file formats
Other
395 stars 154 forks source link

Reading stiffness matrix from op4 fails with large matrices #661

Open Jrois opened 3 years ago

Jrois commented 3 years ago

Hi,

I am having a issues reading in a stiffness matrix from an .op4 file. Everything worked fine until I started scaling up the number of nodes, which led me to believe my issue is similar to a previous issue. However my error traceback was different so I decided to make a separate issue. I have tried two set-ups and they both produce a different error:

Python v3.8 & pyNastran v1.3.1 give the following traceback:

Traceback (most recent call last): File "test2.py", line 14, in MAT = model.read_op4("./MSCNpyBenchmarkN10/MSCNpyBenchmarkN10_kelems.op4", matrix_names=["KAA"]) File "c:\user\anaconda3\envs\py38\lib\site-packages\pyNastran\op4\op4.py", line 137, in read_op4 return self.read_op4_binary(op4_filename, matrix_names, precision) File "c:\user\anaconda3\envs\py38\lib\site-packages\pyNastran\op4\op4.py", line 686, in read_op4_binary (name, form, matrix) = self._read_matrix_binary(op4, precision, matrix_names) File "c:\user\anaconda3\envs\py38\lib\site-packages\pyNastran\op4\op4.py", line 804, in _read_matrix_binary A = self._read_real_binary(op4, nrows, ncols, Type, is_sparse, is_big_mat) File "c:\user\anaconda3\envs\py38\lib\site-packages\pyNastran\op4\op4.py", line 895, in _read_real_binary A = self._read_real_sparse_binary(op4, nrows, ncols, matrix_type, is_big_mat) File "c:\user\anaconda3\envs\py38\lib\site-packages\pyNastran\op4\op4.py", line 914, in _read_real_sparse_binary (icol, irow, nwords) = self.get_markers_sparse(op4, is_big_mat) File "c:\user\anaconda3\envs\py38\lib\site-packages\pyNastran\op4\op4.py", line 1354, in get_markers_sparse unused_a, icol, irow, nwords = self.read_start_marker(op4) File "c:\user\anaconda3\envs\py38\lib\site-packages\pyNastran\op4\op4.py", line 726, in read_start_marker assert self.n == op4.tell(), 'n=%s tell=%s' % (self.n, op4.tell()) AssertionError: n=5177352 tell=5177344

When debug=True I get a different error:

Traceback (most recent call last): File "C:\user\anaconda3\envs\py38\lib\site-packages\IPython\core\interactiveshell.py", line 3441, in run_code exec(code_obj, self.user_global_ns, self.user_ns) File "", line 1, in runfile('C:/user/WF/python/MSCN-PY/test2.py', wdir='C:/SpaceLabs/WF/python/MSCN-PY') File "C:\user\PyCharm Community Edition 2021.2.1\plugins\python-ce\helpers\pydev_pydev_bundle\pydev_umd.py", line 198, in runfile pydev_imports.execfile(filename, global_vars, local_vars) # execute the script File "C:\user\PyCharm Community Edition 2021.2.1\plugins\python-ce\helpers\pydev_pydev_imps_pydev_execfile.py", line 18, in execfile exec(compile(contents+"\n", file, 'exec'), glob, loc) File "C:/user/WF/python/MSCN-PY/test2.py", line 14, in MAT = model.read_op4("./MSCNpyBenchmarkN10/MSCNpyBenchmarkN10_kelems.op4", matrix_names=["KAA"]) File "C:\user\anaconda3\envs\py38\lib\site-packages\pyNastran\op4\op4.py", line 137, in read_op4 return self.read_op4_binary(op4_filename, matrix_names, precision) File "C:\user\anaconda3\envs\py38\lib\site-packages\pyNastran\op4\op4.py", line 686, in read_op4_binary (name, form, matrix) = self._read_matrix_binary(op4, precision, matrix_names) File "C:\user\anaconda3\envs\py38\lib\site-packages\pyNastran\op4\op4.py", line 804, in _read_matrix_binary A = self._read_real_binary(op4, nrows, ncols, Type, is_sparse, is_big_mat) File "C:\user\anaconda3\envs\py38\lib\site-packages\pyNastran\op4\op4.py", line 895, in _read_real_binary A = self._read_real_sparse_binary(op4, nrows, ncols, matrix_type, is_big_mat) File "C:\user\anaconda3\envs\py38\lib\site-packages\pyNastran\op4\op4.py", line 938, in _read_real_sparse_binary self._show(op4, 100, types='qd') File "C:\user\anaconda3\envs\py38\lib\site-packages\pyNastran\op4\op4.py", line 1015, in _show strings, ints, floats = self._show_data(data, types=types, endian=endian) File "C:\user\anaconda3\envs\py38\lib\site-packages\pyNastran\op4\op4.py", line 1044, in _show_data return self._write_data(sys.stdout, data, types=types, endian=endian) File "C:\user\anaconda3\envs\py38\lib\site-packages\pyNastran\op4\op4.py", line 1092, in _write_data doubles = unpack(b'%s%id' % (endian, ndoubles), data[:ndoubles*8]) TypeError: %b requires a bytes-like object, or an object that implements bytes, not 'str'

Python v3.6 & pyNastran v1.2.1 gives this traceback:

Traceback (most recent call last): File "test2.py", line 5, in MAT = model.read_op4("K.op4") File "c:\user\anaconda3\envs\py38\lib\site-packages\pyNastran\op4\op4.py", line 137, in read_op4 return self.read_op4_binary(op4_filename, matrix_names, precision) File "c:\user\anaconda3\envs\py38\lib\site-packages\pyNastran\op4\op4.py", line 686, in read_op4_binary (name, form, matrix) = self._read_matrix_binary(op4, precision, matrix_names) File "c:\user\anaconda3\envs\py38\lib\site-packages\pyNastran\op4\op4.py", line 804, in _read_matrix_binary A = self._read_real_binary(op4, nrows, ncols, Type, is_sparse, is_big_mat) File "c:\user\anaconda3\envs\py38\lib\site-packages\pyNastran\op4\op4.py", line 895, in _read_real_binary A = self._read_real_sparse_binary(op4, nrows, ncols, matrix_type, is_big_mat) File "c:\user\anaconda3\envs\py38\lib\site-packages\pyNastran\op4\op4.py", line 974, in _read_real_sparse_binary value_list = unpack(str_values, data[0:nvalues * nbytes_per_value]) struct.error: bad char in struct format

Moreover, for some attempts I have gotten the same traceback as the first above (ending in AssertionError).

Below you can find a .zip containing the model and a code snippet to reproduce the error. I also added the op4 file produced since it can take some time to run the model.

Thanks in advance!

Kind regards, Joris

test.zip

SteveDoyle2 commented 3 years ago

I’m on vacation for another week, but have you tried using the op2 instead? If you search for OUTPUT2 you’ll find an example. Obviously you’ll need to change your file number to (I believe) 12 (not 2) to send the data to the op2.

On Fri, Sep 24, 2021 at 10:12 AM Jrois @.***> wrote:

Hi,

I am having a issues reading in a stiffness matrix from an .op4 file. Everything worked fine until I started scaling up the number of nodes, which led me to believe my issue is similar to a previous issue https://github.com/SteveDoyle2/pyNastran/issues/293. However my error traceback was different so I decided to make a separate issue. I have tried two set-ups and they both produce a different error:

Python v3.8 & pyNastran v1.3.1 give the following traceback:

Traceback (most recent call last): File "test2.py", line 14, in MAT = model.read_op4("./MSCNpyBenchmarkN10/MSCNpyBenchmarkN10_kelems.op4", matrix_names=["KAA"]) File "c:\user\anaconda3\envs\py38\lib\site-packages\pyNastran\op4\op4.py", line 137, in read_op4 return self.read_op4_binary(op4_filename, matrix_names, precision) File "c:\user\anaconda3\envs\py38\lib\site-packages\pyNastran\op4\op4.py", line 686, in read_op4_binary (name, form, matrix) = self._read_matrix_binary(op4, precision, matrix_names) File "c:\user\anaconda3\envs\py38\lib\site-packages\pyNastran\op4\op4.py", line 804, in _read_matrix_binary A = self._read_real_binary(op4, nrows, ncols, Type, is_sparse, is_big_mat) File "c:\user\anaconda3\envs\py38\lib\site-packages\pyNastran\op4\op4.py", line 895, in _read_real_binary A = self._read_real_sparse_binary(op4, nrows, ncols, matrix_type, is_big_mat) File "c:\user\anaconda3\envs\py38\lib\site-packages\pyNastran\op4\op4.py", line 914, in _read_real_sparse_binary (icol, irow, nwords) = self.get_markers_sparse(op4, is_big_mat) File "c:\user\anaconda3\envs\py38\lib\site-packages\pyNastran\op4\op4.py", line 1354, in get_markers_sparse unused_a, icol, irow, nwords = self.read_start_marker(op4) File "c:\user\anaconda3\envs\py38\lib\site-packages\pyNastran\op4\op4.py", line 726, in read_start_marker assert self.n == op4.tell(), 'n=%s tell=%s' % (self.n, op4.tell()) AssertionError: n=5177352 tell=5177344

When debug=True I get a different error:

Traceback (most recent call last): File "C:\user\anaconda3\envs\py38\lib\site-packages\IPython\core\interactiveshell.py", line 3441, in run_code exec(code_obj, self.user_global_ns, self.user_ns) File "", line 1, in runfile('C:/user/WF/python/MSCN-PY/test2.py', wdir='C:/SpaceLabs/WF/python/MSCN-PY') File "C:\user\PyCharm Community Edition 2021.2.1\plugins\python-ce\helpers\pydev_pydev_bundle\pydev_umd.py", line 198, in runfile pydev_imports.execfile(filename, global_vars, local_vars) # execute the script File "C:\user\PyCharm Community Edition 2021.2.1\plugins\python-ce\helpers\pydev_pydev_imps_pydev_execfile.py", line 18, in execfile exec(compile(contents+"\n", file, 'exec'), glob, loc) File "C:/user/WF/python/MSCN-PY/test2.py", line 14, in MAT = model.read_op4("./MSCNpyBenchmarkN10/MSCNpyBenchmarkN10_kelems.op4", matrix_names=["KAA"]) File "C:\user\anaconda3\envs\py38\lib\site-packages\pyNastran\op4\op4.py", line 137, in read_op4 return self.read_op4_binary(op4_filename, matrix_names, precision) File "C:\user\anaconda3\envs\py38\lib\site-packages\pyNastran\op4\op4.py", line 686, in read_op4_binary (name, form, matrix) = self._read_matrix_binary(op4, precision, matrix_names) File "C:\user\anaconda3\envs\py38\lib\site-packages\pyNastran\op4\op4.py", line 804, in _read_matrix_binary A = self._read_real_binary(op4, nrows, ncols, Type, is_sparse, is_big_mat) File "C:\user\anaconda3\envs\py38\lib\site-packages\pyNastran\op4\op4.py", line 895, in _read_real_binary A = self._read_real_sparse_binary(op4, nrows, ncols, matrix_type, is_big_mat) File "C:\user\anaconda3\envs\py38\lib\site-packages\pyNastran\op4\op4.py", line 938, in _read_real_sparse_binary self._show(op4, 100, types='qd') File "C:\user\anaconda3\envs\py38\lib\site-packages\pyNastran\op4\op4.py", line 1015, in _show strings, ints, floats = self._show_data(data, types=types, endian=endian) File "C:\user\anaconda3\envs\py38\lib\site-packages\pyNastran\op4\op4.py", line 1044, in _show_data return self._write_data(sys.stdout, data, types=types, endian=endian) File "C:\user\anaconda3\envs\py38\lib\site-packages\pyNastran\op4\op4.py", line 1092, in _write_data doubles = unpack(b'%s%id' % (endian, ndoubles), data[:ndoubles8]) TypeError: %b requires a bytes-like object, or an object that implements bytes*, not 'str'

Python v3.6 & pyNastran v1.2.1 gives this traceback:

Traceback (most recent call last): File "test2.py", line 5, in MAT = model.read_op4("K.op4") File "c:\user\anaconda3\envs\py38\lib\site-packages\pyNastran\op4\op4.py", line 137, in read_op4 return self.read_op4_binary(op4_filename, matrix_names, precision) File "c:\user\anaconda3\envs\py38\lib\site-packages\pyNastran\op4\op4.py", line 686, in read_op4_binary (name, form, matrix) = self._read_matrix_binary(op4, precision, matrix_names) File "c:\user\anaconda3\envs\py38\lib\site-packages\pyNastran\op4\op4.py", line 804, in _read_matrix_binary A = self._read_real_binary(op4, nrows, ncols, Type, is_sparse, is_big_mat) File "c:\user\anaconda3\envs\py38\lib\site-packages\pyNastran\op4\op4.py", line 895, in _read_real_binary A = self._read_real_sparse_binary(op4, nrows, ncols, matrix_type, is_big_mat) File "c:\user\anaconda3\envs\py38\lib\site-packages\pyNastran\op4\op4.py", line 974, in _read_real_sparse_binary value_list = unpack(str_values, data[0:nvalues * nbytes_per_value]) struct.error: bad char in struct format

Moreover, for some attempts I have gotten the same traceback as the first above (ending in AssertionError).

Below you can find a .zip containing the model and a code snippet to reproduce the error. I also added the op4 file produced since it can take some time to run the model.

Thanks in advance!

Kind regards, Joris

test.zip https://github.com/SteveDoyle2/pyNastran/files/7227107/test.zip

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/SteveDoyle2/pyNastran/issues/661, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAICUWOTBUZPB65WNEJZXJTUDSPNZANCNFSM5EWI7JYA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

Jrois commented 3 years ago

Hi Steve,

Thanks for the suggestion. I had not tried that yet. Unfortunately, I couldn't get that working either. I have tried writing the output using

PARAM,EXTOUT,DMIGOP2

After adding matrices to read using the method model.set_additional_matrices_to_read (as suggested by arising errors) I get an message saying there was a FATAL ERROR in the f06:

pyNastran.f06.errors.FatalError: There was a Nastran FATAL Error. Check the F06. last table=b'MUG1'; post=-2

However, upon inspecting the f06, I could not find any fatal errors or other signs of errors.

I also tried to output using

EXTSEOUT(STIF, EXTID=30, DMIGOP2=13)

But that lead to the same results. Moreover, both small and large problems yield the same error.

I understand that you don't have time to look at the problem now. So when you do, please send me an update.

Enjoy you holiday!

SteveDoyle2 commented 3 years ago

Oh I didn’t realize you were trying to do superelements. Those get weird.

Try: PARAM,EXTSEOUT,DMIGPCH

you should be able to read it with the BDF reader and punch=True. Nastran is weird with their definition of a punch file. Sometimes it’s a BDF with BDF cards and sometimes it’s not.

On Mon, Sep 27, 2021 at 1:34 AM Jrois @.***> wrote:

Hi Steve,

Thanks for the suggestion. I had not tried that yet. Unfortunately, I couldn't get that working either. I have tried writing the output using

PARAM,EXTOUT,DMIGOP2

After adding matrices to read using the method model.set_additional_matrices_to_read (as suggested by arising errors) I get an message saying there was a FATAL ERROR in the f06:

pyNastran.f06.errors.FatalError: There was a Nastran FATAL Error. Check the F06. last table=b'MUG1'; post=-2

However, upon inspecting the f06, I could not find any fatal errors or other signs of errors.

I also tried to output using

EXTSEOUT(STIF, EXTID=30, DMIGOP2=13)

But that lead to the same results. Moreover, both small and large problems yield the same error.

I understand that you don't have time to look at the problem now. So when you do, please send me an update.

Enjoy you holiday!

— You are receiving this because you commented.

Reply to this email directly, view it on GitHub https://github.com/SteveDoyle2/pyNastran/issues/661#issuecomment-927609224, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAICUWLHJ6YWV2Z7Y2EK3ETUEANBDANCNFSM5EWI7JYA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

Jrois commented 3 years ago

I am not using superelements, at least not explicitly. I use an API to recreate the model such that all elements have unique nodes. This way the whole model becomes decoupled, which allows me to extract element stiffness matrices for all elements. I am relatively new to Nastran and I'm not yet familiar with superelements so I can't say for sure that this process is related to superelements.

As for the reading of the matrix, I didn't realise you could use the BDF reader to also read punch files. Instead I now wrote my own script that reads the matrix. This however can take some time when large matrices are concerned and I believe OP4 or OP2 would be faster. So if ever you fix the problem with large matrices I would definitely like to know!

Thanks for your time and help

Kind regards, Joris