pydf is failing to get all load steps in a 12GB rst file, but only at a certain load steps and above

mcMunich commented 8 months ago

Before submitting the issue

[X] I have checked for Compatibility issues
[X] I have searched among the existing issues
[X] I am using a Python virtual environment

Description of the bug

The issue is strange. The rst does not appear corrupt, the result values are availlable in Mechanical, but when pypdf is used externally to gray the results for each of the 14 load steps, it will fail at load step 10. We have done some testing on smaller files and there is no issue. We have solved on the same machine with more load steps and still no problem.

The traceback:

Traceback (most recent call last):

 File "c:\Users\maretom02\Documents_LOCAL\GT_TOOLS\Crack_Initiation\Troubleshooting\script_for_temperature.py", line 36, in <module>

   stress_comp = stressop.outputs.fields_container()

                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

 File "C:\Users\maretom02\AppData\Local\Programs\Python\Python311\Lib\site-packages\ansys\dpf\core\outputs.py", line 73, in __call__

   return self.get_data()

          ^^^^^^^^^^^^^^^

 File "C:\Users\maretom02\AppData\Local\Programs\Python\Python311\Lib\site-packages\ansys\dpf\core\outputs.py", line 70, in get_data

   return self._operator.get_output(self._pin, type_output)

          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

 File "C:\Users\maretom02\AppData\Local\Programs\Python\Python311\Lib\site-packages\ansys\dpf\core\dpf_operator.py", line 494, in get_output

   parameters = {type_tuple[2]: type_tuple[1](self, pin)}

                                ^^^^^^^^^^^^^^^^^^^^^^^^

 File "C:\Users\maretom02\AppData\Local\Programs\Python\Python311\Lib\site-packages\ansys\dpf\gate\generated\operator_capi.py", line 404, in operator_getoutput_fields_container

   raise errors.DPFServerException(sError.value)

ansys.dpf.gate.errors.DPFServerException: S:8<-to_nodal_fc:14<-mapdl::rst::S:11<-mapdl::rst::S<-the requested set is out of range, failed to read stresses from rst file.

Steps To Reproduce

This will require some interfacing.

Which Operating System causes the issue?

Windows

Which DPF/Ansys version are you using?

Ansys 2023 R2

Which Python version causes the issue?

3.10

Installed packages

Code ran until recently. Not quite sure why. The .rst is not shareable.

TommasoMarella commented 8 months ago

Hi all, I'm the user with this problem. Here the versions that I have: python version 3.11.3

python -m pip list Package Version

ansys-dpf-core 0.10.0 asttokens 2.2.1 backcall 0.2.0 cachetools 5.3.2 certifi 2023.11.17 charset-normalizer 3.3.2 colorama 0.4.6 comm 0.1.3 contourpy 1.0.7 cycler 0.11.0 debugpy 1.6.7 decorator 5.1.1 easygui 0.98.3 et-xmlfile 1.1.0 executing 1.2.0 fonttools 4.39.4 google-api-core 2.15.0 google-api-python-client 2.110.0 google-auth 2.25.2 google-auth-httplib2 0.1.1 googleapis-common-protos 1.62.0 grpcio 1.60.0 httplib2 0.22.0 idna 3.6 importlib-metadata 7.0.0 ipykernel 6.23.1 ipython 8.14.0 jedi 0.18.2 julia 0.6.1 jupyter_client 8.2.0 jupyter_core 5.3.0 kiwisolver 1.4.4 matplotlib 3.7.1 matplotlib-inline 0.1.6 nest-asyncio 1.5.6 numpy 1.24.3 openpyxl 3.1.2 packaging 23.1 pandas 2.0.2 parso 0.8.3 pickleshare 0.7.5 Pillow 9.5.0 pip 22.3.1 platformdirs 3.5.1 prompt-toolkit 3.0.38 protobuf 4.25.1 psutil 5.9.5 pure-eval 0.2.2 pyasn1 0.5.1 pyasn1-modules 0.3.0 Pygments 2.15.1 pyparsing 3.0.9 python-dateutil 2.8.2 pytz 2023.3 pywin32 306 pyzmq 25.1.0 requests 2.31.0 rsa 4.9 scipy 1.10.1 seaborn 0.12.2 setuptools 65.5.0 six 1.16.0 stack-data 0.6.2 tornado 6.3.2 tqdm 4.66.1 traitlets 5.9.0 tzdata 2023.3 uritemplate 4.1.1 urllib3 2.1.0 wcwidth 0.2.6 xlwings 0.30.8 zipp 3.17.0

PProfizi commented 8 months ago

@mcMunich As discussed offline, please consider checking the load steps detected by DPF in the result file to verify that is correct, using a time_freq_provider or simply by printing the Model.

TommasoMarella commented 8 months ago

Hi @PProfizi!

I'm using a simple code to test it:

_rst = r'C:\my\path\to\rst'

r = Model(rst)
print (r)
components = r.metadata.available_named_selections
print ("COMPONENTS : ")
print (components)

nodes = r.metadata.meshed_region.nodes
print ('TOTAL NODES : ', nodes.n_nodes)

tf = r.metadata.time_freq_support
tf_array=tf.time_frequencies.data
tf_list=list(tf_array)
print ("TIME STEPS (%d) LIST : " %(len(tf_list)))
print (tf_list)

#top = dpf.operators.result.structural_temperature()
#top.inputs.streams_container.connect(r.metadata.streams_provider.outputs.streams_container)
#top.inputs.time_scoping.connect(tf_list)
#top.inputs.requested_location('Nodal')
#t_comp = top.outputs.fields_container()

stressop = dpf.operators.result.stress()
stressop.inputs.streams_container.connect(r.metadata.streams_provider.outputs.streams_container)
stressop.inputs.time_scoping.connect(tf_list)
stressop.inputs.requested_location('Nodal')
stress_comp = stressop.outputs.fields_container()

temp=[]
stress=[]

for t in range (0,len(tf_list)):
    try:
        #temp.append(t_comp[t].data)
        stress.append(stress_comp[t].data)
    except Exception as e:
        print (stress_comp[t])
        print (t)
        print("Temperature for timesteps %d (n. %d) is not found with the following error:" % (stress_comp[t], t))
        print(e)
    print (len(stress_comp[t]))

and this is the ouput:

DPF Model
Static analysis
Unit system: NMM: mm, ton, N, s, mA, degC
Physics Type: Mechanical
Available results:
     -  displacement: Nodal Displacement
     -  reaction_force: Nodal Force
     -  element_nodal_forces: ElementalNodal Element nodal Forces
     -  stress: ElementalNodal Stress
     -  elemental_volume: Elemental Volume
     -  stiffness_matrix_energy: Elemental Energy-stiffness matrix
     -  artificial_hourglass_energy: Elemental Hourglass Energy
     -  thermal_dissipation_energy: Elemental thermal dissipation energy
     -  kinetic_energy: Elemental Kinetic Energy
     -  co_energy: Elemental co-energy
     -  incremental_energy: Elemental incremental energy
     -  elastic_strain: ElementalNodal Strain
     -  plastic_strain: ElementalNodal  Plastic Strain
     -  thermal_strain: ElementalNodal Thermal Strains
     -  thermal_strains_eqv: ElementalNodal Thermal Strains eqv
     -  swelling_strains: ElementalNodal Swelling Strains
     -  structural_temperature: ElementalNodal Temperature
------------------------------
DPF  Meshed Region:
  1615824 nodes
  1022745 elements
  Unit: mm
  With solid (3D) elements, shell (2D) elements, shell (3D) elements
------------------------------
DPF  Time/Freq Support:
  Number of sets: 14
Cumulative     Time (s)       LoadStep       Substep
1              1.000000       1              1
2              425.000000     2              1
3              546.000000     3              1
4              592.000000     4              1
5              800.000000     5              1
6              850.000000     6              1
7              1300.000000    7              1
8              100000.000000  8              1
9              100090.000000  9              1
10             100413.000000  10             1
11             100543.000000  11             1
12             100672.000000  12             1
13             100679.000000  13             1
14             200000.000000  14             1

COMPONENTS :
['REMOTEDISPALL', 'SELECTION_2', '_ALLCONTACTS', '_CM5453_1', '_CM5453_2', '_CM5453_3', 
'_CM5453_5', '_CM5455_1', '_CM5455_2', '_CM5455_3', '_CM5455_4', '_CM5455_5', '_CM5455_6', '_CM5457_1', '_CM5457_2', '_CM5457_3', '_CM5457_4', '_CM5457_5', '_CM5457_6', '_CM5459_1', '_CM5459_2', '_CM5459_3', '_CM5459_5', '_CM5461_1', '_CM5461_2', '_CM5461_3', '_CM5461_5', '_CM5463_1', '_CM5463_2', '_CM5463_3', '_CM5463_5', '_CM5465_1', '_CM5465_2', '_CM5465_3', '_CM5465_5', '_CM5467_1', '_CM5467_2', '_CM5467_3', '_CM5467_5', '_CM5469_1', '_CM5469_2', '_CM5469_3', '_CM5469_5', '_DISPZEROUX', '_DISPZEROUY']
TOTAL NODES :  1615824
TIME STEPS (14) LIST :
[1.0, 425.0, 546.0, 592.0, 800.0, 850.0, 1300.0, 100000.0, 100090.0, 100413.0, 100543.0, 100672.0, 100679.0, 200000.0]
Traceback (most recent call last):
  File "c:\Users\maretom02\Documents_LOCAL\GT_TOOLS\Crack_Initiation\Troubleshooting\script_for_temperature.py", line 37, in <module>
    stress_comp = stressop.outputs.fields_container()
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\maretom02\AppData\Local\Programs\Python\Python311\Lib\site-packages\ansys\dpf\core\outputs.py", line 73, in __call__
    return self.get_data()
           ^^^^^^^^^^^^^^^
  File "C:\Users\maretom02\AppData\Local\Programs\Python\Python311\Lib\site-packages\ansys\dpf\core\outputs.py", line 70, in get_data
    return self._operator.get_output(self._pin, type_output)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\maretom02\AppData\Local\Programs\Python\Python311\Lib\site-packages\ansys\dpf\core\dpf_operator.py", line 494, in get_output
    parameters = {type_tuple[2]: type_tuple[1](self, pin)}
                                 ^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\maretom02\AppData\Local\Programs\Python\Python311\Lib\site-packages\ansys\dpf\gate\generated\operator_capi.py", line 402, in operator_getoutput_fields_container
    res = capi.dll.Operator_getoutput_FieldsContainer(op._internal_obj if op is not None else None, utils.to_int32(iOutput), ctypes.byref(utils.to_int32(errorSize)), ctypes.byref(sError))
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
OSError: exception: access violation reading 0x0000000000000030

all the timesteps are present in the output, but the generation of the operator is wrong in same way. Also using the [time_freq_provider] I saw all the timesteps (the same of the previous case)

PProfizi commented 8 months ago

Thank you @TommasoMarella Could you please try: stress_comp = r.results.stress.on_location(location=dpf.locations.nodal).on_all_time_freqs().eval() instead of the problematic line

TommasoMarella commented 8 months ago

Hi @PProfizi !

Same error:

Traceback (most recent call last):
  File "c:\Users\maretom02\Documents_LOCAL\GT_TOOLS\Crack_Initiation\Troubleshooting\script_for_temperature.py", line 46, in <module>
    stress_comp = r.results.stress.on_location(location=dpf.locations.nodal).on_all_time_freqs().eval()
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\maretom02\AppData\Local\Programs\Python\Python311\Lib\site-packages\ansys\dpf\core\dpf_operator.py", line 677, in eval
    return self.outputs._outputs[0]()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\maretom02\AppData\Local\Programs\Python\Python311\Lib\site-packages\ansys\dpf\core\outputs.py", line 73, in __call__
    return self.get_data()
           ^^^^^^^^^^^^^^^
  File "C:\Users\maretom02\AppData\Local\Programs\Python\Python311\Lib\site-packages\ansys\dpf\core\outputs.py", line 70, in get_data
    return self._operator.get_output(self._pin, type_output)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\maretom02\AppData\Local\Programs\Python\Python311\Lib\site-packages\ansys\dpf\core\dpf_operator.py", line 494, in get_output
    parameters = {type_tuple[2]: type_tuple[1](self, pin)}
                                 ^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\maretom02\AppData\Local\Programs\Python\Python311\Lib\site-packages\ansys\dpf\gate\generated\operator_capi.py", line 402, in operator_getoutput_fields_container
    res = capi.dll.Operator_getoutput_FieldsContainer(op._internal_obj if op is not None else None, utils.to_int32(iOutput), ctypes.byref(utils.to_int32(errorSize)), ctypes.byref(sError))
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
OSError: exception: access violation reading 0x0000000000000030

PProfizi commented 8 months ago

Hi @mcMunich @TommasoMarella, following a discussion with @rafacanton it seems this issue may be linked to a known bug fixed in 2024 R1. Would it be possible to try with DPF Server version 2024.1.pre0?

TommasoMarella commented 8 months ago

Hi @PProfizi,

Yes I can try it. But please consider that we have a script that it is working on the most of the case! Only with one rst file it doesn't work. Have you understood if the bug is related with some "non conventional" option in the rst file?

rafacanton commented 8 months ago

@TommasoMarella I agree this is a severe issue. That's why, if the issue you are exposing is the same that we are referring to, we fixed it straight away when it was first found, and then it is fixed in all subsequent versions of the DPF server. It was not affecting any particular rst option, it could potentially arise for any rst files.

ak2022v commented 6 months ago

Yes, I have the same issue. I have raised this issue in Ansys developer's forum: https://discuss.ansys.com/discussion/2953/oserror-while-trying-to-output-uy-component-data-using-pydpf#latest

TommasoMarella commented 5 months ago

Hello all,

I confirm that the error is still there, different but always there.

Using the new dpf server (2024R1) all the application doesn't work (with all the file rst, while before it worked with most of them). I obtained the following error:

Script ended with error:

Traceback (most recent call last):

 File "/home/crackinitiation/src/src/main.py", line 56, in main

  extracted_properties = run_extraction(job, log_fd)

 File "/home/crackinitiation/src/src/main.py", line 100, in run_extraction

  extracted_properties = perform_extraction(job, log_fd)

 File "/home/crackinitiation/src/src/lcf_data_extraction/extraction_processes_manager.py", line 55, in perform_extraction

  results = perform_extraction_property_split(job, log_fd)

 File "/home/crackinitiation/src/src/lcf_data_extraction/extraction_processes_manager.py", line 90, in perform_extraction_property_split

  time_steps = check_timesteps_number(job, log_fd)

 File "/home/crackinitiation/src/src/lcf_data_extraction/extraction_processes_manager.py", line 531, in check_timesteps_number

  check = ExtractorContainer(job, log_fd)

 File "/home/crackinitiation/src/src/lcf_data_extraction/extraction.py", line 63, in __init__

  self.__handle_exception(e)

 File "/home/crackinitiation/src/src/lcf_data_extraction/extraction.py", line 328, in __handle_exception

  raise e

 File "/home/crackinitiation/src/src/lcf_data_extraction/extraction.py", line 58, in __init__

  self.timesteps, nodes_num = self.__setup()

 File "/home/crackinitiation/src/src/lcf_data_extraction/extraction.py", line 203, in __setup

  tf = self.r.metadata.time_freq_support

 File "/net/opt/apps/prod/python/3.10.5-no-static/lib/python3.10/site-packages/ansys/dpf/core/model.py", line 104, in metadata

  self._metadata = Metadata(self._data_sources, self._server)

 File "/net/opt/apps/prod/python/3.10.5-no-static/lib/python3.10/site-packages/ansys/dpf/core/model.py", line 271, in __init__

  self._set_data_sources(data_sources)

 File "/net/opt/apps/prod/python/3.10.5-no-static/lib/python3.10/site-packages/ansys/dpf/core/model.py", line 407, in _set_data_sources

  self._cache_streams_provider()

 File "/net/opt/apps/prod/python/3.10.5-no-static/lib/python3.10/site-packages/ansys/dpf/core/model.py", line 295, in _cache_streams_provider

  self._stream_provider.inputs.connect(self._data_sources)

AttributeError: 'NoneType' object has no attribute 'connect'

Are some changes to the old code removed/changed in the last released?

ayush-kumar-423 commented 3 months ago

@TommasoMarella Can you please try adding either of these lines (prefer 241 if you have WB 241 installed) at the top of your script (after ansys.dpf.core module import)?

dpf.start_local_server(ansys_path=r"C:\Program Files\ANSYS Inc\v232") or dpf.start_local_server(ansys_path=r"C:\Program Files\ANSYS Inc\v241")

TommasoMarella commented 3 months ago

Hello Ayush, It is not working as well. Another different error, but still an error:

Traceback (most recent call last):
  File "c:\Users\maretom02\AppData\Local\Programs\Python\Python311\Lib\site-packages\ansys\dpf\core\errors.py", line 101, in wrapper
    out = func(*args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^
  File "c:\Users\maretom02\AppData\Local\Programs\Python\Python311\Lib\site-packages\ansys\dpf\core\model.py", line 352, in time_freq_support
    self._time_freq_support = timeProvider.get_output(0, types.time_freq_support)
                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "c:\Users\maretom02\AppData\Local\Programs\Python\Python311\Lib\site-packages\ansys\dpf\core\dpf_operator.py", line 494, in get_output
    parameters = {type_tuple[2]: type_tuple[1](self, pin)}
                                 ^^^^^^^^^^^^^^^^^^^^^^^^
  File "c:\Users\maretom02\AppData\Local\Programs\Python\Python311\Lib\site-packages\ansys\dpf\gate\generated\operator_capi.py", line 570, in operator_getoutput_time_freq_support
    raise errors.DPFServerException(sError.value)
ansys.dpf.gate.errors.DPFServerException: TimeFreqSupportProvider:4<-mapdl::rst::TimeFreqSupportProvider:5<-mapdl::rst::TimeFreqSupportProvider<-Unable to open the following file: C:\Users\maretom02\Documents_LOCAL\GT_TOOLS\Crack_Initiation\Troubleshooting\tanu\Mechanical.rst

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "c:\Users\maretom02\Documents_LOCAL\GT_TOOLS\Crack_Initiation\Troubleshooting\timestepcheck\from ansys.py", line 13, in <module>
    tf = metadata.time_freq_support
         ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "c:\Users\maretom02\AppData\Local\Programs\Python\Python311\Lib\site-packages\ansys\dpf\core\errors.py", line 106, in wrapper
    raise DPFServerException(details)
ansys.dpf.gate.errors.DPFServerException: TimeFreqSupportProvider:4<-mapdl::rst::TimeFreqSupportProvider:5<-mapdl::rst::TimeFreqSupportProvider<-Unable to open the following file: ....

Consider also that I only try to open the model and print it:

from ansys.dpf import core as dpf
server = dpf.start_local_server(ansys_path=r'C:\Program Files\ANSYS Inc\v232')

path2file =r'C:\Users\Mechanical.rst'

model = dpf.Model(path2file)
print(model)

ayush-kumar-423 commented 3 months ago

@TommasoMarella thanks for the update. And this affects all the RSTs or a particular RST?

TommasoMarella commented 3 months ago

We have this issue on 4 different RSTs. I checked also the workbench projects but there are not strange options.

ayush-kumar-423 commented 3 months ago

@TommasoMarella, Can you share any of the problematic RSTs for further investigation?

TommasoMarella commented 3 months ago

Unfortunately I can't, but we can try to debug it in a specific meeting together. let me know

ansys / pydpf-core