This was reported by Greg Fischer. The attached files represent the same object in both Silo and binary tecplot format. When I do a filled boundary plot of 'material' from the silo file in VisIt, it eats over 8 gigs of memory and then cores. This is our current trunk on Linux. Greg was testing on Windows and had similar problems; memory bloat and eventual crash of engine. Additionally, the same data from Tecplot takes 3x longer to read and put up a mesh plot and renders each frame much, much slower. Greg Fischer says Tecplot itself handles the tecplot file fine.
-----------------------REDMINE MIGRATION-----------------------
This ticket was migrated from Redmine. As such, not all
information was able to be captured in the transition. Below is
a complete record of the original redmine ticket.
Ticket number: 1716
Status: Pending
Project: VisIt
Tracker: Bug
Priority: Normal
Subject: material boundary plot taking gobbs of memory and then coreing
Assigned to: -
Category: -
Target version: -
Author: Mark Miller
Start: 01/28/2014
Due date:
% Done: 0%
Estimated time:
Created: 01/28/2014 05:07 pm
Updated: 02/05/2014 11:25 am
Likelihood: 3 - Occasional
Severity: 4 - Crash / Wrong Results
Found in version: 2.7.0
Impact:
Expected Use:
OS: All
Support Group: Any
Description:
This was reported by Greg Fischer. The attached files represent the same object in both Silo and binary tecplot format. When I do a filled boundary plot of 'material' from the silo file in VisIt, it eats over 8 gigs of memory and then cores. This is our current trunk on Linux. Greg was testing on Windows and had similar problems; memory bloat and eventual crash of engine. Additionally, the same data from Tecplot takes 3x longer to read and put up a mesh plot and renders each frame much, much slower. Greg Fischer says Tecplot itself handles the tecplot file fine.
Comments:
I did just a quick analysis from debug logs. The engine indeed does run out of memory and debug logs show...I found the section of code where the message Setting up connectivity array for occurs. It is in avt/MIR/Base/MIRConnectivity.C, MIRConnectivity::SetUpConnectivity.There a few allocations totaling approximately 11 ncells (427426400=72760800 cells) sizeof(int) or approximately 3.05 Gigabytes. So, somewhere there is another 5 Gigabytes being used up to total 8 gigabytes.Here is a stack dump from valgrind 4172 new/new[] failed and should throw an exception, but Valgrind 4172 cannot throw exceptions and so is aborting instead. Sorry. ==4172== at 0x4C268EB: VALGRIND_PRINTF_BACKTRACE (valgrind.h:4550) ==4172== by 0x4C26CDD: operator new[](unsigned long) (vg_replace_malloc.c:363) ==4172== by 0x97E270D: VisItArray::reserve(int) (VisItArray.h:106) ==4172== by 0x97E11EF: VisItArray::push_back(long long const&) (VisItArray.h:118) ==4172== by 0x981FF9A: ZooMIR::ReconstructCleanMesh(vtkDataSet, avtMaterial) (ZooMIR.C:868) ==4172== by 0x981D47F: ZooMIR::ReconstructMesh(vtkDataSet, avtMaterial, int) (ZooMIR.C:200) ==4172== by 0x981D1B6: ZooMIR::Reconstruct3DMesh(vtkDataSet, avtMaterial) (ZooMIR.C:127) ==4172== by 0x94BF2F2: avtGenericDatabase::GetMIR(int, char const, int, vtkDataSet, avtMaterial, int, bool, bool, bool, bool, int, int, int, float, float, int, bool, bool&, bool&, bool, avtMaterial&) (avtGenericDatabase.C:4679) ==4172== by 0x94BB5A0: avtGenericDatabase::MaterialSelect(vtkDataSet, avtMaterial, std::vector<avtMixedVariable, std::allocator<avtMixedVariable> >, int, char const, int, std::vector<std::string, std::allocator >&, std::vector<std::string, std::allocator >&, bool, bool, bool, bool, bool, bool, bool, int, int, int, float, float, int, bool, bool&, bool&, bool) (avtGenericDatabase.C:3732) ==4172== by 0x94CA97F: avtGenericDatabase::MaterialSelect(avtDatasetCollection&, std::vector<int, std::allocator >&, ref_ptr&, avtSourceFromDatabase, bool) (avtGenericDatabase.C:7712) ==4172== by 0x94B2464: avtGenericDatabase::GetOutput(ref_ptr, avtSourceFromDatabase*) (avtGenericDatabase.C:837) ==4172== by 0x94EBC07: avtSourceFromDatabase::FetchDataset(ref_ptr, ref_ptr&) (avtSourceFromDatabase For comparison purposes, I did a test using threshold operator. I created a scalar variable using the expression coord(mesh)[0] and then thresholded min...800. That took about a minute to run and chewed up about 2.5 gigabytes but otherwise ran fine and produced a plot. So, it appears that threshold operator, which is basically doing the same thing in terms of converting the mesh to an unstructured mesh, is able to do so much more efficiently than MIR does.
biagas
commented
5 years ago
This was reported by Greg Fischer. The attached files represent the same object in both Silo and binary tecplot format. When I do a filled boundary plot of 'material' from the silo file in VisIt, it eats over 8 gigs of memory and then cores. This is our current trunk on Linux. Greg was testing on Windows and had similar problems; memory bloat and eventual crash of engine. Additionally, the same data from Tecplot takes 3x longer to read and put up a mesh plot and renders each frame much, much slower. Greg Fischer says Tecplot itself handles the tecplot file fine.
-----------------------REDMINE MIGRATION----------------------- This ticket was migrated from Redmine. As such, not all information was able to be captured in the transition. Below is a complete record of the original redmine ticket.
Ticket number: 1716 Status: Pending Project: VisIt Tracker: Bug Priority: Normal Subject: material boundary plot taking gobbs of memory and then coreing Assigned to: - Category: - Target version: - Author: Mark Miller Start: 01/28/2014 Due date: % Done: 0% Estimated time: Created: 01/28/2014 05:07 pm Updated: 02/05/2014 11:25 am Likelihood: 3 - Occasional Severity: 4 - Crash / Wrong Results Found in version: 2.7.0 Impact: Expected Use: OS: All Support Group: Any Description: This was reported by Greg Fischer. The attached files represent the same object in both Silo and binary tecplot format. When I do a filled boundary plot of 'material' from the silo file in VisIt, it eats over 8 gigs of memory and then cores. This is our current trunk on Linux. Greg was testing on Windows and had similar problems; memory bloat and eventual crash of engine. Additionally, the same data from Tecplot takes 3x longer to read and put up a mesh plot and renders each frame much, much slower. Greg Fischer says Tecplot itself handles the tecplot file fine.
Comments: I did just a quick analysis from debug logs. The engine indeed does run out of memory and debug logs show...I found the section of code where the message Setting up connectivity array for occurs. It is in avt/MIR/Base/MIRConnectivity.C, MIRConnectivity::SetUpConnectivity.There a few allocations totaling approximately 11 ncells (427426400=72760800 cells) sizeof(int) or approximately 3.05 Gigabytes. So, somewhere there is another 5 Gigabytes being used up to total 8 gigabytes.Here is a stack dump from valgrind 4172 new/new[] failed and should throw an exception, but Valgrind 4172 cannot throw exceptions and so is aborting instead. Sorry. ==4172== at 0x4C268EB: VALGRIND_PRINTF_BACKTRACE (valgrind.h:4550) ==4172== by 0x4C26CDD: operator new[](unsigned long) (vg_replace_malloc.c:363) ==4172== by 0x97E270D: VisItArray::reserve(int) (VisItArray.h:106) ==4172== by 0x97E11EF: VisItArray::push_back(long long const&) (VisItArray.h:118) ==4172== by 0x981FF9A: ZooMIR::ReconstructCleanMesh(vtkDataSet, avtMaterial) (ZooMIR.C:868) ==4172== by 0x981D47F: ZooMIR::ReconstructMesh(vtkDataSet, avtMaterial, int) (ZooMIR.C:200) ==4172== by 0x981D1B6: ZooMIR::Reconstruct3DMesh(vtkDataSet, avtMaterial) (ZooMIR.C:127) ==4172== by 0x94BF2F2: avtGenericDatabase::GetMIR(int, char const, int, vtkDataSet, avtMaterial, int, bool, bool, bool, bool, int, int, int, float, float, int, bool, bool&, bool&, bool, avtMaterial&) (avtGenericDatabase.C:4679) ==4172== by 0x94BB5A0: avtGenericDatabase::MaterialSelect(vtkDataSet, avtMaterial, std::vector<avtMixedVariable, std::allocator<avtMixedVariable> >, int, char const, int, std::vector<std::string, std::allocator >&, std::vector<std::string, std::allocator >&, bool, bool, bool, bool, bool, bool, bool, int, int, int, float, float, int, bool, bool&, bool&, bool) (avtGenericDatabase.C:3732) ==4172== by 0x94CA97F: avtGenericDatabase::MaterialSelect(avtDatasetCollection&, std::vector<int, std::allocator >&, ref_ptr&, avtSourceFromDatabase , bool) (avtGenericDatabase.C:7712) ==4172== by 0x94B2464: avtGenericDatabase::GetOutput(ref_ptr, avtSourceFromDatabase*) (avtGenericDatabase.C:837) ==4172== by 0x94EBC07: avtSourceFromDatabase::FetchDataset(ref_ptr, ref_ptr&) (avtSourceFromDatabase For comparison purposes, I did a test using threshold operator. I created a scalar variable using the expression coord(mesh)[0] and then thresholded min...800. That took about a minute to run and chewed up about 2.5 gigabytes but otherwise ran fine and produced a plot. So, it appears that threshold operator, which is basically doing the same thing in terms of converting the mesh to an unstructured mesh, is able to do so much more efficiently than MIR does.