JuliaVTK / ReadVTK.jl

Julia package for reading VTK XML files
https://juliavtk.github.io/ReadVTK.jl
MIT License
32 stars 10 forks source link

Reading large .pvts files has extremely high memory consumption compared to Paraview #57

Open FR13ndSDP opened 5 months ago

FR13ndSDP commented 5 months ago

I have a large structured .pvts file, and when I read it with the following code, I was surprised to find that the memory always overflows, but Paraview runs normally. It seems that getting the coordinates is very memory expensive:

using ReadVTK

fname = "./plt-100.pvts"

vtk = PVTKFile(fname)

# point data
p_data = get_point_data(vtk)

# mesh cordinate
x,y,z = get_coordinate_data(vtk)
Nx, Ny, Nz = size(x)

# variables
p = get_data_reshaped(p_data["p"])
u = get_data_reshaped(p_data["u"])
v = get_data_reshaped(p_data["v"])
T = get_data_reshaped(p_data["T"])
ρ = get_data_reshaped(p_data["rho"])

File metadata:

<?xml version="1.0" encoding="utf-8"?>
<VTKFile type="PStructuredGrid" version="1.0" byte_order="LittleEndian">
  <PStructuredGrid GhostLevel="0" WholeExtent="0 2799 0 599 0 599">
sloede commented 5 months ago

It is very likely you are the first person to use this code for a large file, so there might very well be some inefficiencies nobody has noticed yet 😅. Can you boil it down to which call exactly is the one that causes the most allocations?

FR13ndSDP commented 5 months ago

It is very likely you are the first person to use this code for a large file, so there might very well be some inefficiencies nobody has noticed yet 😅. Can you boil it down to which call exactly is the one that causes the most allocations?

Sure, I'll see what I can do.