Improve Memory Usage Efficiency and Other Miscellaneous - Githubissues

yt-project / libyt

In-situ analysis with yt

https://libyt.readthedocs.io/

BSD 3-Clause "New" or "Revised" License

9 stars 3 forks source link

Improve Memory Usage Efficiency and Other Miscellaneous #63

Closed cindytsai closed 1 year ago

cindytsai commented 1 year ago

Improve Memory Usage Efficiency and Other Miscellaneous

58

New Look Up Grid Hierarchy API

Using passed in NumPy array to look up hierarchy info.
yt_getGridInfo_Dimensions
yt_getGridInfo_LeftEdge
yt_getGridInfo_RightEdge
yt_getGridInfo_ParentId
yt_getGridInfo_Level
yt_getGridInfo_ProcNum
yt_getGridInfo_ParticleCount
yt_getGridInfo_FieldData

Improve Memory Usage

Free grids_local array once yt_commit_grids is done.
Only one copy of full hierarchy inside Python and libyt. libyt look up NumPy array and its info in Python.
Do not allocate new NumPy array in yt and assign the one allocated by libyt to attributes in libyt frontend.
Do not create key-value pair in grid_data if there is no data to append.

Miscellaneous

Pass in particle_count_list and do the sumension to grid_particle_count in libyt frontend.
Make yt_hierarchy_mpi_type, yt_rma_grid_info_mpi_type, yt_rma_particle_info_mpi_type.
Make GitHub Action generate data through code, not directly store data in txt file.
Make example more simple.
Mark in-memory field data read-only API when wrapping data array.
Make general info like MyRank and MySize global.

cindytsai commented 1 year ago

Finally... :sweat_smile: