taichi-dev / taichi

Productive, portable, and performant GPU programming in Python.
https://taichi-lang.org
Apache License 2.0
25.52k stars 2.28k forks source link

How to make an infinitely sized sparse field? #6797

Open Rubydesic opened 1 year ago

Rubydesic commented 1 year ago

Sparse fields let us save space if our region of interest is much smaller than our bounding volume - but what if we don't have a bounding volume? (or the bounding volume is extremely large, i.e. Int.MIN_VALUE to Int.MAX_VALUE). What's the best way to approach this with Taichi?

strongoier commented 1 year ago

or the bounding volume is extremely large, i.e. Int.MIN_VALUE to Int.MAX_VALUE

When you are saying this, you've realized that there is always a bounding volume. Then let's just use that in Taichi.

Rubydesic commented 1 year ago

The problem is then I have to either allocate a very large grid of pointers, which takes up too much memory, or use many levels of pointer SNodes, which is bad for performance. Ideally I would want something like a hashtable, but I'm not sure how I would implement that in taichi.

edit: what I want is essentially a 3D sparse grid where the dimensions are 2^32 x 2^32 x 2^32, but without exhausting all of the GPU memory.

strongoier commented 1 year ago

Ah I got it. Taichi used to have a hash SNode but it isn't available now. We are planning to restore its support (#4569). For now, unfortunately, you may need multi levels of pointer SNodes.