Backend documentation.

[RuRo]

I wasn't able to find a comprehensive description of the available backends in the documentation (CPU backends in particular).

Here are some questions, that don't seem to be addressed in the current documentation. (It's possible, that I just suck at RTFM, but I wasn't able to find this info)

Backends

Is there a list of all supported backends with proper descriptions for them somewhere?

• gpu and cpu backends are documented pretty well, but they seem to be "virtual" backends that get automatically converted to some concrete backend, and it's not very clear, how this process works. The docs only mention that ti.gpu uses ti.cuda if it's available, but that's it.

• x64, x86_64 and arm64 are barely mentioned in the documentation.

  • I am assuming that these are concrete versions of ti.cpu?
  • Is x64 and x86_64 the same thing?
  • Are these just exposing the arch from llvm or are there some actual functional differences between x64 and arm processors in taichi?
  • Are they sequential backends (see Multithreading section)

• metal, opengl and vulkan

  • seem to be concrete versions of ti.gpu, but it's not clear how taichi selects one of them if CUDA is missing
  • a comparison table (performance, features, notes) would be nice

• cc seems to be the backend for the "Export Taichi kernels to C source" feature, but once again there is not a lot of info about it.

  • Does this backend result in strictly sequential code?

• js, wasm are mentioned in passing on the "Export Taichi kernels to C source" page

• dx11, opencl and amdgpu are barely mentioned in the documentation

Multithreading

The ti.init method has a cpu_max_num_threads parameter, which seems to imply that there is some CPU parallelism at play here, but the documentation doesn't adequately explain

• what multithreading mechanism is used:

  • There are some references to OpenMP and TBB in older commits, but it looks like they were removed at some point.
  • I think, that currently it's just plain C++ threads, but not 100% sure. I was under the impression, that C++ threads aren't very good for this kind of parallelism, so maybe I am wrong.
  • Anyway, this should be mentioned in the documentation, IMO.

• how are these threads used:

  • by taichi internally (during compilation, optimization, by llvm, etc)
  • by the cpu backend directly (e.g. the parallel for loops are parallelized across cpu_max_num_threads threads)
  • by other backends for parallel scheduling and stuff like that
  • some combination of the above

[bobcao3]

Hi! The cpu_max_num_threads is only for the CPU backend. (currently only x64 and arm64) For other backends this would not make a difference. As for the actual thread scheduling, I don't think we schedule much at all. We use grid strided loops and we only launch enough threads to occupy the CPU. I don't think openmp or TBB is used currently anywhere. As for the codegen / runtime, they do not use any form of threading currently (there are some attempts at multi threaded compiling iirc), the threads are only for the CPU.

As for other GPU backends, we are working on bench marking framework that can compare accross backends, and we should probably add some documentation about the arch selection process. @k-ye

[qiao-bo]

Agree that much backend related information is missing in the doc. A section on backend selection process sounds a good plan.

[k-ye]

I wasn't able to find a comprehensive description of the available backends in the documentation (CPU backends in particular).

Here are some questions, that don't seem to be addressed in the current documentation. (It's possible, that I just suck at RTFM, but I wasn't able to find this info)

Thanks for bringing this up @RuRo . These are really great questions!

Backends

Is there a list of all supported backends with proper descriptions for them somewhere?

The best you could find as of now is in https://docs.taichi.graphics/. Please do note that only the CPU/CUDA backends are feature complete currently. Other backends mostly only support dense SNodes.

• gpu and cpu backends are documented pretty well, but they seem to be "virtual" backends that get automatically converted to some concrete backend, and it's not very clear, how this process works. The docs only mention that ti.gpu uses ti.cuda if it's available, but that's it.

Maybe the best way to explain this for now is a code walkthrough.

You're absolutely right that gpu is just a virtual name for a series of physical backends:

https://github.com/taichi-dev/taichi/blob/d51fe9968693397a178902d865f3baba6e32ee0d/python/taichi/lang/misc.py#L153

When you pass in ti.init(arch=ti.gpu), eventually the code goes here and iterates over the GPU list:

https://github.com/taichi-dev/taichi/blob/d51fe9968693397a178902d865f3baba6e32ee0d/python/taichi/lang/misc.py#L642-L653

+1 for documenting this part. @turbo0628 @bobcao3 @qiao-bo

• x64, x86_64 and arm64 are barely mentioned in the documentation.

  • I am assuming that these are concrete versions of ti.cpu?

Yep

• Is x64 and x86_64 the same thing?

I think so, to be confirmed, tho...

• Are these just exposing the arch from llvm or are there some actual functional differences between x64 and arm processors in taichi?

They are just exposing the archs.

• Are they sequential backends (see Multithreading section)

  • metal, opengl and vulkan

• seem to be concrete versions of ti.gpu, but it's not clear how taichi selects one of them if CUDA is missing

• a comparison table (performance, features, notes) would be nice

  • cc seems to be the backend for the "Export Taichi kernels to C source" feature, but once again there is not a lot of info about it.

Right. Please don't use this one, it's a pretty legacy stuff.

• Does this backend result in strictly sequential code?
  • js, wasm are mentioned in passing on the "Export Taichi kernels to C source" page
  • dx11, opencl and amdgpu are barely mentioned in the documentation

None of these have been past their experimental stage :-( Would be great if we can have more people help us develop these backends..

Again, these are great questions. We should really make sure the doc covers them well. cc: @Vissidarte-Herman @strongoier @ailzhang

[ailzhang]

@neozhaoliang would you mind help adding this to the docs? Thanks a lot!

[writinwaters]

Agreed that these topics have not been adequately addressed and that RuRo has raised really great questions. I'll make sure this request takes higher priority.