search

ggerganov / ggml

Tensor library for machine learning
MIT License
11.23k stars 1.04k forks source link

Is there interest in `ggml_reduce` or `ggml_add_ext`? #868

Open balisujohn opened 4 months ago

balisujohn commented 4 months ago

So I need to reduce a 4d tensor along a dimension with the operation addition. I can either add ggml_add_ext that lets you specify a dimension for reduction, or I can add a new op ggml_reduce that lets you specify a dimension and an op as an argument (maybe +,/,-,* to start) and reduces along that dimension with that op. Which of these would be preferable?

In the meantime, I can implement this in tortoise.cpp with view slices and and a for loop, but I think a inbuilt reduction op will probably be faster.

balisujohn commented 4 months ago

it occurs to me ggml_add isn't a unary op, so I'd lean towards the ggml_reduce idea.

ggerganov commented 4 months ago

There is already ggml_sum_rows(). A ggml_sum_dim() should be possible to implement via ggml_permute() + ggml_sum_rows() + ggml_reshape() I think, without having to write new kernels