Use pointers rather than pin to simplify the mixers buffer handling

[gwilymk]

Mentioned by @corwinkuiper that this might be a nicer way of handling this

• [x] no changelog update needed