Fix memory leak caused by undestroyed prefetch_signal_ in roc::Device

[saadrahim]

The root cause of this memory was addressed separately. Please reopen the ticket if there are further concerns.