TorchDSP / torchsig

TorchSig is an open-source signal processing machine learning toolkit based on the PyTorch data handling pipeline.
MIT License
170 stars 38 forks source link

Quantization needs to take max of real and imag independently #166

Closed MattCarrickPL closed 1 month ago

MattCarrickPL commented 1 year ago

The quantize() function computes the maximum quantization based on the magnitude of a complex number but quantizes against real and imaginary independently.

max_value = max(np.abs(tensor)) + 1e-9
bins = np.linspace(-max_value, max_value, num_levels + 1)

# Digitize to bins
quantized_real = np.digitize(tensor.real, bins)
quantized_imag = np.digitize(tensor.imag, bins)

https://github.com/TorchDSP/torchsig/blob/v0.4.2/torchsig/transforms/functional.py#L1029

The solution is to compute the max of real and imaginary independently, then take the max of their max, ex:

max_value = max( np.abs(np.real(tensor)), np.abs(np.imag(tensor)) )
MattCarrickPL commented 1 month ago

Old issue, closing.