feat: reduce computations in backprop of `lfilter`

yoyolicoris commented 2 months ago

This PR update the backpropagation computation of DifferentiableIIR. The update is based on my recent work (https://arxiv.org/abs/2406.05128), which uses just one DifferentiableIIR::apply instead of two to compute the gradients for both input and a_coeffs. The algorithm has been tested in torchlpc.

Below is the benchmark to version 2.4.1 lfilter. The backward computation runs slightly faster especially when using just one thread.

v2.4.1

[-------------- IIR filter -------------]
                  |  forward  |  backward
1 threads: ------------------------------
      [32, 256]   |    404.0  |   1104.5
      [32, 1024]  |    694.3  |   1783.2
      [32, 4096]  |   2637.1  |   6579.2
2 threads: ------------------------------
      [32, 256]   |    381.5  |   1110.2
      [32, 1024]  |    574.7  |   1638.3
      [32, 4096]  |   1729.0  |   5724.7
4 threads: ------------------------------
      [32, 256]   |    362.9  |   1084.0
      [32, 1024]  |    502.0  |   1537.7
      [32, 4096]  |   1485.5  |   5376.4

Times are in microseconds (us).

This version

[-------------- IIR filter -------------]
                  |  forward  |  backward
1 threads: ------------------------------
      [32, 256]   |    391.9  |    935.1
      [32, 1024]  |    678.7  |   1499.8
      [32, 4096]  |   2481.2  |   5782.2
2 threads: ------------------------------
      [32, 256]   |    371.5  |    994.5
      [32, 1024]  |    550.3  |   1551.0
      [32, 4096]  |   1655.0  |   5896.0
4 threads: ------------------------------
      [32, 256]   |    349.1  |    981.6
      [32, 1024]  |    484.8  |   1518.2
      [32, 4096]  |   1380.2  |   4969.1

Times are in microseconds (us).

pytorch-bot[bot] commented 2 months ago

:link: Helpful Links

:test_tube: See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/audio/3831

:page_facing_up: Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

:x: 2 New Failures, 3 Cancelled Jobs, 6 Unrelated Failures

As of commit 28575a77b065e3bc206027172ab21bf0deb86db7 with merge base 97ed7b36b7a741253d4e41e4da3c901d83294503 ():

NEW FAILURES - The following jobs have failed:

* [Integration Test / build (3.8)](https://hud.pytorch.org/pr/pytorch/audio/3831#30142722292) ([gh](https://github.com/pytorch/audio/actions/runs/10861157218/job/30142722292)) * [Unit-tests on Linux GPU / tests (3.10, 11.8) / linux-job](https://hud.pytorch.org/pr/pytorch/audio/3831#30142722438) ([gh](https://github.com/pytorch/audio/actions/runs/10861157225/job/30142722438)) `RuntimeError: Command docker exec -t 273ccb60ecc5704c8cc9801f73771505967f02bfa36fb7d13c0088b1622322d2 /exec failed with exit code 2`

CANCELLED JOBS - The following jobs were cancelled. Please retry:

* [Unit-tests on Linux CPU / tests (3.10) / linux-job](https://hud.pytorch.org/pr/pytorch/audio/3831#30142722518) ([gh](https://github.com/pytorch/audio/actions/runs/10861157222/job/30142722518)) `##[error]The operation was canceled.` * [Unit-tests on Linux CPU / tests (3.8) / linux-job](https://hud.pytorch.org/pr/pytorch/audio/3831#30142722609) ([gh](https://github.com/pytorch/audio/actions/runs/10861157222/job/30142722609)) `##[error]The operation was canceled.` * [Unit-tests on Linux CPU / tests (3.9) / linux-job](https://hud.pytorch.org/pr/pytorch/audio/3831#30142722732) ([gh](https://github.com/pytorch/audio/actions/runs/10861157222/job/30142722732)) `##[error]The operation was canceled.`

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

* [Build Linux Wheels / pytorch/audio / upload / manywheel-py3_9-cpu](https://hud.pytorch.org/pr/pytorch/audio/3831#30142988214) ([gh](https://github.com/pytorch/audio/actions/runs/10861157251/job/30142988214)) (detected as infra flaky with no log or failing log classifier) * [Unittests on Windows CPU / unittests-windows-cpu / windows-job](https://hud.pytorch.org/pr/pytorch/audio/3831#30142722333) ([gh](https://github.com/pytorch/audio/actions/runs/10861157226/job/30142722333)) (detected as infra flaky with no log or failing log classifier)

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

* [Unit-tests on Linux GPU / tests (3.8, 11.8) / linux-job](https://hud.pytorch.org/pr/pytorch/audio/3831#30142722579) ([gh](https://github.com/pytorch/audio/actions/runs/10861157225/job/30142722579)) ([trunk failure](https://hud.pytorch.org/pytorch/audio/commit/97ed7b36b7a741253d4e41e4da3c901d83294503#30015022717)) `##[error]The operation was canceled.` * [Unit-tests on Linux GPU / tests (3.9, 11.8) / linux-job](https://hud.pytorch.org/pr/pytorch/audio/3831#30142722686) ([gh](https://github.com/pytorch/audio/actions/runs/10861157225/job/30142722686)) ([trunk failure](https://hud.pytorch.org/pytorch/audio/commit/97ed7b36b7a741253d4e41e4da3c901d83294503#30015023211)) `torchaudio_unittest/models/hdemucs/hdemucs_gpu_test.py::HDemucsFloat32GPUTest::test_import_recreate_low_model_3` * [Unit-tests on Macos CPU / tests / macos-job](https://hud.pytorch.org/pr/pytorch/audio/3831#30142722360) ([gh](https://github.com/pytorch/audio/actions/runs/10861157230/job/30142722360)) ([trunk failure](https://hud.pytorch.org/pytorch/audio/commit/97ed7b36b7a741253d4e41e4da3c901d83294503#30015021251)) `torchaudio_unittest/prototype/functional/pyroomacoustics_compatibility_test.py::CompatibilityTest::test_simulate_rir_ism_single_band_1` * [Unittests on Windows GPU / unittests-windows-gpu / windows-job](https://hud.pytorch.org/pr/pytorch/audio/3831#30142722466) ([gh](https://github.com/pytorch/audio/actions/runs/10861157240/job/30142722466)) ([trunk failure](https://hud.pytorch.org/pytorch/audio/commit/97ed7b36b7a741253d4e41e4da3c901d83294503#30015021592)) `torchaudio_unittest\prototype\functional\pyroomacoustics_compatibility_test.py::CompatibilityTest::test_simulate_rir_ism_single_band_1`

This comment was automatically generated by Dr. CI and updates every 15 minutes.

yoyolicoris commented 2 months ago

Thanks, looks good. Do you want to update the docstring to mention the new algorithm / paper?

Oh, sure, I totally forgot. Will update the docstring later this week.

yoyolicoris commented 2 months ago

The code passes related unit tests on my computer.

pytorch / audio