noisetorch / NoiseTorch

Real-time microphone noise suppression on Linux.
Other
9.12k stars 228 forks source link

Issues with incompatible devices #30

Open FDrico opened 4 years ago

FDrico commented 4 years ago

I've been reading the wiki and a prior issue related to what seems to be a very similar problem, but i've been unable yet to solve it using the techniques described by others. As requested, i'm opening this new issue.

pulseaudio --version: pulseaudio 13.0

$ pactl list sources short
0   alsa_output.pci-0000_00_1f.3.analog-stereo.monitor  module-alsa-card.c  s16le 2ch 44100Hz   RUNNING
1   alsa_input.pci-0000_00_1f.3.analog-stereo   module-alsa-card.c  s16le 2ch 44100Hz   RUNNING
2   echoCancel_source   module-echo-cancel.c    float32le 1ch 32000Hz   RUNNING
3   echoCancel_sink.monitor module-echo-cancel.c    float32le 1ch 32000Hz   RUNNING
4   record_mono module-remap-source.c   s16le 1ch 44100Hz   RUNNING
5   nui_mic_denoised_out.monitor    module-null-sink.c  s16le 2ch 48000Hz   RUNNING
6   nui_mic_raw_in.monitor  module-ladspa-sink.c    float32le 2ch 48000Hz   RUNNING
7   nui_mic_remap   module-remap-source.c   s16le 2ch 48000Hz   RUNNING

I'm attaching the output of pacmd list-modules. modules.txt

I'm also uploading the noisetorch.log file. noisetorch.log

Things i tried: 1) I've set the Noise Activation Threshold to 0. 2) I've checked the Input Devices tab in pavucontrol, The only bar not moving is the NoiseTorch Microphone. On this, i think i should provide some more details: I have enabled the echo cancelling with a PulseAudio module, so there is already a virtual microphone. I have disabled it, however, already many times while testing NoiseTorch, and that hasn't solved the problem. I'm attaching a screenshot where you can also check that NoiseTorch is not my default microphone. image 3) I went to the "Output devices" tab and talked to the microphone again. None of the bars moved, not even a little. image 4) I've modified the file /etc/pulse/daemon.conf to set the sample rate to 44100. In the wiki, it is stated that the modified file should be in ~/.pulse/, but i don't have that directory enabled. 5) I've disabled the suspend-on-idle pulseaudio module .

Just in case it is of any help, i'm also attaching the "Recording" tab in pavucontrol, as NoiseTorch seems to modify it. The three first bars move when i speak, but the last one does not. image

The output of my pacmd list-sources list-sources

Ok, while creating this issue, i got a log with journalctl -n300 --no-pager -b /usr/bin/pulseaudio and i can see there is a problem with module-snap-policy.c. log

i'd appreciate any help! And thank you for your work! :)

lawl commented 4 years ago

(Sorry, I should have taken the time to read your issue thoroughly, just as you took time to write a good issue, deleted my previous quickfire response)

So yes, my first suspicion was the noise-canceling module. Did you unload it during runtime, or remove it from the config and restart pulse?

Your noisetorch log looks normal. So as far as it's concerned everything should be fine. I see you have an issue with it getting stuck during unloading, which will eventually be fixed with #27 , but the work around is to just restart noisetorch.

As for your PulseAudio log, can you try again with | grep -v AppArmor this log spam is from a buggy patch canonical applies, but is harmless as long as we're not running as a snap.

The only thing that does look abnormal to me is in fact your "Recording" tab. As on my test machine, it does not get modified and show "Remapped Stream". But that might just be since you're running a different version of pavucontrol.

The next thing I see is that "Loopback to LADSPA ..." does seem to show some activity as there's a small blue bar, but none on the last entry.

What does all of that mean? No clue, I think we should try looking at the pulse log without the AppArmor spam and see if we can find something.

If that also doesn't seem to show anything, you could try enabling verbose logging in pulse.

FDrico commented 4 years ago

Thank you for your interest!

The way i disable the noise-cancelling module is by commenting out the load-module module-echo-cancel... line in /etc/pulse/default.pa, and killing and starting pulseaudio (pulseaudio -k && pulseaudio --start). Here is my default.pa file, just in case you are curious. I haven't touched it much, besides adding (and commenting out, as can be seen) some lines, in my quest to denoise my mic a little bit.

Well, anyway, i disabled it again to show you what happens. Disabling the echo-cancelling module and firing up NoiseTorch again, the "remapped stream" desappears, but the NoiseTorch mic still won't work.

image image image

The new log, without the AppArmor spam, is here. You'll see that i disabled again the echo cancelling a few minutes ago, so at first there will be many lines regarding that module.

I will post the verbose log in a few minutes, if i can get it work! Couldn't get it to work easily. Will try later with more time if you think it would help.

lawl commented 4 years ago

Verbose log may not be neccessary

jul 19 13:13:40 fede-dk pulseaudio[12612]: W: [pulseaudio] module-loopback.c: Cannot set requested sink latency of 9223372036854742,00 ms, adjusting to 2000,00 ms

I think this is the issue. The same one as in #16 which I currently believe to be a pulseaudio/alsa/driver bug (I mean there's definitely a bug there somewhere, the question is if we're also doing something wrong). But I'm not sure. There may be something we could try, if I send you a custom build/source patch for NoiseTorch

Let me dig through the PulseAudio source for a bit.

lawl commented 4 years ago

Okay, I've gotten a bit further with this.

This bug is triggered by sources that don't support dynamic latency. On my device this would look like this:

    flags: DECIBEL_VOLUME LATENCY DYNAMIC_LATENCY

on both reports where the latency underflows, or show an extremely high number in the logs that gets clamped for other reasons, this looks like this:

    flags: HARDWARE HW_MUTE_CTRL DECIBEL_VOLUME LATENCY 

or

        flags: HARDWARE HW_MUTE_CTRL HW_VOLUME_CTRL DECIBEL_VOLUME LATENCY 

with seems to match up with the suspicious code path in pulse. This gives me an idea on how to write a work-around for this, may require some patience from you, this might be done very soon, or take a while.

lawl commented 4 years ago

Okay, that went faster than expected, now we need to hope it works. Here's a build:

noisetorch.zip

You can unpack the binary in the zip, mark it executable chmod +x noisetorch and run it, and hopefully it will just work.

If you prefer to build from source, you can find this experimental patch in this branch here: https://github.com/lawl/NoiseTorch/tree/fixed-latency

If it doesn't work, please attach /tmp/noisetorch.log again.

FDrico commented 4 years ago

Just tested it (downloaded, extracted, made it executable and run it), and sadly, it doesn't seem to solve the problem. Same behavior, as far as i can see.

Here is the noisetorch.log file.

Thank you for your time!

lawl commented 4 years ago

Are you sure you were running the right version? Because there's a new log line that should be printed unconditionally that is missing from the log, and I just double checked if i uploaded the correct file, and it prints it for me.

2020/07/19 19:53:23 Loaded ladspa sink as idx: 26
2020/07/19 19:53:24 Current input device supports dynamic latency: ...

it should now print if the device is dynamic latency or not, this line is missing in your new log.

lawl commented 4 years ago

Did you maybe type noisetorch instead of ./noisetorch and executed the wrong one?

lawl commented 4 years ago

@SpartanJ @midget3111 Sorry for pinging you again (just tell me if I should stop)

We may have a new lead on the underflowing latency issue in noisetorch and an experimental build if any of you want to give it a try.

FDrico commented 4 years ago

Oops, so embarrasing! You were right. Sorry! Here is the log file. Still not working! (will be able to continue on this in about 2 hours, sorry)

lawl commented 4 years ago

Does the pulse log still say the same thing?

You could also try this one: noisetorch.zip

Same thing, but I increased the time by 5ms. Maybe it helps. Sorry to spam you guys with experimental builds, but unfortunately I do not have the hardware to reproduce this, so I have no other option.

If this also doesn't work, this one sets the latency to 1k, won't be usable but may help figure what's going on, would be happy if those affected can test these, and post pulse log + noisetorch logs.

noisetorch2.zip

SpartanJ commented 4 years ago

@SpartanJ @midget3111 Sorry for pinging you again (just tell me if I should stop)

Interesting, great. I tried the latest build but still crashes PulseAudio. I was able to make this work with a workaround. First of all my microphone wasn't working because of some cable mismanagement, I thought that that was going to fix the crash but it didn't. My workaround consists of adding the echo canceling module and loading the NoiseTorch module using as input the echo canceling input, that way I avoid interacting directly with the real microphone input and PulseAudio doesn't crash.

The pulse audio module load:

load-module module-echo-cancel source_name=echosource aec_method=webrtc
set-default-source echosource

And this is how it looks configured: Screenshot_20200719_175707

Edit: The "solution" seems to be a little bit unstable too, PulseAudio crashed at some point with latency errors:

Jul 19 17:56:05 makyo pulseaudio[24418]: Cannot set requested sink latency of 2666.00 ms, adjusting to 2000.00 ms
Jul 19 17:56:05 makyo pulseaudio[24418]: Cannot set requested source latency of 2666.00 ms, adjusting to 100.14 ms
Jul 19 17:56:05 makyo pulseaudio[24418]: Doing resync
Jul 19 17:56:05 makyo pulseaudio[24418]: Playback after capture (-18503), drop sink 2408
Jul 19 17:56:05 makyo pulseaudio[24418]: No remapping configured, proceeding nonetheless!
FDrico commented 4 years ago

Sorry for the delay.

Using the first of the files you sent, the pulseaudio log is here. The NoiseTorch log is the same i shared before, but i'll link it here for convenience.

Using the second file, the pulseaudio log is here (starting right where previous log finished). The NoiseTorch log, starting where the previous run left, is here.

And finally the last one: pulseaudio log, and noisetorch log.

It doesn't work for me in any of the three cases. In all cases, the pulseaudio echo cancelling module is disabled.

Just to clarify, the solution that seems to have nearly worked for SpartanJ did not work for me.

lawl commented 4 years ago

Last idea for today:

Binary: noisetorch.zip

Source: https://github.com/lawl/NoiseTorch/tree/fixed-latency

FDrico commented 4 years ago

At least for me, still nothing. Torch log, pulseaudio log.

In any case, thank you for your time and open sourcing this!

lawl commented 4 years ago

I spoke to a pulseaudio dev and they said this does seem like a bug in pulse. They asked me if I could test a patch, but as I'm not affected by this, unfortunately I cant.

So if anyone here can compile PulseAudio and apply this patch: https://pastebin.com/zWwynGBS we might get it fixed in PulseAudio.

gist mirror, just in case

SpartanJ commented 4 years ago

@lawl Awesome! Thank you for taking the time to investigate this issue. I'll try to test the proposed patch over the week.

FDrico commented 4 years ago

I had an unrelated issue with my Ubuntu installation, and i had to reinstall everything. Just for the sake of it, i decided to try again with the last build of NoiseTorch. Somehow, it seems to work now. Not entirely OK, because it seems to distort a little bit too much my voice, but that is probably a different issue.

Just wanted to let you know that, in my case, reinstalling Xubuntu magically solved the "silent-NoiseTorch-mic" problem. I saved the pulseaudio config files i had before reinstalling, so in a couple of days i will try to rule out a pulseaudio config problem (basically, replacing the new, default config files with the old ones i had). There may be something to learn.

Thank you for your time and effort!

lawl commented 4 years ago

@FDrico Not necessarily a different issue, similar report here.

mystchonky commented 4 years ago
I spoke to a pulseaudio dev and they said this does seem like a bug in pulse. They asked me if I could test a patch, but as I'm not affected by this, unfortunately I cant.

So if anyone here can compile PulseAudio and apply this patch: https://pastebin.com/zWwynGBS we might get it fixed in PulseAudio

I tried this patch. But the build failed. Error :

modules/module-loopback.c: In function ‘set_sink_input_latency’:
modules/module-loopback.c:1020:32: error: lvalue required as left operand of assignment
 1020 | +            requested_latency = PA_MIN(requested_latency, (latency - u->core->default_fragment_size_msec * PA_USEC_PER_MSEC) / 2);
      |                                ^

Steps I followed.

  1. Cloned https://github.com/pulseaudio/pulseaudio
  2. Followed the "HACKING" section in the README image
lawl commented 3 years ago

@luiq54

Sorry for the late reply, I was fairly busy. It compiles fine for me. How did you apply the patch?

I see a + sign at the beginning of the line, did you edit the file manually?

Here's how I applied this patch. Copied the entire pastebin content to a file named pa.patch in the root of the cloned pulseaudio folder, then typed git apply pa.patch and followed the HACKING instructions you quoted.

If you did edit it manually, you can first reset your manual changes with git checkout src/modules/module-loopback.c and then try applying the patch the way i described.

Edit:

You can verify the patch was applied correctly by typing

git diff src/modules/module-loopback.c

and it should look like this:

image

(The plus signs at the beginning signify that this line was newly inserted, it's not part of the code if you look at the file in an editor)

SpartanJ commented 3 years ago

@lawl I applied the patch to PulseAudio and it seems to work, at least PulseAudio does not crash (it does crash but Cadmus with the usual errors, that part is weird). So at least there seems to be a fix for that. Now regarding the actual noise cancelation, I'm not being able to cancel any noises, if I use the threshold below 95% it will do nothing and if I use it at 95% generates a very noisy/crackled output and ends up being worst than using nothing, and sometimes when I start recording I can't even hear my voice. :( But well, at least now it doesn't crash! Thanks for your help!

lawl commented 3 years ago

@SpartanJ then it may not be fully fixed. i've read from other people getting distorted audio with fixed latency devices. i was hoping this would take care of that too.

Maybe we can get someone else affected to also test it and see if the results match.

mystchonky commented 3 years ago

@lawl Sorry, I did not notice the '+' at the start.(I manually patched the file as I don't know how to do it using programs). After the fix, it did compile and I managed to run it using the steps in the github page. But on that build, my headset mic wasn't detected and hence I can't test the problem at all.

lawl commented 3 years ago

Okay, I have an update as well, I've managed to reproduce the issue by forcing all my devices to be in fixed latency mode, but I haven't gotten to digging deeper into this yet.

winterstein commented 3 years ago

Hello

I have this or a related bug - i.e. the NoiseTorch microphone is silent. I'm running Linux Mint 20 Xfce (Cinnamon desktop), which is based off Ubuntu 20.04.

I'm not expecting a fix - just commenting to let you know. I can provide diagnostics or run a test if you want (though my sys-admin skills are quite basic).

All the best,

lawl commented 3 years ago

@winterstein Is your microphone in orange and does it say "(incompatible?)" in front in noisetorch? Just to make sure this is really the same issue.

winterstein commented 3 years ago

No, it looks fine in the UI (screenshot attached). snapshot_2020-08-18_8709

lawl commented 3 years ago

@winterstein then please open a new issue as it's not the same. (Please attach logs)

lawl commented 3 years ago

Can anyone affected by these try the following?

Append

resample-method = trivial

to ~/.config/pulse/daemon.conf (or create it if it doesn't exist). Then restart pulseaudio with pulseaudio -k. Verify it was set correctly:

$ pulseaudio --dump-conf | grep resample
resample-method = trivial

And see if that changes anything when you use NoiseTorch with that pulseaudio setting?

Unfortunately I still cannot reproduce the issue. I could for a short while, and now I can't again :(

mystchonky commented 3 years ago

@lawl I tested this method but it did not work. The mic is still silent.

lawl commented 3 years ago

noisetorch.zip

Anyone willing to give this build a shot? I was developing some top secret new noisetorch stuff and came across an issue similar to this. So now I'm wondering if the fix ends up being the same.

Also please check if settings the NoiseTorch microphone as fallback like this fixes it, alone, or in conjunction with the new build.

Trillon008 commented 3 years ago

Yes, it works for me. I had a lot of latency, around 2-5 seconds, but the pavucontrol workaround seems to fix it. Edit: pavucontrol fallback is not always working

AtilioA commented 3 years ago

Edit: pavucontrol fallback is not always working

@Trillon008 Try executing systemctl --user restart pulseaudio.service on the command line if that doesn't work.

lawl commented 3 years ago

There is a fix available for this now, but it requires building both PulseAudio and NoiseTorch from source. PulseAudio requires this patch which has been merged upstream a month ago to trickle down into distributions.

Additionally: We have another fix on our side which has been committed and will be is part of the next current NoiseTorch release.

Unless you are willing to build PulseAudio from source, there is currently nothing you can do, but wait.

hbeni commented 3 years ago

Hi, i had a similar issue on debian/bullseye (testing) after some upgrade recently. Before that all went good.

The cause was interference with the suspend-on-idle module.

Commenting the module in the /etc/pulse/default.pa and restarting pulse ($ pulseaudio -k) made it operable again:

### Automatically suspend sinks/sources that become idle for too long
#load-module module-suspend-on-idle

This might be worth to be added to the wiki/Troubleshooting page.

Compizfox commented 2 years ago

I have the same issue here, with a Logitech C270 webcam and PulseAudio (it is shown in orange with the comment "(incompatible?)".

I'm running NoiseTorch 0.11.4, should it be fixed in that version?

I tried @hbeni's suggestion above, but no cigar.

lawl commented 2 years ago

does it cause issues if you try to use it anyways @Compizfox ? if so, you also need a more recent pulseaudio. If not, well, then it's fixed i guess. it's difficult/impossible to properly detect for noisetorch though.

Compizfox commented 2 years ago

@lawl Unfortunately yes, it doesn't work: the NoiseTorch Microphone device is completely silent.

I'm running Arch and I'm on pulseaudio 15.0, which I think is the latest version.

Compizfox commented 2 years ago

I switched from PulseAudio to Pipewire-pulse, and Noisetorch is working now :)