Conduct an Audit

[Dadadah]

I assume you have regained control of your security keys.

I think this project is large enough to fund an open source audit to alleviate any concerns about security compromises.

These things can happen and there are things the repository owners can do to address it.

Initial research shows an audit taking about US$100/1000 lines of code.

Perhaps consider starting a fundraiser for it @lawl ?

[lawl]

No i havent. I believe its unrelated to noisetorch, but better safe than sorry. I am dead right now and dont have the energy to rebuild my computing devices from scratch.

Everythings still compromised, but hopefully enouch people know now.

[OldiLo]

Should I remove NoiseTorch and ayy project?

[CheaterTim]

@OldiLo

Should I remove NoiseTorch and ayy project?

I would remove both for now and watch out for any updates regarding the situation

[OldiLo]

I assume you have regained control of your security keys.

I think this project is large enough to fund an open source audit to alleviate any concerns about security compromises.

These things can happen and there are things the repository owners can do to address it.

Initial research shows an audit taking about US$100/1000 lines of code.

Perhaps consider starting a fundraiser for it @lawl ?

Maybe the community should do it

[lawl]

No, dont give money. Patreon is scheduled for deletion cant do it instantly.

[OldiLo]

No, dont give money. Patreon is scheduled for deletion cant do it instantly.

No, the community can do an audit

[YannikSc]

Can't you just revert from a fork? How long ago was the breach?

EDIT

With that I don't want to say, that you should fix it, its more like a thought, if someone wants to revive the project, and I'm rather asks for an estimation if this could be plausible way to get some sort of fix together.

Would be hard though if it was a couple months or even years ago.

[comfmai]

So I did a little digging: This is the original binary version, for some reason it was compressed with UPX, which should have raised my suspicion (but I can see the previous versions are also UPXed, @lawl is this on purpose?)

I put it through VirusTotal, it came out clean https://www.virustotal.com/gui/file/726a3dd0b72d2de56b55d2920fd1ea64c5017983eb308f3e3bfba2dbe867ea94/detection

I also the unpacked version through VirusTotal, it came out clean as well https://www.virustotal.com/gui/file/23d531efde629161f64294a6f13a7c9ad7a6df06435d2fcc9f6604848959a2ac/detection

But, when firing WireShark I can see this: TLS connection being made to 82.118.227.155 , which seems to be already reported: https://www.abuseipdb.com/check/82.118.227.155

This looks like a command & control server...

[lawl]

Yes upx was on purpose, but ypu shouldnt trust me when i say that because i already said my sytem(s?) were compromised.

[lawl]

82.118.227.155 have you checked this isnt the update server? Does noisetorch.epicgamer.org resolve there?

[Xunjin]

Yes upx was on purpose, but ypu shouldnt trust me when i say that because i already said my sytem(s?) were compromised.

I see well I've looked for this IP and I found not a single reference using wireshark as @comfmai found out. However, I'm not an expert on this.

@lawl I know you are tired and feeling really depressed about it, however if there is a way for us to help you, maybe conduct some kind of audit. This project is way too good to die :cry:

Edit: tho I'm using the arch repo https://aur.archlinux.org/packages/noisetorch, and it's indeed old.

[lawl]

Ok ive pointed the readme to this thread. If we can get enough people to check the code, maybe we can work from there.

[papaya2k]

Name: noisetorch.epicgamer.org Address: 82.118.227.155 Name: noisetorch.epicgamer.org Address: 2a01:8740:1:fe3f:dc78:593f:d16c:1

[reynoldsme]

So, noisetorch.epicgamer.org does resolve to 82.118.227.155 but it also resolves to the ipv6 address 2a01:8740:1:fe3f:dc78:593f:d16c:1 so @Xunjin you wouldn't necessarily see any references in wireshark to the 82.* address if testing from an ipv6 enabled network and machine.

[papaya2k]

So, noisetorch.epicgamer.org does resolve to 82.118.227.155 but it also resolves to the ipv6 address 2a01:8740:1:fe3f:dc78:593f:d16c:1 so @Xunjin you wouldn't necessarily see any references in wireshark to the 82.* address if testing from an ipv6 enabled network and machine.

Don't know where others are from, but a lot of American ISPs including my own don't provide IPv6 access to the Internet.

[comfmai]

Yes upx was on purpose, but ypu shouldnt trust me when i say that because i already said my sytem(s?) were compromised

I'm sorry my bad @lawl I should have double checked, I can see it here: https://github.com/lawl/NoiseTorch/blob/2663bcc8d6aea1e15d0e52e47ce55f60283ef53b/Makefile#L22

82.118.227.155 have you checked this isnt the update server? Does noisetorch.epicgamer.org resolve there?

My double apology @lawl, Yes you are correct noisetorch.epicgamer.org is resolved to this address. I was confused because when I checked from a different machine I got a different IP, so I just assumed it was not the same one. Bottom line, this seems like a totally valid connection. @Xunjin this was the confusing part for me, if you try to resolve you would probably get a different IP (I'm assuming CDN in the middle)

I run WireShark and waited for several minutes, I could not detect any suspicious communication. So I think this is a good sign 🤞

@lawl if there is anything I can help with, please feel free to suggest. NoiseTorch is awesome, you have earned my support and appreciation regardless of this unpleasant security breach.

EDIT: You can see here noisetorch.epicgamer.org is resolved to a different IP, anyhow I think this was false alarm on my end. apologies everyone 😓 https://www.virustotal.com/gui/file/23d531efde629161f64294a6f13a7c9ad7a6df06435d2fcc9f6604848959a2ac/behavior

[Xunjin]

Thank you all for the info and feedbacks. Anyone with experience in audition? Would love tips on how to do it properly. :sweat_smile:

[reynoldsme]

But, when firing WireShark I can see this: TLS connection being made to 82.118.227.155 , which seems to be already reported: https://www.abuseipdb.com/check/82.118.227.155

This looks like a command & control server...

I have no idea how reliable this AbuseIPDB is, but if accurate, it does seem to suggest that the host running noisetorch.epicgamer.org may be compromised (this assumes that 82.118.227.155 maps directly to the host and no NAT/proxy is involved). The requests in the reports look like a wordpress spam bot. That seems odd to be coming from a CDN address.

I see Caddy is serving requests which suggests this is a cloud vm instead of something like a shared web hosting platform?

[lawl]

Its a tiny vm with natted ipv4.

[Xunjin]

@lawl you pointed out in another issue these two commits https://github.com/lawl/NoiseTorch/commit/8c34658b64f1efeab501bef57d2bfa9579fe34e2 and https://github.com/lawl/NoiseTorch/commit/38787e4195f2a34d7ec4421caf17cb99bc31fa2b might be hiding something, I will skirmish through them. Any more commits that you believe might be a problem?

[10maurycy10]

Sorry if this is hijacking this thread, but what was the exact scope of the compromize?

[lawl]

@Xunjin litterally all of them. These are just where i'd hide as an attacker, probably. As always absence of evidence != evidence of absence.

[Dadadah]

Rad, I'm glad this took off. Before I start taking a look at this, I wanna ask why the vendor folder is checked in to git. Usually with go mod you put the vendor folder in the .gitignore.

I'll do my part and try to comb through diffs and the code in general. While I'm not a security engineer, I do work in Go professionally.

[Dadadah]

My guess is that the biggest possible vector for attack is your update server. Perhaps we can disable that feature until we get everything looked over.

[Technetium1]

@10maurycy10 assume the worst until proven otherwise.

[Dadadah]

So just catching back up to this, I did a pretty in depth review of most of the stuff in this repo. Here's what I covered and as far as I can tell is working as expected:

Noisetorch requesting root for modifying capabilities Noisetorch updating from a remote server (ASSUMING REMOTE SERVER IS TRUSTED) Noisetorch changing the RLimit of PulseAudio Noisetorch module loading and unloading Noisetorch CLI The last year of commit history for this repository

Some things I haven't checked:

Anything UI related The PulseAudio library in use, which can be found at lawl/pulseaudio Untar code Any C code

I am confident that when it comes to root access used by Noisetorch, the source is clean. However, do keep in mind that I am not a security engineer.

[Dadadah]

While not a fix, #257 should make an audit much easier.

[MagicRB]

@lawl I've just learned of this from a nixpkgs PR and I'm so sorry. Security keys are one of the most private things a developer has. I'm not a golang developer so idk if I reasonably help. But good luck to everyone, especially you.

PS. I know this is hard on you, but would you be able to talk about precisely what happened and how you lost your keys? Your experience could help others to not loose their keys. If it's too soon, it's fine, take your time.

[Xunjin]

@lawl I believe @Dadadah has a good point about the remote server, might help in reducing the surface of attack (however I'm not a security engineer, so take my point with a pinch of salt)

[MagicRB]

Ill just list what i have and know, so that if i can help, you people can tell me. I know Rust, Haskell, Nix, system admin stuff and I have 3 free VPSs rn in Oracle Cloud. IDK if any of that will be of use.

[principis]

I don't quite understand the need to check the entire source. Wouldn't it be good enough to check the latest commits? The key must have been compromised for a long time to become a major problem. If so, I suspect someone would have noticed it earlier?

[lawl]

yeah you can remove the remote server that's being pinged by people's builds by... shipping a new binary. otherwise, yeah the whole update code has to be ripped out at some point, but that's easy and not a problem.

[lawl]

Wouldn't it be good enough to check the latest commits?

i'm now assuming it must likely have occured sometime in the second half 2021.

[lawl]

since someone asked, my assumption right now is that it was most likely just someone looking for bitcoin wallets. still not sure how they got in exactly, and now sure if we'll ever know, but as i already said, probably nothing related to noisetorch. and no bitcoin either.

still better safe than sorry.

[fuomag9]

82.118.227.155 is either a c&c server (lots of old and vulnerable services) because it has a lot of open ports with mainly ssh servers (plus some other stuff like NTP and two web servers) or a compromised host

Intestingly enough, wq.steliosm.net is a domain that resolved to that ip on the 15th of may.

noisetorch.epicgamer.org has ssl certs [dating 2020] (https://search.censys.io/certificates/6358eed412e3ea1d13f0e3be08d943b2c2d56cae10ed3b88fe71c771a458528a), is this intended or you have been compromised from before? @lawl

[lawl]

@fuomag9 as i wrote somewhere else, this is an IPv4 NAT server with like 20 ports per VM, this is entirely expected.

[ZyanKLee]

Not that this is any proof for security, but I just let snyk scan my local clone of NoiseTorch and it came back with 0 issues on the dependencies. Snyk does generate a pretty overview of the dependencies, though. Perhaps someone with a developers background can check the dependency list for sanity: https://paste.ubuntu.com/p/n69FMfmpGK/

As a next step I ran gosec, which did indeed find some issues - but on a first glance by me (a layman) none of those seem to be malicious in nature. I provide this as a paste for others to check it out, too. https://paste.ubuntu.com/p/HhxGmSSvvC/

[principis]

I went over a bunch of commits and looked at the code. I don't think it's tampered with. I will keep my builds for Fedora up, and keep using this awesome piece of software.

@lawl I'm sorry to hear that this happened to you, and I hope you'll recover soon!

[ghost]

Hello !

I went over the whole of the c/ directory and checked for any things that may be malicious and did NOT find anything to be suspicious. I spent about 2 hours looking over it so i'm fairly confident it is all okay. I am by no means a security researcher but program in C professionally and in my spare time.

@lawl I am really sorry to hear the news about all of this happening, just remember that we all have your back and owe you the world for the amazing piece of software you have written, we thank you greatly and hope you feel better soon.

[fuomag9]

Not that this is any proof for security, but I just let snyk scan my local clone of NoiseTorch and it came back with 0 issues on the dependencies. Snyk does generate a pretty overview of the dependencies, though. Perhaps someone with a developers background can check the dependency list for sanity: https://paste.ubuntu.com/p/n69FMfmpGK/

As a next step I ran gosec, which did indeed find some issues - but on a first glance by me (a layman) none of those seem to be malicious in nature. I provide this as a paste for others to check it out, too. https://paste.ubuntu.com/p/HhxGmSSvvC/

I had not much time to check those, but that is indeed dangerous code from a security standpoint

An example, launching a process with os.environ passed is not a great idea per se:

syscall.Exec(self, []string{""}, os.Environ())

Nonetheless, those need to be put in context of when they were added and why so we can exclude them

[AXDOOMER]

@lawl Can you be more specific about how you found out you were compromised? How did it happen? Did you see any signs of an attacker? Do you have log files? Do you still have the latest binary from your server? (the one that may have been compromised)

I have a few old releases of noise torch. I can diff the binaries to find out if any malicious code was added. With the right tools, it's easy.

Unless you're more specific about how it happened and how you found out, we can't know where to focus our efforts. Maybe this repository wasn't even compromised and only the update server got malicious payloads uploaded. I know this is depressing, but you must be transparent so we can make sure no further damage is caused. You'll feel relieved once we'll be able to pinpoint exactly a piece of code that's malicious, a version of the binary that's compromised or even find out that the project was not affected at all.

---

What I believe can be done for now, assuming it wasn't only the update server that got compromised (anybody can participate):

1- For every fork of the project, check if the commit history of this repo matches that of the fork. Also check if the content of each commit matches.

2- For every dependency in the vendor folder, check if the code matches with the same version of the dependency in its original repo. The git repo of every dependency found in vendor/ can be found in go.mod.

Document the versions, forks, dependencies that you've tested and your findings, if any.

[lawl]

Found a very suspicious process in htop. Paniced. Later straced it and it was looking for wallet.dat. The OS itself was fairly fresh (q3 ish?).

Sorry i dont think i have any logs or anything that isnt deleted. As you may see from my history, i paniced fairly hard.

[ntimo]

Did your saw the comment by THOR on the virus total community tab? https://www.virustotal.com/gui/file/726a3dd0b72d2de56b55d2920fd1ea64c5017983eb308f3e3bfba2dbe867ea94/community

[AXDOOMER]

Did your saw the comment by THOR on the virus total community tab? https://www.virustotal.com/gui/file/726a3dd0b72d2de56b55d2920fd1ea64c5017983eb308f3e3bfba2dbe867ea94/community

Too generic of a YARA rule. It will basically detect anything packed with UPX. https://github.com/Neo23x0/signature-base/blob/master/yara/gen_elf_file_anomalies.yar

[ntimo]

I also scanned a older version of noisetorch from my previus ubuntu machine using virustotal and there mcaffee detects some generic Artemis!Trojan not sure how relevant this is. https://www.virustotal.com/gui/file/a4ac5e60d0f3ac1a673fdcd997dbb22f6e2160d8f140afd2d435978a8fb1680a/detection https://service.mcafee.com/?locale=en-US&articleId=TS100414&page=shell&shell=article-view

[AXDOOMER]

I also scanned a older version of noisetorch from my previus ubuntu machine using virustotal and there mcaffee detects some generic Artemis!Trojan not sure how relevant this is. link1 link2

Not a big deal. This file was submitted to VT on 2021-10-01 for the first time and it was only detected by one AV, which means it's probably a false-positive.

[ZyanKLee]

@lawl perhaps pin this topic, so it can be easily found.

[protectroot-com]

Please post a link in your readme to https://www.buymeacoffee.com/ so we can help make some lemon-aid out of these lemons :)

[protectroot-com]

https://www.sandflysecurity.com/

[ntimo]

@protectroot-com did you used https://www.sandflysecurity.com/ to check your devices? Did it found something?