Open adrelanos opened 4 years ago
HelloZeroNet
commented
4 years ago As far I know bitpay/copay like issue could also happen on any Linux repository: at that case someone took over the maintenance of a no longer maintained package, then commited a problematic code.
After a quick look the only packages that are missing from Debian repo: bencode.py, coincurve, merkletools
If someone creates package for these, then it should work
filips123
commented
4 years ago I'm against using distribution package repositories. Packages in them are often outdated, take a lot of time to update, and many of them simply aren't available. They also aren't portable.
How would distribution package repositories prevent their developers from submitting malicious version? The same happened with event-stream. It was not because of NPM issue, but because event-stream used dependency by the malicious author which added malicious code. Same can happen with APT repositories.
purplesyringa
commented
4 years ago I was personally attacked and discredited
these people are fascists criminals
adrelanos
commented
4 years ago Filip Š:
I'm against using distribution package repositories. Packages in them are often outdated, take a lot of time to update, and many of them simply aren't available.
It works for thousands of applications available from packages.debian.org. Among them, popular applcations such as firefox, gimp, libreoffice, and whatnot.
How would distribution package repositories prevent their developers from submitting malicious version?
Debian developers don't allow malware into Debian. That fortunately hasn't happened to Debian yet.
filips123
commented
4 years ago It works for thousands of applications available from
But that applications are not written in Python and are not using Python dependencies.
Debian developers don't allow malware into Debian.
NPM and PyPI also don't allow malware. But malicious maintainer of the package can still push it.
HelloZeroNet
commented
4 years ago Among them, popular applcations such as firefox, gimp, libreoffice, and whatnot.
Actually Firefox includes the dependencies inside the package: Just pdf.js (the part they use to display pdf files) has over 200 dependencies that are not installed using apt repository: https://github.com/mozilla/pdf.js If this is fine, then we could do the same.
adrelanos
commented
4 years ago Filip Š:
It works for thousands of applications available from
But that applications are not written in Python and are not using Python dependencies.
Many python based applications in Debian are using python dependencies also variable from Debian.
Here is a list of popular python based applications:
https://en.wikipedia.org/wiki/List_of_Python_software#Applications
Among them deludge (bittorrent client) and pip just to pick two examples.
Some of them are available from Debian package sources.
Debian developers don't allow malware into Debian.
NPM and PyPI also don't allow malware. But malicious maintainer of the package can still push it.
This has never happened in Debian yet which has thousands of packages.
leycec
commented
4 years ago Linux ≠ Debian. While popular, Debian-based Linux distributions occupy only a small fraction of the Linux space. Devolving ZeroNet from its current distro-agnostic installation instructions to distro-specific installation instructions makes little to no sense. Mostly, no sense.
The current distro-agnostic installation instructions are fine. What's not fine is the sluggish pace of package publication by Debian developers – but that's entirely on them. ZeroNet isn't to blame here. When even Gentoo and Arch Linux provide up-to-date packages for ZeroNet, you know that something's wrong with your rolling release model... or lack thereof.
It is also a third party package manager.
pip is the official Python package manager bundled with Python itself. It doesn't get more first-party than that. With respect to Python packaging, it's system-wide package managers like Debian's apt and CentOS' yum that are third-party – not pip.
Relative perspective: it matters.
Could you please use dependencies available from distribution package repositories such as packages.debian.org?
No, because distro-specific package repositories are (wait for it)... distro-specific. ZeroNet would only be installable under Debian-based Linux distributions – a significant regression from ZeroNet's current applicability to all Linux distributions. Does that actually sound like a good idea to anyone in attendance who is not @adrelanos?
If we really want to walk down this road in a sane way, we might consider publishing a distro-agnostic AppImage for ZeroNet. Unlike distro-specific packages, AppImages portably run on all Linux distributions. They also bundle all requisite dependencies in a safe, secure, robust, reproducible manner isolated from any package manager – pip and apt included.
If portability, security, and ease of installation are your concerns, AppImage is your answer. Debian is a non-answer to a question that no one asked.
This is also a blocker for ever getting ZeroNet accepted into official packages.debian.org.
No. The blockers here are Debian developers – not ZeroNet. Again, Gentoo and Arch Linux both publish ZeroNet packages. If the little guys with effectively no funding can do it, the big guys with absurd piles of multinational corporate venture capital absolutely can, too.
This is entirely on Debian. Debian's well-known internal torpor is entirely a Debian issue.
This issue is a non-issue and should probably be closed.
HulaHoop0
commented
4 years ago pip still has no idea about code signing in this day and age. It was conisdered a big leap forward when they implemented fetching over HTTPS and reject plaintext connections which is really sad considering the poor security assumptions that even this would have:
https://github.com/pypa/pip/issues/1167
Here are two different real world cases of malware snuck into PyPi being executed becuase of pip's non existent security:
The problem is ultimately the result of developers and administrators who fail to inspect packages thoroughly. Adding to the insecurity, the widely used pip package management system (pictured above), which most Python developers rely on, doesn't require cryptographic signature before executing code when a package is installed.
https://www.zdnet.com/article/twelve-malicious-python-libraries-found-and-removed-from-pypi/
Please consider an alternative packaging/installation model that doesn't rely on pip. Either using stuff in Debian backports or just making an AppImage (which includes code that is audited by you beforehand) - Electrum does the latter since their release cycle is far too rapid for it to sync with Debian's.
We really love Zeronet and want to include it in our privacy distro by default, but unless we can have a secure installation mechanism, we just can't do it.
PS. I am only interested in Zeronet dev replies. If you are an enthusiast please express yourself in another thread.
filips123
commented
4 years ago APT by default uses HTTP and there were already vulnerabilities to allow man-in-the-middle attacks (here and here.
Cryptographically signing packages does not prevent their authors to publish malicious packages, which happened in the case of npm and pip.
The only way to prevent this would me to manually review and check every single release of every single package but I don't think this happens on APT either.
Also, distribution package managers are often outdated and take a lot of time to update. Many APT packages do not have their latest version published, even if it exists on other package repositories. This means that we would need to wait a long time to get new features. And sometimes, if you use a bit old distribution, they are just not updated.
And using distribution package managers would require to set things for each of package managers. There are many Linux distributions so this would require a lot of work. Pip, on the other hand, is platform-independent and you don't need to configure packages for each platform separately. And you will still need to use yet another thing for Windows and macOS.
So, distro-specific managers are not a solution. The proper solution is to use distro-agnostic managers (AppImage, Flatpack, Snap) where they are available and continue using Pip where they are not.
HulaHoop0
commented
4 years ago Cryptographicly signing packages does not prevent their authors to publich malicious packages, which happened in case of npm and pip.
Right. What we want to do is ensure no one else, but the package author has modified its code.
The transport layer security is a nice plus in case the package manger itself has a RCE bug, but this is besides the point for code integrity.
I guess we are on the same page about AppImage which is a good sign. We know how to deal with those and they suit us.
emdee-net
commented
2 years ago Malicious code has been found on pypi https://www.zdnet.com/article/twelve-malicious-python-libraries-found-and-removed-from-pypi/ but Debian and Ubuntu have been running insecure apt for years https://www.guardicore.com/labs/a-vulnerability-in-debians-apt-allows-for-easy-lateral-movement-in-data-centers-2/ The only solution if you want a secure system is to run Gentoo which has a package manager that supports signing the original source with the originator's key and builds from source.
I can't believe in the day of state MITM that ZeroNet ships unsigned releases or binaries - they all should be signed with an originator's key. All the ZeroNet forks @TwinLizzie @canewsin @geekless and releases @HelloZeroNet should be signed and the should make the effort to be supported on Gentoo. https://github.com/leycec/raiagent/issues/101
adrelanos
commented
2 years ago but Debian and Ubuntu have been running insecure apt for years https://www.guardicore.com/labs/a-vulnerability-in-debians-apt-allows-for-easy-lateral-movement-in-data-centers-2/
https://security-tracker.debian.org/tracker/CVE-2019-3462 as correctly linked in above article was a vulnerability that was promptly fixed. This wasn't and isn't an architectural security issue which is unresolved for years.
The only solution if you want a secure system is to run Gentoo which has a package manager that supports signing the original source with the originator's key and builds from source.
emdee-net
commented
2 years ago The bug in CVE-2019-3462 was introduced in 2009, so taking 10 years to fix it is "promptly fixed" on Debian terms? Last time I looked at apt in Debian wasn't even using https:// for reasons they made famous. And I'm certain that Debian and Devuan do not use secure curl with protection against SSL protocol downgrades. or redirects, to download debs, even though the susceptibility of SSL to these attacks has been well know for a decade or more.
Debian is famous in professional security circles for having refused to address these weaknesses, and it is assumed at this point to be intentional.
emdee-net
commented
2 years ago Please use words and sentences, as just posting links that are 5+ years out of date with regard to the point being made, is not helpful. To recall, the discussion was about a package manager that supports signing the original source with the originator's key.
adrelanos
commented
2 years ago The bug in CVE-2019-3462 was introduced in 2009, so taking 10 years to fix it is "promptly fixed" on Debian terms?
It doesn't matter when the issue was introduced. You haven't shown evidence that it was introduced 10 years ago either. If a vulnerability stays undetected for years, then no bad intention can be deducted. Once CVE-2019-3462 was reported, CVE-2019-3462 was promptly fixed.
Last time I looked at apt in Debian wasn't even using https:// for reasons they made famous.
Long time since you didn't look then.
And I'm certain that Debian and Devuan do not use secure curl with protection against SSL protocol downgrades. or redirects, to download debs, even though the susceptibility of SSL to these attacks has been well know for a decade or more.
Link to bug report(s) required.
Debian is famous in professional security circles for having refused to address these weaknesses, and it is assumed at this point to be intentional.
Citation required.
Please use words and sentences, as just posting links that are 5+ years out of date with regard to the point being made, is not helpful.
By how that bug reports went, I don't expect anything has changed in 5+ years. An interested reader that cares about the gentoo package manager security is free to check if it's resolved meanwhile and/or to rehash the discussion.
To recall, the discussion was about a package manager that supports signing the original source with the originator's key.
Since I opened the ticket...
@adrelanos
This is also a blocker for pre-installation of ZeroNet by default in Whonix.
Since Whonix is based on Debian... Then went off-topic by making it about Debian vs Gentoo package manager security. So for this issue to be resolved... Well, see original post.
adrelanos
commented
2 years ago So if you'r requesting packaging for Gentoo, upload to Gentoo... I suggest opening your own ticket.
Though, not sure how much sense that makes due to https://github.com/HelloZeroNet/ZeroNet/issues/2326. But any of the forks my have a higher chance to implement this.
emdee-net
commented
2 years ago The decision on a Gentoo overlay was to drop ZeroNet entirely https://github.com/leycec/raiagent/commit/e3cbb3bc276eaca8f09c2b45e25253d49f0de912 https://github.com/leycec/raiagent/issues/101
My suggestion for you is to change the title to "please do not depend on python-pip3 or apt - security risk - use dependencies available from originator signed distributions" in case you wanted meaningful security for ZeroNet. @filips123 what Gentoo is doing is integrating the originator's keys into its package management, and then distributing the orginator's signed source plus a Gentoo signed recipe to build from the source.
Or for WhoNixxed distributions that are maintained using unsigned apt like KickSecure; it always struck me as curious that anything aiming to be "watertight" would use unsigned apt. There was a port of most of KickSecure plus to Gentoo, but it was abandoned we they realized there is no way of doing updates securely.
( The links to the "required" information you overlooked are in the links @filips123 posted.)
adrelanos
commented
2 years ago A distribution with originator signed source code sounds nice. It's certainly a nice item to have for a (security) features checklist for a Linux distribution comparison. There's a lot other checklist items. Added here: https://www.whonix.org/wiki/Dev/Operating_System#Criteria_for_Choosing_a_Base_Distribution
unsigned apt
APT metadata isn't unsigned. It's signed. Can be easily seen here for example:
It even has a valid-until field which prevents indefinite freeze attacks and rollback attacks.
https://wiki.debian.org/DebianRepository/Format
Check out The Update Framework (TUF) threat model: https://theupdateframework.io/security/
It's really excellent.
Years ago the security researchers from TUF found many security issues with package managers in many distributions (Debian among others) which have been fixed. Among the improvements was the valid-until field.
https://www2.cs.arizona.edu/stork/packagemanagersecurity/attacks-on-package-managers.html
As for Gentoo vs TUF, see above links in my previous posts.
On top of the signed APT metadata, Debian offers downloads TLS and even onion. Onion provides an end-to-end (Tor to Tor) transport encryption alternative to TLS which is independent from any TLS certificate authorities.
I guess your critical point here is basically that Debian isn't a source-based distribution. The question and answer here seems very much on point: https://unix.stackexchange.com/questions/285635/how-is-the-authenticity-of-debian-packages-guaranteed
Source packages for Debian that are signed by Debian Maintainers and uploaded to Debian are verified by buildd (build server) which then builds binaries which are then distributed with signed APT metadata. There's indeed no direct chain from the Debian Maintainer digital signature to the APT running on the user's computer verifying the package was signed by the maintainer's key. That's I guess what you mean by originator signed.
Debian indeed isn't using the source-based distribution model. This affects all binary based distributions.
The solution for that reproducible builds and it is being worked on.
https://reproducible-builds.org/
This feature request to avoid python-pip3 is useful for both, Debian and Gentoo users.
There was a port of most of KickSecure plus to Gentoo,
Where?
we they realized there is no way of doing updates securely.
We? They? Who?
There was a port of most of KickSecure plus to Gentoo, but it was abandoned we they realized there is no way of doing updates securely.
All of Kicksecure source code is signed for years. All git commits are signed as well as all git tags are also signed. If there's something missing, please report a bug since I'd be interested to know and most likely would accommodate the request. APT isn't needed at all to acquire signed Kicksecure source code.
emdee-net
commented
2 years ago That's I guess what you mean by originator signed. No I don't mean maintainer signed; I mean by the originator of the package.
As for Gentoo vs TUF, see above links in my previous posts.
Your links in the previous posts are years out of date, as is the trivial discussion of Gentoo in https://www.whonix.org/wiki/Dev/Operating_System. In the last 2 years, Gentoo has introduced originator signed source (and in some cases binary) packaging which is what I was referring to. It's a huge undertaking to have the package manager manage the originator keys.
I think you can get part of the way there in Debian with signed-by= in sources.list.d, and it surprised me that KickSecure chose not to do this: https://forums.whonix.org/t/apt-seccomp-bpf-sandboxing/7702
adrelanos
commented
2 years ago emdee-net:
That's I guess what you mean by originator signed. No I don't mean maintainer signed; I mean by the originator of the package.
Any link for that?
I am reading https://wiki.gentoo.org/wiki/Project:Portage/Repository_verification#How_does_it_work.3F where it's written "Portage verifies the latest commit, which will be signed automatically (not by a developer) ..."
As for Gentoo vs TUF, see above links in my previous posts.
Your links in the previous posts are years out of date,
These things don't magically change. I've shown evidence that Gentoo does not pass TUF. Kinda proving a negative.
Now you're claiming Gentoo has fixed it. Therefore the burden of proof is on you.
But let's check again. The issue described in https://github.com/Kicksecure/Gentoo-Port/issues/19 / https://bugs.gentoo.org/show_bug.cgi?id=539954 is still valid.
I went to the stage 3 folder.
https://gentoo.osuosl.org/releases/amd64/autobuilds/current-stage3-amd64-systemd/
There are signatures. But no valid-until field.
Since https://bugs.gentoo.org/show_bug.cgi?id=539954 is 7 years old, if you like to see the security of Gentoo improved, you could ask if the ticket could be re-opened and re-considered or if it's already resolved meanwhile.
as is the trivial discussion of Gentoo in https://www.whonix.org/wiki/Dev/Operating_System. In the last 2 years, Gentoo has introduced originator signed source (and in some cases binary) packaging which is what I was referring to.
It's a huge undertaking to have the package manager manage the originator keys.
That's for sure. However, that's different from valid-until field / TUF.
I think you can get part of the way there in Debian with signed-by= in sources.list.d,
signed-by is different. signed-by can pin a deb repository line to a specific signing key.
https://forums.whonix.org/t/apt-repository-signing-keys-per-apt-sources-list-signed-by/12302
and it surprised me that KickSecure chose not to do this: https://forums.whonix.org/t/apt-seccomp-bpf-sandboxing/7702
Kicksecure is doing that as per reply in that forum thread. Here's the related git commit:
https://github.com/Kicksecure/security-misc/commit/3558a9949fe9924d027b267152125b33e25085c8
emdee-net
commented
2 years ago I was surprised to see no signed-by: in /etc/apt/sources.list.d/whonix.list of KickSecure.
You're reading Portage/Repository_verification which is assumed, and not what I'm talking about: I'm talking about originator signing.
Any link for that?
Look in /usr/portage/sec-keys/ of any current gentoo.
Or run "eix sec-keys/*" on any Gentoo. They are originator keys.
Maybe you can add a sentence about originator signed source (and in some cases binary) packaging in the Gentoo section of https://www.whonix.org/wiki/Dev/Operating_System . That would double you coverage of Gentoo there. (You also might make the point that you can also can avoid systemd too which is a real bonus, or for some, a requirement.)
But in the case of Zeronet you might as well close this issue because there is no Linux distribution on the planet that contains the obsolete/abandoned/vulnerable prerequisite requirements of Zeronet (e.g. merkletools). Look at what a Gentoo overlay had to do to build a Zeronet https://github.com/leycec/raiagent/commit/e3cbb3bc276eaca8f09c2b45e25253d49f0de912#diff-86f2ac264ef17e6d26b9986736b59a61f52bf22d7cae7b73923e81f3c54cf045 Cecil eventually decided to drop ZeroNet completely.
A more helpful issue is to request that abandoned or obsolete requirements in ZeroNet be replaced or vendored. See Cecil's suggestions for doing that in https://github.com/leycec/raiagent/issues/101 But in the case of merkletools you would have to replace it as it is known to be vulnerable to attacking-merkle-trees-with-a-second-preimage-attack - see https://github.com/Tierion/pymerkletools/issues/13#issuecomment-471181413 They never did update the README to disclose this.
adrelanos
commented
2 years ago emdee-net:
I was surprised to see no signed-by: in /etc/apt/sources.list.d/whonix.list of KickSecure.
Kicksecure uses signed-by.
Kicksecure doesn't use whonix.list. /etc/apt/sources.list.d/whonix.list is deprecated. Migrated. No longer in use.
In Kicksecure sources list that is file /etc/apt/sources.list.d/derivative.list signed-by is being used:
To check:
cat /etc/apt/sources.list.d/derivative.list
The deb line will include:
deb [signed-by=/usr/share/keyrings/derivative.asc] ...
Link to source code where this is implemented:
https://github.com/Kicksecure/repository-dist/blob/master/usr/bin/repository-dist#L361
emdee-net
commented
2 years ago There's no /etc/apt/sources.list.d/derivative.list in my KickSecure and no signed-by in any of the sources.list.d files.
Not a problem - there is no distribution I know of that ships merkletools (and coincurve and pyelliptic and others?) so to support for it with dependencies available from distribution package repositories is not known to be possible.
adrelanos
commented
2 years ago emdee-net:
There's no /etc/apt/sources.list.d/derivative.list in my KickSecure and no signed-by in any of the sources.list.d files.
Probably an outdated version.
If already updated, any other files related to Whonix / Kicksecure in /etc/apt/sources.list.d/ folder are safe to delete.
Then just run repository-dist to have /etc/apt/sources.list.d/derivative.list created.
https://www.kicksecure.com/wiki/APT_Repository
If it created the old version, then it wasn't updated yet.
New downloadable releases use /etc/apt/sources.list.d/derivative.list and signed-by by default.
Instructions on Linux currently require to run
sudo python3 -m pip install -r requirements.txt. Please consider to drop this if possible.python pip is a security risk since it does not verify software signatures on packages it downloads. It is also a third party package manager.
Previously a bitcoin wallet was hacked that depended on a third party package manager that didn't verify software signatures.
https://bitpay.com/blog/npm-package-vulnerability-copay/
Could you please use dependencies available from distribution package repositories such as packages.debian.org?
This is also a blocker for ever getting ZeroNet accepted into official packages.debian.org.
This is also a blocker for pre-installation of ZeroNet by default in Whonix.
Related packaging issues:
https://github.com/HelloZeroNet/ZeroNet/issues/382 https://github.com/HelloZeroNet/ZeroNet/issues/241 https://github.com/HelloZeroNet/ZeroNet/issues/1786