animetosho / Nyuu

Flexible usenet binary posting tool
215 stars 30 forks source link

Same Yenc Header Name with {rand(N)} #132

Closed Antidote2151 closed 2 weeks ago

Antidote2151 commented 4 weeks ago

Hello,

I noticed that if you enter "{rand(64)}" in the --yenc-name attribute for a command, the name is random, but the name is the same for every article and not different for every article.

Is that how it should be? If so, is it possible to include the option that the name is random for each article?

Regards

animetosho commented 3 weeks ago

Yes, that is correct. The name is generated for the file and applied to all articles of that file.

I have no intention of changing this behaviour unless you can convince me otherwise.
You might be able to find some Nyuu forks that implement what you want however.

Antidote2151 commented 3 weeks ago

Well, it would be interesting if the articles were really completely obfuscated, which would save us the complete rar with password protection, for example. This way we can also follow your post where you have already described this nicely and it would probably be an incentive for more people not to do this anymore, because it is then simply really very difficult to get to the files without the NZB.

animetosho commented 3 weeks ago

it is then simply really very difficult to get to the files without the NZB.

  1. Why does this matter?
  2. Do you have any evidence/proof for or against this, or is this purely wild speculation?
  3. Was it already very difficult to get to the files without any of this?
Antidote2151 commented 3 weeks ago

Why does this matter? Inspired by your blog post 'Stop RAR Uploads' I wanted to optimize my workflow: https://github.com/animetosho/Nyuu/wiki/Stop-RAR-Uploads Main drivers would be 'faster uploads/downloads', 'no password/encryption surprises', and potential streaming. And, of course, it would prevent others from de-obfuscate the articles (see below).

Do you have any evidence/proof for or against this, or is this purely wild speculation? At least one Usenet provider can de-obfuscate and combine articles to a NZB file that do not have password-protected RAR files and offers them on his own search engine. This is done automatically. If the 'subject', 'from', and 'name' for each article would be random, the Usenet provider will not be able to do that anymore. I am happy to share proofs with you in a more private environment. Matrix would be nice or any other private channel of your choice.

Was it already very difficult to get to the files without any of this? I do not know how difficult it was but it can be done (see above answer).

Tensai75 commented 3 weeks ago

Antidote2151 is right. The Easynews index does analyse the yenc header in the body and articles with the same name in the yenc header are indeed indexed. So the Easynews search will provide results and an NZB file which will enable you to download the whole file if you know what string to search for. You can argue that no one will know the name string, but a 100% obfuscation should make it impossible for the indexer to connect the individual articles together at all. And to achieve this, all 3 key information strings, subject, from and the name string in the yenc header must be randomized for every article in order to cut any connection between the articles.

Regards, Tensai

animetosho commented 3 weeks ago

Thanks for the response!

If the 'subject', 'from', and 'name' for each article would be random, the Usenet provider will not be able to do that anymore

If the Usenet provider is determined enough, why do you think they will not just simply resort to other techniques (e.g. using other fields of the yEnc header)?

I do not know how difficult it was but it can be done

You haven't addressed how de-obfuscation occurs with a random filename. Joining articles together doesn't clarify what the random filename signifies.

You can argue that no one will know the name string, but a 100% obfuscation should make it impossible for the indexer to connect the individual articles together at all.

You didn't address your premise though. If no one knows the name string, why does it matter if the indexer connected the individual articles?

all 3 key information strings, subject, from and the name string in the yenc header must be randomized

Why give everything a random unique identifier if your aim is obfuscation? Naming everything 'file' would surely be less identifying, no?

Tensai75 commented 3 weeks ago

If no one knows the name string, why does it matter if the indexer connected the individual articles?

Because there is a risk that someone who does not have the original NZB file could, in the worst case, download the file, e.g. by simply scanning through the (newly) indexed uploads in the Easynews search, even though they do not know what they are downloading.

why do you think they will not just simply resort to other techniques (e.g. using other fields of the yEnc header)?

It is certainly possible that indexer can make use of the other yenc header fileds in order to index the uploads but currently it doesn't seem to be the case. This however can change and we will then need to adapt the obfuscation method probably once again.

Naming everything 'file' would surely be less identifying, no?

The obfuscation should prevent the upload from appearing in the indexer at all. Naming everything the same would contradict this, although the resulting NZB file would probably be useless.

Ultimately, it's probably a question of how much paranoia you have. But as I said, in my opinion, obfuscation should always aim to prevent uploads from appearing in the indexer at all. And as the indexer evolves, so must the obfuscation method.

animetosho commented 3 weeks ago

even though they do not know what they are downloading.

How often do people download something they don't know the contents of?

The obfuscation should prevent the upload from appearing in the indexer at all.

Why?

Naming everything the same would contradict this

Does it?
If all uploads have the same name, the indexer can no longer use the name as a reliable means of joining articles together. Perhaps this indexer displays multiple entries for 'file', but whether the articles even match up is a different matter.

Ultimately, it's probably a question of how much paranoia you have

Unfortunately I don't really believe in unfounded paranoia, as you can probably tell from my responses.
(hence my general stance against many obfuscation practices)

And as the indexer evolves, so must the obfuscation method.

I also have no interest in participating in this sort of contest.

Tensai75 commented 3 weeks ago

I also have no interest in participating in this sort of contest.

But you could at least give the option to the users who want to be that paranoid. A new cmd switch and a few lines of code. Shouldn't be that hard to implement... Would you accept a PR to implement this?

Regards, Tenasi

animetosho commented 3 weeks ago

Nyuu provides many options, but I need to draw a line somewhere and don't intend to support every crazy idea, that someone thought was sensible, out there.

Altering the name per article violates the yEnc spec, which is something I don't intend to allow easily, particularly if the reason is questionable paranoia.

You're welcome to maintain your own fork however.
Apologies if that's not what you wanted to hear, but hopefully you understand.

Tensai75 commented 3 weeks ago

Sure, no problem. I am working on my own, new "crazy idea" of usenet upload anyway ;-) In any case I very much appreciate your work, thanks mate!

Regards, Tensai

animetosho commented 3 weeks ago

I wish you the best of luck with your endeavor!

Antidote2151 commented 2 weeks ago

Thank you for the discussion!