animetosho / Nyuu

Flexible usenet binary posting tool
224 stars 32 forks source link

any arg in '--subject' results in invalid .nzb #43

Closed nonspin closed 6 years ago

nonspin commented 6 years ago

Let me explain what i was trying to do: These days, file-'obfuscation' is almost mandatory, that's not a big secret anymore.

I wrote a PowerShell-Module that handles everything from creating .rar-files -> .par2-files -> indiv.obfuscation [MACTripleDES/MD5 .. and some shifty algo] To the point that the .par2 is still able to restore everything..

Even though nyuu i pretty much the most thought-out solution and by far the fastest .. it can't handle obfuscated files. Well.. it can, but addind [01/XX] and ' yenc' and filesize to the subject it more like a "doh!'-effect.

So i gave it a shot .. and since {filename} is the only requirement, --subject {filename} or any variant like "{filename}" should result in something like subject=""03A0EDCE510E75BD" (1/42)"

--subject {filename} messes with the contruction in /lib/uploadmgr.js

Line 153 if(fileNumTotal > 1) preSubj += '[' + '0000000000000000'.substr(0, (''+fileNumTotal).length - (''+fileNum).length) + fileNum + '/' + fileNumTotal + '] - '; // TODO: should we revert to single part titles if only 1 part? preSubj += '"' + fileName.replace(RE_QUOTE, '') + '" yEnc ('; var postSubj = '/' + enc.parts + ') ' + fileSize + (this.opts.comment2 ? ' ' + this.opts.comment2 : ''); to /** if(fileNumTotal > 1) preSubj += '[' + '0000000000000000'.substr(0, (''+fileNumTotal).length - (''+fileNum).length) + fileNum + '/' + fileNumTotal + '] - '; **/ // TODO: should we revert to single part titles if only 1 part? preSubj += '"' + fileName.replace(RE_QUOTE, '') + '" ('; var postSubj = '/' + enc.parts + ') ' + (this.opts.comment2 ? ' ' + this.opts.comment2 : '');

preSubj starts the contruction of the string no matter what which of course breaks the whole thing ... it also suppresses filenum, which then invalidates the .nzb segment contruction.

animetosho commented 6 years ago

I'm a little confused at what you're trying to say, but if you want your subjects to be like "03A0EDCE510E75BD" (1/42) in the NZB, then you'll need to specify them as such. For example --subject "\"{filename}\" ({part}/{parts})" (CMD/Bash) or --subject """{filename}"" ({part}/{parts})" (PowerShell)
Nyuu takes what you tell it to do quite literally. If you tell it to only put a filename in the subject, it will only put that there and nothing else (including part numbers etc).

Be aware that most shells don't differentiate between {filename} and "{filename}" because quotes have special meaning. See your shell's documentation for details, for example PowerShell requires quotes to be doubled (or escaped with a `) for it to be treated as a literal. For example echo a and echo "a" produce the same text in most shells (except CMD, as echo is a special operator there).

The code you've pointed to isn't used if --subject is supplied, because --subject completely overrides the default behavior.

nonspin commented 6 years ago

Of course you're right - now that understand how to address it, let me explain why i thought it was an issue:

Coming from the "YencPowerPost"-era, doing the same over-and-over, certain things harden as facts in your head... for example: $filename is the only necessity for the subject - which is still true, i guess.

Therefore, --subject ""{filename}"" (or "\"{filename}\") to me is 'true'. Addionally - "yenc" and "[parts/parts]" aren't technically part of the actual subject (i know. =/) .. because not necessary (at that's what "YencPP" taught us and $F was the only variable needed).

So i took a peek at the construction of the string and that's where world's collide i guess.

So, maybe consider this: An override for the CLI like "--bare-minimum" or "--filename-only" or "--min-subject" .. etc that would use a "YencPP $F-like template"

and/or some valid example(s) for the help.txt to make it a bit easier to let go of the old stuff.

Thanks in advance ;D

animetosho commented 6 years ago

Thanks for the reply and explanation!

One thing I don't quite understand is the reason to change the subject at all. I can't see how removing "yEnc" or the filesize actually improves obfuscation (the correct details exist in the yEnc headers anyway).

As such, I don't really see a reason why most would change the subject, so I consider it to be an advanced feature (i.e. "I know what I'm doing, do as I say").

I don't really want to complicate the command line any more than necessary, as it's already quite complicated, so I'm a bit against including additional flags. The help.txt does provide the default subject format as a guide, which can be used as a basis for customization.
I can put a note in the help.txt about the --subject header being the complete subject rather than just part of the subject.

because not necessary (at that's what "YencPP" taught us and $F was the only variable needed).

Actually, the yEnc specifications require yEnc to be present in the subject:

The keyword "yEnc" is mandatory, and must appear between the filename and the size

Omitting it would violate the spec, though indexers and such probably don't care. As such, unfortunately, what you've been taught from those posters is incorrect.

nonspin commented 6 years ago

I guess we don't need to argue what is uploaded to the usenet, right ? Keeping that in mind, obfuscation is more or less necessary these days. A subject looking like: "[01/50] " yenc [parts/parts] still sticks out and renders the obfuscation useless, wouldn't you agree ? ( .. I'm leaving the timestamp and '@nyuu' part from the segments out, because i've already changed it and it would go too far.)

I know you have to draw the line somewhere and i totally understand it, but since nyuu is very advanced in terms of "customizability", a feature that would - in theory - support (or allow) the obfuscation of contents isn't much of a stretch ?

animetosho commented 6 years ago

Well, if your aim is to hide it from some indexers, I suppose whatever works is fine. Though, you probably don't really need the part indicator if that's the case. You could probably even just make all subjects completely random.

Otherwise I don't see how removing "yEnc" makes a post stick out any less than having it there. The size of the article alone is likely a very strong indicator that it's a binary post (and all binary posts are yEnc these days). Combined with the fact that it has the same poster (or X-Trace) doesn't leave much doubt.
I won't debate on whether you should obfuscate or not, but I do believe in something more solid than hand-wavy reasoning.

The problem with obfuscation is that everyone has their own secret way of doing things (which may or may not make any sense). I don't intend to try to cater for every strategy out there (assuming people will even tell you about them). You may think obfuscated subjects should be "{random}" ({part}/{parts}), someone else will think otherwise. Nyuu provides a base level of control over various aspects, which you can use to implement whatever method you choose, but I guess I'd have to draw the line there.