search

sosreport / sos

A unified tool for collecting system logs and other debug information
http://sos.rtfd.org
GNU General Public License v2.0
508 stars 542 forks source link

Gather collection within its own directory and make it clearer about what is shareable and not #2215

Open slashdd opened 4 years ago

slashdd commented 4 years ago

This output has been made from Ubuntu which uses /tmp as tmp-dir, but same behaviour is found for the enforced /var/tmp location.

Current situation:

ls /tmp/sosreport-*

sosreport-host0-2020-08-26-eywxccq-obfuscated.tar.xz     
sosreport-host0-2020-08-26-eywxccq-obfuscated.tar.xz.md5 
sosreport-host0-2020-08-26-eywxccq-private_map            
sosreport-sosfocal-2020-08-26-eywxccq-obfuscation.log
sosreport-sosfocal-2020-08-26-eywxccq.tar.xz
sosreport-sosfocal-2020-08-26-eywxccq.tar.xz.md5

As we implement new sos functionality, we could easily pollute the tmp-dir (especially if multiples sosreport archives are found) and make it confusing to users as to what need to be sent or not.

Would it make sense to isolate the collection inside its own root directory such as sosreport-sosfocal-eywxccq/ (sosreport-HOSTNAME-RANDOM_HASH)? And possibly separate private and shareable files into obvious folder name for a user to exactly know what could be sent to a 3rd party vendor or/and not.

Example:

/tmp/sosreport-sosfocal-eywxccq/shareable:
sosreport-host0-2020-08-26-eywxccq-obfuscated.tar.xz     
sosreport-host0-2020-08-26-eywxccq-obfuscated.tar.xz.md5 
sosreport-sosfocal-2020-08-26-eywxccq.tar.xz
sosreport-sosfocal-2020-08-26-eywxccq.tar.xz.md5

/tmp/sosreport-sosfocal-eywxccq/private:
sosreport-host0-2020-08-26-eywxccq-private_map            
sosreport-sosfocal-2020-08-26-eywxccq-obfuscation.log
slashdd commented 4 years ago

In addition, having isolate sosreport directories inside tmp-dir could also largely help Ubuntu/Debian to move from /tmp to default location /var/tmp since it will ease the feasibility to do a tmpfiles.d cleanup directive if not directly found at the root of /var/tmp.

TurboTurtle commented 4 years ago

First, the entire point of the obfuscation is that the un-obfuscated report is not shareable - so I wouldn't be on board with adding it to a "shareable" section/directory/etc.

But beyond that I don't see how this text, which is printed at the end of every sos clean run, does not make it explicitly obvious what is to be shared and what is to be kept private:

A mapping of obfuscated elements is available at
    /var/tmp/sosreport-host0-02732787-2020-08-26-zegjdtl-private_map

The obfuscated archive is available at
    /var/tmp/sosreport-host0-02732787-2020-08-26-zegjdtl-obfuscated.tar.xz

    Size    1.97MiB
    Owner   root

Please send the obfuscated archive to your support representative and keep the mapping file private
slashdd commented 4 years ago

First, the entire point of the obfuscation is that the un-obfuscated report is not shareable - so I wouldn't be on board with adding it to a "shareable" section/directory/etc.

Well it's up to the user to decide if they want to send the obfuscated one or not, but both are imho considered "shareable". After that it is only depending on one internal policy.

But I'm okay with other approaches

shareable/obfuscated
shareable/non-obfuscated
....

As an example ^

But beyond that I don't see how this text, which is printed at the end of every sos clean run, does not make it explicitly obvious what is to be shared and what is to be kept private:

Right, good point.

A mapping of obfuscated elements is available at
  /var/tmp/sosreport-host0-02732787-2020-08-26-zegjdtl-private_map

The obfuscated archive is available at
  /var/tmp/sosreport-host0-02732787-2020-08-26-zegjdtl-obfuscated.tar.xz

  Size    1.97MiB
  Owner   root

Please send the obfuscated archive to your support representative and keep the mapping file private
slashdd commented 4 years ago

While I'm +1 to change the printed output to make it obvious, I don't think it's fair to assume that users do fully read the whole printed output and won't necessarily remember it when time will come to actually upload stuff. I still do think having a directory structure that reflect that along with the output change will also help.

TurboTurtle commented 4 years ago

While I'm +1 to change the printed output to make it obvious, I don't think it's fair to assume that users do fully read the whole printed output and won't necessarily remember it when time will come to actually upload stuff.

Just to be clear, I'm not proposing a change to the printed text. The above is what is already printed today.

Personally, I do think it's fair to assume users read our output. Why else are we outputting it? If users can't be assumed to read our printed text then why do we bother with our legal disclaimers?

I still do think having a directory structure that reflect that along with the output change will also help.

I'm not convinced that this would actually be a net benefit. Especially when we'd then have two different output approaches on the filesystem (report vs clean, or even report vs report --clean). But I'm also not saying this is a hard nack. If broad consensus builds in favor of it, then I'll be inclined to reconsider.

bmr-cymru commented 4 years ago

What would soscleaner do?

slashdd commented 4 years ago

It does the following for now:

08-30 12:59:44 soscleaner CONSOLE: Creating IP Report - /tmp/soscleaner-2866036090755265/soscleaner-2866036090755265-ip.csv
08-30 12:59:44 soscleaner CONSOLE: Creating Hostname Report - /tmp/soscleaner-2866036090755265/soscleaner-2866036090755265-hostname.csv
08-30 12:59:44 soscleaner CONSOLE: Creating Domainname Report - /tmp/soscleaner-2866036090755265/soscleaner-2866036090755265-dn.csv
08-30 12:59:44 soscleaner CONSOLE: Creating Username Report - /tmp/soscleaner-2866036090755265/soscleaner-2866036090755265-username.csv
08-30 12:59:44 soscleaner CONSOLE: Creating MAC address Report - /tmp/soscleaner-2866036090755265/soscleaner-2866036090755265-mac.csv
08-30 12:59:44 soscleaner CONSOLE: Creating keyword address Report - /tmp/soscleaner-2866036090755265/soscleaner-2866036090755265-kw.csv
08-30 12:59:44 soscleaner CONSOLE: Creating sosreport Report - /tmp/soscleaner-2866036090755265/soscleaner-2866036090755265-sosreport.csv
08-30 12:59:44 soscleaner CONSOLE: Creating SOSCleaner Archive - /tmp/soscleaner-2866036090755265/soscleaner-2866036090755265.tar.gz

Placing everything inside the same folder created at each execution. The next step was for me to implement a structure using a 'shareable' and 'keptprivate' approach.

Hence, this is why I'm opening the discussion for it now, cause I do think there is a value to make it more obvious to the eyes but also the users.