4350pChris / slack-vuesualizer

Display and search your Slack export
https://slack-vuesualizer.de
MIT License
57 stars 9 forks source link

Support for attachments inside the .zip file #838

Open santi-ifontana opened 3 months ago

santi-ifontana commented 3 months ago

Hello. I love your app, so far it is the nicest and fastest solution for visualizing Slack export files.

But quick question, is it possible or planned to support attachments in the .zip? I have used this tool: https://github.com/grundleborg/slack-advanced-exporter?tab=readme-ov-file Why? Because we are going to close our Slack account and we need to keep a working archival for legal purposes (including the attachments).

Please, would love to hear from you.

Thank you.

4350pChris commented 3 months ago

Could you maybe produce a sanitized .zip file with attachments for me to test? That would make it a lot easier, otherwise I don't even really know what I'm dealing with. I'll try to look it up though, if you provide a file I'm fairly sure I can do that in a timely manner.

Also, if this is a format specific to your company, or you need some more support integrating, I'm open to freelance work.

santi-ifontana commented 3 months ago

Hey Chris,

Here is the new .zip file with the attachments inside:

Screenshot 2024-06-28 at 14 08 35 Screenshot 2024-06-28 at 14 08 42 Screenshot 2024-06-28 at 14 08 49

Checking a channel .json file, the links are kept unchanged. "url_private": "https:\/\/files.slack.com\/files-pri\/T08KDGR8S-F01QF9TSN9P\/image_from_ios.png?t=xoxe-<token>", "url_private_download": "https:\/\/files.slack.com\/files-pri\/T08KDGR8S-F01QF9TSN9P\/download\/image_from_ios.png?t=xoxe-<token>",

As the .zip is over 100 GB I cannot even finish unzipping it before I run out of space. Let me know if this helps you or if you really need the .zip, I will check how I can get it unzipped and sanitized.

Just to mention, the tool used to feed the attachment into the .zip is part of the mattermost import process: https://docs.mattermost.com/onboard/migrate-from-slack.html

Thanks.

4350pChris commented 3 months ago

So what exactly would you expect from this app? I don't quite understand what to do with the attachments. Make them downloadable in the file section instead of having the URL to the download?

santi-ifontana commented 2 months ago

Hey,

Would be great to be able to have the files downloaded locally and not dependent on the Slack servers. In our case, we are planning on ending our Slack subscription and we would like to keep an archival running.

This solution, although it crashes often, it allows you to "cache" all the files: https://github.com/thayakawa-gh/SlackLogViewer

So, to sum up, instead of trying to upload a 113 GB .zip file with all the attachments (with the tool I provided earlier), maybe upload the regular .zip archive offered by the official Slack export tool and then, have the option to download locally all the files.

Sorry, English is not my first language. I hope I am clear enough.

Thank you!

4350pChris commented 2 months ago

Ah I see. So basically just a bulk download of files? I think I can manage that, although I can't test it with as many attachments. Maybe chunking would be a good idea here, so you don't have to download the whole >100GB at once?

santi-ifontana commented 2 months ago

So basically just a bulk download of files?

Yes, so the app can run standalone (no need to reach Slack's servers).

I can't test it with as many attachments

We can help you test.

Maybe chunking would be a good idea here, so you don't have to download the whole >100GB at once?

That is no problem. We have a small server running in GCP, so if it downloads all at once or in chunks is fine.

What would be great is to be able to define the mount dir in Docker / Composer, so I can manage where the files are stored.

Thank you.