rusq / slackdump

Save or export your private and public Slack messages, threads, files, and users locally without admin privileges.
GNU General Public License v3.0
1.51k stars 70 forks source link

slackdump exports aren't viewable using slack-export-viewer #282

Closed ritiek closed 5 months ago

ritiek commented 5 months ago

Describe the bug Hi, thanks for creating slackdump! Not sure if this should be reported on https://github.com/hfaran/slack-export-viewer/ (I can do so). I'm having a problem viewing my slackdump exports using slack-export-viewer.

To Reproduce

  1. Export a channel using slackdump with myexport as the export directory.
  2. Execute slack-export-viewer with $ slack-export-viewer -z myexport -I 0.0.0.0.

Expected behavior Archive should be visible through slack-export-viewer in the web browser.

Output Instead, I get a 500 Internal Server Error from slack-export-viewer with the following traceback:

$ slack-export-viewer -z myexport -I 0.0.0.0
$ slack-export-viewer -z myexport -I 0.0.0.0
WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
 * Running on all addresses (0.0.0.0)
 * Running on http://127.0.0.1:5000
 * Running on http://192.168.5.11:5000
Press CTRL+C to quit
192.168.5.10 - - [09/Apr/2024 14:17:15] "GET / HTTP/1.1" 500 -
INFO:werkzeug:192.168.2.5 - - [09/Apr/2024 14:17:15] "GET / HTTP/1.1" 500 -
Error on request:
Traceback (most recent call last):
  File "/home/ritiek/.local/pipx/venvs/slack-export-viewer/lib/python3.11/site-packages/werkzeug/serving.py", line 364, in run_wsgi
    execute(self.server.app)
  File "/home/ritiek/.local/pipx/venvs/slack-export-viewer/lib/python3.11/site-packages/werkzeug/serving.py", line 325, in execute
    application_iter = app(environ, start_response)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ritiek/.local/pipx/venvs/slack-export-viewer/lib/python3.11/site-packages/flask/app.py", line 2213, in __call__
    return self.wsgi_app(environ, start_response)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ritiek/.local/pipx/venvs/slack-export-viewer/lib/python3.11/site-packages/flask/app.py", line 2193, in wsgi_app
    response = self.handle_exception(e)
               ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ritiek/.local/pipx/venvs/slack-export-viewer/lib/python3.11/site-packages/flask/app.py", line 2190, in wsgi_app
    response = self.full_dispatch_request()
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ritiek/.local/pipx/venvs/slack-export-viewer/lib/python3.11/site-packages/flask/app.py", line 1486, in full_dispatch_request
    rv = self.handle_user_exception(e)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ritiek/.local/pipx/venvs/slack-export-viewer/lib/python3.11/site-packages/flask/app.py", line 1484, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ritiek/.local/pipx/venvs/slack-export-viewer/lib/python3.11/site-packages/flask/app.py", line 1469, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ritiek/.local/pipx/venvs/slack-export-viewer/lib/python3.11/site-packages/slackviewer/app.py", line 119, in index
    return dm_id(dms[0])
           ^^^^^^^^^^^^^
  File "/home/ritiek/.local/pipx/venvs/slack-export-viewer/lib/python3.11/site-packages/slackviewer/app.py", line 67, in dm_id
    return flask.render_template("viewer.html", messages=messages,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ritiek/.local/pipx/venvs/slack-export-viewer/lib/python3.11/site-packages/flask/templating.py", line 151, in render_template
    return _render(app, template, context)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ritiek/.local/pipx/venvs/slack-export-viewer/lib/python3.11/site-packages/flask/templating.py", line 132, in _render
    rv = template.render(context)
         ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ritiek/.local/pipx/venvs/slack-export-viewer/lib/python3.11/site-packages/jinja2/environment.py", line 1301, in render
    self.environment.handle_exception()
  File "/home/ritiek/.local/pipx/venvs/slack-export-viewer/lib/python3.11/site-packages/jinja2/environment.py", line 936, in handle_exception
    raise rewrite_traceback_stack(source=source)
  File "/home/ritiek/.local/pipx/venvs/slack-export-viewer/lib/python3.11/site-packages/slackviewer/templates/viewer.html", line 65, in top-level template code
    {% if "Thread Reply:" in message.msg %}
TypeError: argument of type 'NoneType' is not iterable
ERROR:werkzeug:Error on request:
Traceback (most recent call last):
  File "/home/ritiek/.local/pipx/venvs/slack-export-viewer/lib/python3.11/site-packages/werkzeug/serving.py", line 364, in run_wsgi
    execute(self.server.app)
  File "/home/ritiek/.local/pipx/venvs/slack-export-viewer/lib/python3.11/site-packages/werkzeug/serving.py", line 325, in execute
    application_iter = app(environ, start_response)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ritiek/.local/pipx/venvs/slack-export-viewer/lib/python3.11/site-packages/flask/app.py", line 2213, in __call__
    return self.wsgi_app(environ, start_response)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ritiek/.local/pipx/venvs/slack-export-viewer/lib/python3.11/site-packages/flask/app.py", line 2193, in wsgi_app
    response = self.handle_exception(e)
               ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ritiek/.local/pipx/venvs/slack-export-viewer/lib/python3.11/site-packages/flask/app.py", line 2190, in wsgi_app
    response = self.full_dispatch_request()
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ritiek/.local/pipx/venvs/slack-export-viewer/lib/python3.11/site-packages/flask/app.py", line 1486, in full_dispatch_request
    rv = self.handle_user_exception(e)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ritiek/.local/pipx/venvs/slack-export-viewer/lib/python3.11/site-packages/flask/app.py", line 1484, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ritiek/.local/pipx/venvs/slack-export-viewer/lib/python3.11/site-packages/flask/app.py", line 1469, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ritiek/.local/pipx/venvs/slack-export-viewer/lib/python3.11/site-packages/slackviewer/app.py", line 119, in index
    return dm_id(dms[0])
           ^^^^^^^^^^^^^
  File "/home/ritiek/.local/pipx/venvs/slack-export-viewer/lib/python3.11/site-packages/slackviewer/app.py", line 67, in dm_id
    return flask.render_template("viewer.html", messages=messages,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ritiek/.local/pipx/venvs/slack-export-viewer/lib/python3.11/site-packages/flask/templating.py", line 151, in render_template
    return _render(app, template, context)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ritiek/.local/pipx/venvs/slack-export-viewer/lib/python3.11/site-packages/flask/templating.py", line 132, in _render
    rv = template.render(context)
         ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ritiek/.local/pipx/venvs/slack-export-viewer/lib/python3.11/site-packages/jinja2/environment.py", line 1301, in render
    self.environment.handle_exception()
  File "/home/ritiek/.local/pipx/venvs/slack-export-viewer/lib/python3.11/site-packages/jinja2/environment.py", line 936, in handle_exception
    raise rewrite_traceback_stack(source=source)
  File "/home/ritiek/.local/pipx/venvs/slack-export-viewer/lib/python3.11/site-packages/slackviewer/templates/viewer.html", line 65, in top-level template code
    {% if "Thread Reply:" in message.msg %}
TypeError: argument of type 'NoneType' is not iterable

Desktop:

rusq commented 5 months ago

Hey @ritiek , this is not slackdump problem. You should probably file an issue in that repository. However, I did some research for you, and I believe that this is the commit that is the source of this error: https://github.com/hfaran/slack-export-viewer/commit/f5851dd62a1c47419e7e6fa16063bb5c88f54ba7

What happens is, there's a message with just files, so the condition on line 64 evaluates to true, and the code in the "if" statement executes. But on the next line 65, it tries to examine message.msg, which, is "None".

You can submit a fix to that repo, all it has to do, is something like

-{% if "Thread Reply:" in message.msg %}
+{% if message.msg and "Thread Reply:" in message.msg %}

Alternatively, you can check out the experimental v3 branch and run

go run ./cmd/slackdump view myexport.zip

the viewer will be available on http://localhost:8080

Keep in mind that this is an early experimental feature, and v3 is in heavy development and not a release quality.

rusq commented 5 months ago

Another solution is use the previous version of slack-export-viewer (v1.4.1) that did not have this change which caused the issue.

ritiek commented 5 months ago

I see, thanks! v1.4.1 indeed seems to work well. I opened an issue on https://github.com/hfaran/slack-export-viewer/issues/187.


I also tried out the v3 branch as suggested. Though, my initial export was made using slackdump v2.5.8 via:

$ slackdump -export myexport.zip -export-type standard https://myexport.slack.com/archives/D0123ABCXYZ

but got this traceback when trying to view the exports (that were made on v2.5.8) using the v3 branch:

$ go run ./cmd/slackdump view -listen 0.0.0.0:25674 myexport.zip
$ go run ./cmd/slackdump view -listen 0.0.0.0:25674 myexport.zip
view: 2024/04/10 05:02:30 listening on 0.0.0.0:25674
view: 2024/04/10 05:02:30 unable to open browser: exec: "xdg-open,x-www-browser,www-browser": executable file not found in $PATH
2024/04/10 05:02:40 "GET http://192.168.5.10:25674/ HTTP/1.1" from 192.168.5.11:42942 - 200 2900B in 624.354µs
2024/04/10 05:02:40 "GET http://192.168.5.10:25674/favicon.ico HTTP/1.1" from 192.168.5.11:42942 - 200 2900B in 115.982µs
2024/04/10 05:02:46 "GET http://192.168.5.10:25674/archives/D0123ABCXYZ HTTP/1.1" from 192.168.5.11:42942 - 000 0B in 97.320347ms
2024/04/10 05:02:46 http: panic serving 192.168.5.11:42942: runtime error: invalid memory address or nil pointer dereference
goroutine 7 [running]:
net/http.(*conn).serve.func1()
        /home/ritiek/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.1.linux-arm64/src/net/http/server.go:1898 +0xb0
panic({0xa2e680?, 0x16c4b50?})
        /home/ritiek/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.1.linux-arm64/src/runtime/panic.go:770 +0x124
github.com/rusq/slackdump/v3/internal/viewer/source.(*Export).walkChannelMessages.func1({0x400080b940, 0x40}, {0x7ffe98502b20?, 0x40000d2b00?}, {0x0?, 0x0?})
        /media/services/slackdump/slackdump-git/internal/viewer/source/export.go:120 +0x178
io/fs.walkDir({0xfd0680, 0x40001a6080}, {0x400080b940, 0x40}, {0x7ffe98502b20, 0x40000d2b00}, 0x40009070a0)
        /home/ritiek/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.1.linux-arm64/src/io/fs/walk.go:73 +0x5c
io/fs.walkDir({0xfd0680, 0x40001a6080}, {0x40005e3848, 0x17}, {0xfd7fa0, 0x40004e6f08}, 0x40009070a0)
        /home/ritiek/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.1.linux-arm64/src/io/fs/walk.go:95 +0x24c
io/fs.walkDir({0xfd0680, 0x40001a6080}, {0x40006b0370, 0xb}, {0xfd7d70, 0x40004fa200}, 0x40009070a0)
        /home/ritiek/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.1.linux-arm64/src/io/fs/walk.go:95 +0x24c
io/fs.WalkDir({0xfd0680, 0x40001a6080}, {0x40006b0370, 0xb}, 0x400036f0a0)
        /home/ritiek/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.1.linux-arm64/src/io/fs/walk.go:122 +0x8c
github.com/rusq/slackdump/v3/internal/viewer/source.(*Export).walkChannelMessages(0x40003296c0, {0x400048ef0e, 0xb}, 0x400036f138)
        /media/services/slackdump/slackdump-git/internal/viewer/source/export.go:107 +0x188
github.com/rusq/slackdump/v3/internal/viewer/source.(*Export).AllMessages(0x30?, {0x400048ef0e?, 0x400041e780?})
        /media/services/slackdump/slackdump-git/internal/viewer/source/export.go:86 +0x44
github.com/rusq/slackdump/v3/internal/viewer.(*Viewer).channelHandler(0x40004d4000, {0x7ffe984b05f8, 0x40002790c0}, 0x400045e240, {0x400048ef0e, 0xb})
        /media/services/slackdump/slackdump-git/internal/viewer/handlers.go:56 +0x54
github.com/rusq/slackdump/v3/internal/viewer.New.(*Viewer).newFileHandler.func4({0x7ffe984b05f8, 0x40002790c0}, 0x400045e240)
        /media/services/slackdump/slackdump-git/internal/viewer/handlers.go:51 +0x80
net/http.HandlerFunc.ServeHTTP(0x40003292b0?, {0x7ffe984b05f8?, 0x40002790c0?}, 0xb1b700?)
        /home/ritiek/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.1.linux-arm64/src/net/http/server.go:2166 +0x38
net/http.(*ServeMux).ServeHTTP(0x16cf1a0?, {0x7ffe984b05f8, 0x40002790c0}, 0x400045e240)
        /home/ritiek/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.1.linux-arm64/src/net/http/server.go:2683 +0x1a4
github.com/go-chi/chi/v5/middleware.init.0.RequestLogger.func1.1({0xfd6538, 0x40001762a0}, 0x400045e120)
        /home/ritiek/go/pkg/mod/github.com/go-chi/chi/v5@v5.0.12/middleware/logger.go:55 +0x128
net/http.HandlerFunc.ServeHTTP(0x0?, {0xfd6538?, 0x40001762a0?}, 0x400036fb50?)
        /home/ritiek/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.1.linux-arm64/src/net/http/server.go:2166 +0x38
net/http.serverHandler.ServeHTTP({0x4000565920?}, {0xfd6538?, 0x40001762a0?}, 0x6?)
        /home/ritiek/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.1.linux-arm64/src/net/http/server.go:3137 +0xbc
net/http.(*conn).serve(0x4000198750, {0xfd7a80, 0x40005658c0})
        /home/ritiek/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.1.linux-arm64/src/net/http/server.go:2039 +0x508
created by net/http.(*Server).Serve in goroutine 1
        /home/ritiek/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.1.linux-arm64/src/net/http/server.go:3285 +0x3f0


However, if I also do the export using the v3 branch using:

$ go run ./cmd/slackdump export -o myexport.zip https://myexport.slack.com/archives/D0123ABCXYZ

then go run ./cmd/slackdump view -listen 0.0.0.0:25674 myexport.zip works fine.

I felt this might be a lil weird, since trying out with slack-export-viewer v1.4.1, both the slackdump exports made using v2.5.8 as well as made using v3 branch load fine for me.

rusq commented 5 months ago

Hey @ritiek , thanks for trying it out! Interesting, I'll have a look tonight why is it panicking

rusq commented 5 months ago

Hahahah, it literally stepped on the same rakes that the slack-export-viewer stepped a day earlier

        for _, m := range em {
            if err := fn(&slack.Message{Msg: *m.Msg}); err != nil {
                return err
            }
        }

see this tiny *m.Msg I don't know how this is possible, but for some reason in the slack export that you provided, it's nil.

rusq commented 5 months ago

If you pull the latest v3, it should be fixed! That stack trace was really helpful, thank you

ritiek commented 5 months ago

yay, works good now, thanks!