search

hfaran / slack-export-viewer

A Slack Export archive viewer that allows you to easily view and share your Slack team's export
https://pypi.python.org/pypi/slack-export-viewer
MIT License
901 stars 189 forks source link

Error 500 / KeyError: 'ts' #165

Open user68329 opened 1 year ago

user68329 commented 1 year ago

As soon as the app loads, i'm getting the below message. The browser shows "Internal Server Error"

Any idea what this could be caused by?

apple ~ % slack-export-viewer -z /Users/username/Documents/Downloads/x/ -p 5001
Archive already extracted. Viewing from /Users/username/Documents/Downloads/x...
 * Serving Flask app 'slackviewer.app'
 * Debug mode: off
 * Running on http://localhost:5001 (Press CTRL+C to quit)
127.0.0.1 - - [04/May/2023 16:18:02] "GET / HTTP/1.1" 500 -
INFO:werkzeug:127.0.0.1 - - [04/May/2023 16:18:02] "GET / HTTP/1.1" 500 -
Error on request:
Traceback (most recent call last):
  File "/Users/username/.local/pipx/venvs/slack-export-viewer/lib/python3.11/site-packages/werkzeug/serving.py", line 335, in run_wsgi
    execute(self.server.app)
  File "/Users/username/.local/pipx/venvs/slack-export-viewer/lib/python3.11/site-packages/werkzeug/serving.py", line 322, in execute
    application_iter = app(environ, start_response)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/username/.local/pipx/venvs/slack-export-viewer/lib/python3.11/site-packages/flask/app.py", line 2213, in __call__
    return self.wsgi_app(environ, start_response)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/username/.local/pipx/venvs/slack-export-viewer/lib/python3.11/site-packages/flask/app.py", line 2193, in wsgi_app
    response = self.handle_exception(e)
               ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/username/.local/pipx/venvs/slack-export-viewer/lib/python3.11/site-packages/flask/app.py", line 2190, in wsgi_app
    response = self.full_dispatch_request()
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/username/.local/pipx/venvs/slack-export-viewer/lib/python3.11/site-packages/flask/app.py", line 1486, in full_dispatch_request
    rv = self.handle_user_exception(e)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/username/.local/pipx/venvs/slack-export-viewer/lib/python3.11/site-packages/flask/app.py", line 1484, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/username/.local/pipx/venvs/slack-export-viewer/lib/python3.11/site-packages/flask/app.py", line 1469, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/username/.local/pipx/venvs/slack-export-viewer/lib/python3.11/site-packages/slackviewer/app.py", line 93, in index
    return channel_name(channels[0])
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/username/.local/pipx/venvs/slack-export-viewer/lib/python3.11/site-packages/slackviewer/app.py", line 19, in channel_name
    return flask.render_template("viewer.html", messages=messages,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/username/.local/pipx/venvs/slack-export-viewer/lib/python3.11/site-packages/flask/templating.py", line 151, in render_template
    return _render(app, template, context)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/username/.local/pipx/venvs/slack-export-viewer/lib/python3.11/site-packages/flask/templating.py", line 132, in _render
    rv = template.render(context)
         ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/username/.local/pipx/venvs/slack-export-viewer/lib/python3.11/site-packages/jinja2/environment.py", line 1301, in render
    self.environment.handle_exception()
  File "/Users/username/.local/pipx/venvs/slack-export-viewer/lib/python3.11/site-packages/jinja2/environment.py", line 936, in handle_exception
    raise rewrite_traceback_stack(source=source)
  File "/Users/username/.local/pipx/venvs/slack-export-viewer/lib/python3.11/site-packages/slackviewer/templates/viewer.html", line 64, in top-level template code
    {{render_message(message, None, no_external_references)}}
  File "/Users/username/.local/pipx/venvs/slack-export-viewer/lib/python3.11/site-packages/jinja2/runtime.py", line 777, in _invoke
    rv = self._func(*arguments)
         ^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/username/.local/pipx/venvs/slack-export-viewer/lib/python3.11/site-packages/slackviewer/templates/util.html", line 16, in template
    <div id="{{ message.id }}">
  File "/Users/username/.local/pipx/venvs/slack-export-viewer/lib/python3.11/site-packages/jinja2/environment.py", line 485, in getattr
    return getattr(obj, attribute)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/username/.local/pipx/venvs/slack-export-viewer/lib/python3.11/site-packages/slackviewer/message.py", line 108, in id
    return self.time
           ^^^^^^^^^
  File "/Users/username/.local/pipx/venvs/slack-export-viewer/lib/python3.11/site-packages/slackviewer/message.py", line 51, in time
    tsepoch = float(self._message["ts"].split(".")[0])
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
KeyError: 'ts'
ERROR:werkzeug:Error on request:
Traceback (most recent call last):
  File "/Users/username/.local/pipx/venvs/slack-export-viewer/lib/python3.11/site-packages/werkzeug/serving.py", line 335, in run_wsgi
    execute(self.server.app)
  File "/Users/username/.local/pipx/venvs/slack-export-viewer/lib/python3.11/site-packages/werkzeug/serving.py", line 322, in execute
    application_iter = app(environ, start_response)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/username/.local/pipx/venvs/slack-export-viewer/lib/python3.11/site-packages/flask/app.py", line 2213, in __call__
    return self.wsgi_app(environ, start_response)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/username/.local/pipx/venvs/slack-export-viewer/lib/python3.11/site-packages/flask/app.py", line 2193, in wsgi_app
    response = self.handle_exception(e)
               ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/username/.local/pipx/venvs/slack-export-viewer/lib/python3.11/site-packages/flask/app.py", line 2190, in wsgi_app
    response = self.full_dispatch_request()
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/username/.local/pipx/venvs/slack-export-viewer/lib/python3.11/site-packages/flask/app.py", line 1486, in full_dispatch_request
    rv = self.handle_user_exception(e)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/username/.local/pipx/venvs/slack-export-viewer/lib/python3.11/site-packages/flask/app.py", line 1484, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/username/.local/pipx/venvs/slack-export-viewer/lib/python3.11/site-packages/flask/app.py", line 1469, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/username/.local/pipx/venvs/slack-export-viewer/lib/python3.11/site-packages/slackviewer/app.py", line 93, in index
    return channel_name(channels[0])
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/username/.local/pipx/venvs/slack-export-viewer/lib/python3.11/site-packages/slackviewer/app.py", line 19, in channel_name
    return flask.render_template("viewer.html", messages=messages,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/username/.local/pipx/venvs/slack-export-viewer/lib/python3.11/site-packages/flask/templating.py", line 151, in render_template
    return _render(app, template, context)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/username/.local/pipx/venvs/slack-export-viewer/lib/python3.11/site-packages/flask/templating.py", line 132, in _render
    rv = template.render(context)
         ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/username/.local/pipx/venvs/slack-export-viewer/lib/python3.11/site-packages/jinja2/environment.py", line 1301, in render
    self.environment.handle_exception()
  File "/Users/username/.local/pipx/venvs/slack-export-viewer/lib/python3.11/site-packages/jinja2/environment.py", line 936, in handle_exception
    raise rewrite_traceback_stack(source=source)
  File "/Users/username/.local/pipx/venvs/slack-export-viewer/lib/python3.11/site-packages/slackviewer/templates/viewer.html", line 64, in top-level template code
    {{render_message(message, None, no_external_references)}}
  File "/Users/username/.local/pipx/venvs/slack-export-viewer/lib/python3.11/site-packages/jinja2/runtime.py", line 777, in _invoke
    rv = self._func(*arguments)
         ^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/username/.local/pipx/venvs/slack-export-viewer/lib/python3.11/site-packages/slackviewer/templates/util.html", line 16, in template
    <div id="{{ message.id }}">
  File "/Users/username/.local/pipx/venvs/slack-export-viewer/lib/python3.11/site-packages/jinja2/environment.py", line 485, in getattr
    return getattr(obj, attribute)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/username/.local/pipx/venvs/slack-export-viewer/lib/python3.11/site-packages/slackviewer/message.py", line 108, in id
    return self.time
           ^^^^^^^^^
  File "/Users/username/.local/pipx/venvs/slack-export-viewer/lib/python3.11/site-packages/slackviewer/message.py", line 51, in time
    tsepoch = float(self._message["ts"].split(".")[0])
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
KeyError: 'ts'

Tried 24-hour export and still the same issue.

Thanks

user68329 commented 1 year ago

Debug mode adds following to the end:

127.0.0.1 - - [05/May/2023 08:20:00] "GET /?__debugger__=yes&cmd=resource&f=debugger.js HTTP/1.1" 200 -
INFO:werkzeug:127.0.0.1 - - [05/May/2023 08:20:00] "GET /?__debugger__=yes&cmd=resource&f=debugger.js HTTP/1.1" 200 -
127.0.0.1 - - [05/May/2023 08:20:00] "GET /?__debugger__=yes&cmd=resource&f=style.css HTTP/1.1" 200 -
INFO:werkzeug:127.0.0.1 - - [05/May/2023 08:20:00] "GET /?__debugger__=yes&cmd=resource&f=style.css HTTP/1.1" 200 -
127.0.0.1 - - [05/May/2023 08:20:00] "GET /?__debugger__=yes&cmd=resource&f=style.css HTTP/1.1" 304 -
INFO:werkzeug:127.0.0.1 - - [05/May/2023 08:20:00] "GET /?__debugger__=yes&cmd=resource&f=style.css HTTP/1.1" 304 -
127.0.0.1 - - [05/May/2023 08:20:00] "GET /?__debugger__=yes&cmd=resource&f=debugger.js HTTP/1.1" 304 -
INFO:werkzeug:127.0.0.1 - - [05/May/2023 08:20:00] "GET /?__debugger__=yes&cmd=resource&f=debugger.js HTTP/1.1" 304 -
127.0.0.1 - - [05/May/2023 08:20:00] "GET /?__debugger__=yes&cmd=resource&f=console.png HTTP/1.1" 200 -
INFO:werkzeug:127.0.0.1 - - [05/May/2023 08:20:00] "GET /?__debugger__=yes&cmd=resource&f=console.png HTTP/1.1" 200 -
127.0.0.1 - - [05/May/2023 08:20:00] "GET /?__debugger__=yes&cmd=resource&f=console.png HTTP/1.1" 304 -
INFO:werkzeug:127.0.0.1 - - [05/May/2023 08:20:00] "GET /?__debugger__=yes&cmd=resource&f=console.png HTTP/1.1" 304 -
127.0.0.1 - - [05/May/2023 08:20:00] "GET /?__debugger__=yes&cmd=resource&f=console.png HTTP/1.1" 304 -
INFO:werkzeug:127.0.0.1 - - [05/May/2023 08:20:00] "GET /?__debugger__=yes&cmd=resource&f=console.png HTTP/1.1" 304 -
mistershah commented 1 year ago

Just started having the same issue, not sure what is causing the problem. Happening on fresh installation of Python and slack-export-viewer as well.

eye-doctor-chris commented 1 year ago

@user68329 Try two things:

  1. Click on Viewing from "/Users/username/Documents/Downloads/x..." and open the location to make sure you have given permission to access the folder to VS Code.
  2. Try running from another port. Port 5000 or 5001 may be in use, for example, Airplay uses port 5000. slack-export-viewer --port 3000 -z /path/to/export/zip
mistershah commented 1 year ago

Neither of these worked for me. To make it work, I have had success removing a lot of channels from the exported zip file from Slack, re-zipping it, then loading Slack Export Viewer. I have not determined a pattern as to how many channels works properly. Also, could be an issue with amount of lines of conversation in any given channel, or perhaps character issue/limit?

MuratDoganer commented 1 year ago

The clue is in KeyError: 'ts'

The issue is happening because it was expecting to find a time stamp in a message, but was unable to find it. Slack uses time stamps for Threads, so its likely caused by that.

The code in question is: tsepoch = float(self._message["ts"].split(".")[0])

Possible Solution:

if "ts" in self._message:
    tsepoch = float(self._message["ts"].split(".")[0])
else:
    tsepoch = None  # Or any suitable default value
MuratDoganer commented 1 year ago

I fixed it by replacing the code at line 48 in message.py with:

@property
def time(self):
    # Check if 'ts' key exists in the dictionary
    if "ts" in self._message:
        # Handle this: "ts": "1456427378.000002"
        tsepoch = float(self._message["ts"].split(".")[0])
        return str(datetime.datetime.fromtimestamp(tsepoch)).split('.')[0]
    else:
        return None  # or return a suitable default value

this works as a workaround for now

user68329 commented 1 year ago

Thanks to @MuratDoganer, I may have a theory on what's causing this. We have retention settings(message history to disappear after certain number of days) set on certain channels and all DMs. It's possible that because of this, the structure of export doesn't meet SlackViewer's expectations and yes, the possible solution above may fix the issue.

We also use Slack Connect with external workspaces and I've seen internal messages disappear because of retention settings, but external message stay within the same channel. This could be the culprit.

MuratDoganer commented 1 year ago

I see, it could be the cause! We use the default message retention values from Slack but was getting this issue on every export, decided to sit down and fix it today :)

I would put in a PR but seems like I cant at the moment