Can't load new exports, but can load old ones. ValueError: Extra data:

shanness commented 6 years ago

ValueError: Extra data: line 33 column 2 - line 60 column 2 (char 1257 - 1952)


  File "/usr/local/bin/slack-export-viewer", line 11, in <module>
    load_entry_point('slack-export-viewer==0.4.2', 'console_scripts', 'slack-export-viewer')()
  File "/home/craig/.local/lib/python2.7/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/home/craig/.local/lib/python2.7/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/home/craig/.local/lib/python2.7/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/craig/.local/lib/python2.7/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/slackviewer/main.py", line 76, in main
    configure_app(app, archive, debug)
  File "/usr/local/lib/python2.7/dist-packages/slackviewer/main.py", line 40, in configure_app
    channels = compile_channels(path, user_data, channel_data)
  File "/usr/local/lib/python2.7/dist-packages/slackviewer/archive.py", line 49, in compile_channels
    day_messages = json.load(f)
  File "/usr/lib/python2.7/json/__init__.py", line 291, in load
    **kw)
  File "/usr/lib/python2.7/json/__init__.py", line 339, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python2.7/json/decoder.py", line 367, in decode
    raise ValueError(errmsg("Extra data", s, end, len(s)))
ValueError: Extra data: line 33 column 2 - line 60 column 2 (char 1257 - 1952)```

shanness commented 6 years ago

I changed the code in compile_channels to catch the error

                    day_messages = json.load(f)
                except ValueError as v:
                    print "Error ",v," with data ",f

Error Extra data: line 33 column 2 - line 60 column 2 (char 1257 - 1952) with data <_io.TextIOWrapper name=u'/tmp/_slackviewer/fabe5eeee0ff9d65446eece86d25dec91615210f/team-site/2017-01-10.json' encoding='utf8'>

And found that json file has two arrays in it. My except block allows me to now load the export.


[
    {
        "user": "U06TFPM3K",
        "text": "<@U06TFPM3K> has joined the channel",
        "type": "message",
        "subtype": "channel_join",
        "ts": "1484049182.000002"
    },
    {
        "user": "U193Y8EH1",
        "inviter": "U06TFPM3K",
        "text": "<@U193Y8EH1> has joined the channel",
        "type": "message",
        "subtype": "channel_join",
        "ts": "1484049182.000003"
    },
    {
        "user": "U3PGV4H3N",
        "inviter": "U06TFPM3K",
        "text": "<@U3PGV4H3N> has joined the channel",
        "type": "message",
        "subtype": "channel_join",
        "ts": "1484049183.000004"
    },
    {
        "user": "U06TFPM3K",
        "topic": "Pingpad can help you capture shared knowledge, organize and act on it. I will save Posts and Pinned messages as wiki pages automatically for you to <https:\/\/pingpad.io>. Say `help` to get started.",
        "text": "<@U06TFPM3K> set the channel topic: Pingpad can help you capture shared knowledge, organize and act on it. I will save Posts and Pinned messages as wiki pages automatically for you to <https:\/\/pingpad.io>. Say `help` to get
 started.",
        "type": "message",
        "subtype": "channel_topic",
        "ts": "1484049183.000005"
    }
][
    {
        "user": "U06TFPM3K",
        "text": "<@U06TFPM3K> has joined the channel",
        "type": "message",
        "subtype": "channel_join",
        "ts": "1484049182.000002"
    },
    {
        "user": "U193Y8EH1",
        "inviter": "U06TFPM3K",
        "text": "<@U193Y8EH1> has joined the channel",
        "type": "message",
        "subtype": "channel_join",
        "ts": "1484049182.000003"
    },
    {
        "user": "U06TFPM3K",
        "members": [
            "U06TFPM3K",
            "U193Y8EH1"
        ],
        "text": "<@U06TFPM3K> archived the channel",
        "type": "message",
        "subtype": "channel_archive",
        "ts": "1484049183.000004"
    }
]

tscritch commented 6 years ago

Hey @shanness, I'm currently refactoring this section of the code. I'll take a look at this as well. Thanks!

shanness commented 6 years ago

Ta. Worked perfectly! I hacked to to filter only messages with a certain annotation, and only include channels with filtered messages, which was perfect for some reporting we had to do. Thanks for the great codebase, easy to work with!

hfaran commented 6 years ago

This looks like the same bug that I fixed in #63. @shanness please try the new release 0.6.0 and see if it works.

hfaran / slack-export-viewer

Can't load new exports, but can load old ones. ValueError: Extra data: #59