Bug Fixes

[joeminicucci]

Fixed bugs including:

• contacts / Filenames being set to None type
• erroneous backslashes in filepaths
• None type message /contact keys in dictionaries

[carderne]

Thanks @joeminicucci!

1. Would you mind just fixing the comment formatting (space between # and text) and change the '' to "" on line 242 so I don't generate extra changes next time I black format.
2. The open(path, "a") I assume is so that it exits cleanly if the file didn't exist before?

[garvinhicking]

Hey @joeminicucci - thanks for sharing your patch. I was only able to start using the exporter thanks to your commit.

@carderne I did expand further on the commit because there are several more followup-errors based on missing attachments and other "None"-Types.

I assume signal has recently introduced those None-Types for groups and maybe the structure changed. I do not know Python, but can read it, so I patched my file with about zero knowledge (coming from PHP) for proper key checks.

I don't know how I can create a diff based on a proposed pull request, but since my diff is also fairly small, I've attached it here inline:

I'd love to get this merged to your branch. I am VERY thankful for your work on this. Getting a working export for Signal is very crucial to me in using the messenger.

Patch:

--- sigexport_c4e19c98d8.py 2021-01-15 14:58:13.530897651 +0100
+++ sigexport_garvin.py 2021-01-15 15:04:13.995993598 +0100
@@ -45,22 +45,26 @@
         contact_path = dest / name / "media"
         contact_path.mkdir(exist_ok=True, parents=True)
         for msg in messages:
-            attachments = msg["attachments"]
-            if attachments:
-                date = datetime.fromtimestamp(msg["timestamp"] / 1000.0).strftime(
-                    "%Y-%m-%d"
-                )
-                for att in attachments:
-                    try:
-                        file_name = f"{date}_{att['fileName']}"
-                        att["fileName"] = file_name
-                        #account for erroneous backslash in path
-                        att_path=str(att["path"]).replace("\\","/")
-                        shutil.copy2(src_att / att_path, contact_path / file_name)
-                    except KeyError:
-                        print(f"Broken attachment:\t{name}\t{att['fileName']}")
-                    except FileNotFoundError:
-                        print(f"Attachment not found:\t{name}\t{att['fileName']}")
+            try:
+                attachments = msg["attachments"]
+                if attachments:
+                    date = datetime.fromtimestamp(msg["timestamp"] / 1000.0).strftime(
+                        "%Y-%m-%d"
+                    )
+                    for att in attachments:
+                        try:
+                            file_name = f"{date}_{att['fileName']}"
+                            att["fileName"] = file_name
+                            #account for erroneous backslash in path
+                            att_path=str(att["path"]).replace("\\","/")
+                            shutil.copy2(src_att / att_path, contact_path / file_name)
+                        except KeyError:
+                            print(f"Broken attachment:\t{name}\t{att['fileName']}")
+                        except FileNotFoundError:
+                            print(f"Attachment not found:\t{name}\t{att['fileName']}")
+            except KeyError:
+                """Void. No attachments"""
+

     return conversations

@@ -98,7 +102,6 @@
                 body = ""
             body = body.replace("`", "")  # stop md code sections forming
             body += "  "  # so that markdown newlines
-            attachments = msg["attachments"]

             if "type" in msg.keys() and msg["type"] == "outgoing":
                 sender = "Me"
@@ -115,24 +118,29 @@
                     print(f"No sender:\t\t{date}")
                     sender = "No-Sender"

-            for att in attachments:
-                file_name = att["fileName"]
-                #some file names are None
-                if file_name is None:
-                    file_name = "None"
-                path = Path("media") / file_name
-                path = Path(str(path).replace(" ", "%20"))
-                if path.suffix and path.suffix.split(".")[1] in [
-                    "png",
-                    "jpg",
-                    "jpeg",
-                    "gif",
-                    "tif",
-                    "tiff",
-                ]:
-                    body += "!"
-                body += f"[{file_name}](./{path})  "
-            print(f"[{date}] {sender}: {body}", file=mdfile)
+            try:
+                attachments = msg["attachments"]
+                for att in attachments:
+                    file_name = att["fileName"]
+                    #some file names are None
+                    if file_name is None:
+                        file_name = "None"
+                    path = Path("media") / file_name
+                    path = Path(str(path).replace(" ", "%20"))
+                    if path.suffix and path.suffix.split(".")[1] in [
+                        "png",
+                        "jpg",
+                        "jpeg",
+                        "gif",
+                        "tif",
+                        "tiff",
+                    ]:
+                        body += "!"
+                    body += f"[{file_name}](./{path})  "
+                print(f"[{date}] {sender}: {body}", file=mdfile)
+            except KeyError:
+                """Void. No attachments."""
+

 def fetch_data(db_file, key, manual=False):
@@ -187,13 +195,16 @@
         if is_group:
             usable_members = []
             # Match group members from phone number to name
-            for member in result[5].split():
-                c2.execute(
-                    "SELECT name, profileName FROM conversations WHERE id=?", [member]
-                )
-                for name in c2:
-                    usable_members.append(name[0] if name else member)
-            contacts[cid]["members"] = usable_members
+            if result[5] is None:
+                # Void, empty group
+            else:
+                for member in result[5].split():
+                    c2.execute(
+                        "SELECT name, profileName FROM conversations WHERE id=?", [member]
+                    )
+                    for name in c2:
+                        usable_members.append(name[0] if name else member)
+                contacts[cid]["members"] = usable_members

     c.execute("SELECT json, conversationId " "FROM messages " "ORDER BY sent_at")
     for result in c:

[carderne]

@garvinhicking thanks for the patch, will merge @joeminicucci's PR first and then have a look at this.

[joeminicucci]

Thanks @joeminicucci!

1. Would you mind just fixing the comment formatting (space between `#` and text) and change the `''` to `""` on line 242 so I don't generate extra changes next time I `black` format.

2. The `open(path, "a")` I assume is so that it exits cleanly if the file didn't exist before?

Sorry for the late reply @carderne. Would you like me to open another PR?

1. I can do that, you want spaces for the between the comment and text correct? and no problem to changing the single to double quotes
2. I believe that is correct, but I cannot remember if that was the exact root cause, my apologies. Since the a mode ensures append operations if the file exists, perhaps w+ would be more appropriate as it truncates unconditionally and ensures the existence of the file.

[carderne]

@joeminicucci I formatted it already in #3 so don't worry. Appreciate the help making this thing work a bit better!