Image Previews not working

[Imjustarandomlol32]

It's as the title says, the previews aren't working, the images are downloaded so that's strange

[JacobBathead]

Same issue here, instead of displaying the images they appear like this: i wonder if it is to do with the urls now including query strings on the end, explaned here.

[Imjustarandomlol32]

If that's the case, as long as it doesn't delete the images, it shouldn't be a problem.

[chylex]

It appears that DHT is not able to detect they're images anymore, I will look into that.

However, when Discord starts enforcing the new rules, all files uploaded to Discord will not be accessible by links but only through the Discord client, so the only way you could have proper archival is by enabling automatic downloads of attachments in DHT, and have those files saved in the archive.

I will need to extend automatic downloads to also download embeds, because if you copy a link to an attachment from another server/channel then those links would also stop working. I will also need to update the way DHT processes these new URLs to avoid duplicating downloads in the archive.

I don't have any ETA.

[Imjustarandomlol32]

Alright, I thank you both for informing me

[Kanzuke117]

Discord's changes to make their CDN links expire are supposed to be rolling out as of two days ago, so as a workaround to make sure I have a functional copy, I extracted every cdn.discordapp.com/media.discordapp.net link from text column of the messages table and inserted those URLs back into the attachments and downloads table so the Viewer will essentially treat any message that contains a CDN link as also having the linked file uploaded with it.

I can confirm this works in the viewer, it's showing the images twice, one as just an embed linking directly to the cdn URL, and another as an attachment linking to the local server /get-attachment/ copy.

My solution is jank as hell, but if anyone else doesn't want to wait for an official solution, here's the nodejs script I used to do the majority of the legwork: dhtnode.txt

You can get all the messages you need out of the database with:

SELECT json_object('id',message_id,'time',timestamp,'text',text) AS json FROM messages WHERE text REGEXP '.*cdn\.discordapp\.com.*' OR text REGEXP '.*media\.discordapp\.net.*';

(You will need to wrap the entire text in [] and add a , before every newline to format it properly as a single JSON array for the script to parse it)

It will output two .sql files, which you can then open in your DB browser and execute to insert into the attachments and downloads tables. Then just open the DHT program and you should have new files enqueued in the attachments tab.

Caveats:

• The attachments table uses the attachment ID as primary key, so no duplicates allowed. If a message links to a file that's already in the table because it's been linked before or you archived the message that originally attached it, it can't be attached again to the new message ID. Viewing the new message will still only show the url in the text and won't give you the local server link instead, even though you have the file
• The size of each attachment and download is usually extracted from the original message when tracking, but since this isn't given for every URL and I didn't want to bother requesting the size from discord beforehand, I just set the size of every new download and attachment to the average my existing downloads table had, which was 1600000. It doesn't look like the program every re-inserts or updates this value when downloading the images, so the size totals in the Download Status will be forever incorrect
• I forgot to do any string sanitisation when exporting back from JSON into an SQL query, so I had to manually fix the two URLs I had which actually had a " in the filename
• The script uses import instead of requires because I don't know what I'm doing, so you'll need to write a package.json with "type": "module" for those to work, and probably manually npm install json-bigint

[chylex]

They haven't enforced the new links yet, but in the next DHT update it should work like this:

• All duplicate attachments and downloads are removed, keeping one of the downloaded files over not-yet-downloaded files
• Without downloading, the viewer will display attachments normally until Discord enforces the new links
• Without downloading, and once Discord enforces the new links, attachments will stop working in the viewer entirely because the viewer uses stripped down CDN URLs that Discord will begin rejecting, so you have to download them if you want to see them
• Downloading new attachments should work as expected
• If you already had older attachments and haven't downloaded them, or the downloads failed, click "Retry Failed Downloads" to clear them out, and then re-track the old messages; Discord will provide new attachment links that DHT can start downloading, so it will fill in the missing attachments as long as they haven't been deleted from Discord

I will re-test that all of this works once Discord starts enforcing the new links. I will also look into improving the UI, such as prompting you whether you want to enable downloads, and remembering your decision so you don't have to manually enable downloads every time you open the app. Stuff might break, backup your databases regularly, usual stuff.

For those using the browser-only version, you can try using this Python script https://github.com/chylex/Discord-History-Tracker/blob/master-browser-only/tools/DownloadEmbedsAttachments.py, but it will not handle duplicate attachments and you have to run it manually every time you update your archive. I will not be working on any better solutions for the browser-only version.

[Kanzuke117]

The app just completed the five hour process of migrating my 124GB database to the new version (if possible a progress bar for this process would be nice), but didn't find any new attachments to download. Is extending automatic downloads to embeds and cdn links in message contents still coming?

[chylex]

Interesting, my 10 GB database took only a few minutes. It would be better if the app wasn't frozen during the process, but I don't know how helpful a progress bar would be because the upgrade effectively takes 5 steps so it would be stuck on each step for an hour.

Downloading anything except for attachments is tracked in #200.

EDIT: I might be able to add a reasonable progress bar after all, there is one step that takes 1/4 of the entire time (final commit), but since you already cannot terminate the app during the process or risk leaving the database in an inconsistent state, I can change the one large commit into several smaller commits.