Ability to download specific posts (either with a provided link or JSON file)

[NoExplorer]

This would be useful for people who want to bulk download many posts (like the saved posts they have) and not an entire sub-reddit or user. I could try to write a few lines of code but seeing that I am inexperienced, that code would be full of vulnerabilities. Still, I'd like to view this idea being implemented, if this is something easy to do that is, and if it is a thing that the Public Reddit API permits.

(Couldn't add a descriptor tag for the issue. This is an idea and not a bug.)

[josephrcox]

Good idea @NoExplorer!

This doesn't sound too difficult, only a couple of hours of code.

I can take a look today and let you know if it's a bigger lift than I think.

[NoExplorer]

Thanks a lot! Sounds good to me.

[josephrcox]

@NoExplorer When you manually download posts, do you prefer that they are still organized into subreddit folders, or should they be in another folder like "Manually entered posts"?

[NoExplorer]

I prefer that they are organized into folders based on the sub-reddit they were posted on. Although I see the option of having them on a folder like you suggested helpful for others if the posts are saved with the sub-reddit they were posted on in the filename (i.e. MEDIA_r_pics-[post-title-here]) if that is something doable. That could be an issue with people who have custom naming schemes. Personally I would go with the sub-reddit based organizing scheme.

[josephrcox]

Thanks!

@NoExplorer I just completed a basic iteration. If you would like to give it a shot while it's still in a testing stage, feel free to clone the branch here https://github.com/josephrcox/easy-reddit-downloader/tree/experimental

To download a list of posts:

1. Enter your post list in download_post_list.txt in the root directory of the project. Make sure to follow the format as shown in the first few comment lines of the file.
2. Go into user_config.json and set download_post_list_options > enabled: true.
3. Run the application as normal.

You may also choose to download these specific posts at a set interval using the repeatForever boolean and timeBetweenRuns integer in the download_post_list_options object in user_config.json. Make sure to enter the timeBetweenRuns in milliseconds.

If you test and it works well to you, I will merge it with the main branch so others can use it as well! 🎉

[NoExplorer]

I have cloned the repo on my computer, but in the user_config file there is no download_post_list_options option (Well, the file is called user_config_DEFAULT so I assume I should add it myself).

I did edit that file and rename it, but the app prompts for me to type a sub-reddit name or user. I might be doing something wrong here

[josephrcox]

Delete the user_config.json and re run the program. It will make a new one for you with the new stuff

[josephrcox]

When you run the program for the first time, your own user_config.json is created by duplicating the default one. This is to protect you in case you mess up the user_config.json and want to revert by copying the default

[josephrcox]

This is what your file structure should look like @NoExplorer

[josephrcox]

It is also possible that you didn't download the right thing. I see that you forked the main branch. You can try downloading this directly: https://github.com/josephrcox/easy-reddit-downloader/archive/refs/heads/experimental.zip

[NoExplorer]

I forked both branches I believe. I ran the code and got an error on my end (Running Arch Linux, that could be the issue)

/home/noexplorer/experimental/index.js:170
        !config.download_post_list_options.enabled
                                           ^

TypeError: Cannot read properties of undefined (reading 'enabled')
    at startScript (/home/noexplorer/experimental/index.js:170:38)
    at Request._callback (/home/noexplorer/experimental/index.js:158:5)
    at self.callback (/home/noexplorer/experimental/node_modules/request/request.js:185:22)
    at Request.emit (node:events:513:28)
    at Request.<anonymous> (/home/noexplorer/experimental/node_modules/request/request.js:1154:10)
    at Request.emit (node:events:513:28)
    at IncomingMessage.<anonymous> (/home/noexplorer/experimental/node_modules/request/request.js:1076:12)
    at Object.onceWrapper (node:events:627:28)
    at IncomingMessage.emit (node:events:525:35)
    at endReadableNT (node:internal/streams/readable:1359:12)

Node.js v19.4.0

Ill also test it real quick in Windows.

[josephrcox]

Do you mind sharing your user_config.json file and the default one? Like the contents of both?

[josephrcox]

Should be fine on any OS by the way. No need to try on windows.

[NoExplorer]

Sure

user_config.json

{
    "testingMode": false,
    "testingModeOptions": {
        "subredditList": ["AskReddit","pics"],
        "numberOfPosts": 25,
        "sorting": "new",
        "time": "month",
        "repeatForever": true,
        "timeBetweenRuns": 30000
    },
    "local_logs": true,
    "local_logs_naming_scheme": {
        "showDateAndTime": true,
        "showSubreddits": true,
        "showNumberOfPosts": true
    },
    "file_naming_scheme": {
        "showDateAndTime": true,
        "showAuthor": true,
        "showTitle": true
    },
    "download_self_posts": true,
    "download_media_posts": true,
    "download_link_posts": true,
    "download_comments": true,
    "separate_clean_nsfw": false,
    "redownload_posts": false,
    "detailed_logs": false
}

user_config_DEFAULT.json

{
    "testingMode": false,
    "testingModeOptions": {
        "subredditList": ["AskReddit","pics"],
        "numberOfPosts": 25,
        "sorting": "new",
        "time": "month",
        "repeatForever": true,
        "timeBetweenRuns": 30000
    },
    "local_logs": true,
    "local_logs_naming_scheme": {
        "showDateAndTime": true,
        "showSubreddits": true,
        "showNumberOfPosts": true
    },
    "file_naming_scheme": {
        "showDateAndTime": true,
        "showAuthor": true,
        "showTitle": true
    },
    "download_self_posts": true,
    "download_media_posts": true,
    "download_link_posts": true,
    "download_comments": true,
    "separate_clean_nsfw": false,
    "redownload_posts": false,
    "detailed_logs": false
}

user_config.txt user_config_DEFAULT..txt

These are the JSON files. It's getting late, Ill be back tomorrow (Timezone is EEST)

[josephrcox]

Have a good night!

This is the user_config_DEFAULT.json that you should be using. https://github.com/josephrcox/easy-reddit-downloader/blob/experimental/user_config_DEFAULT.json

Not sure why it didn't clone properly but that one is the one with the changes.

After you download that, delete your user_config.json file, rerun the app, and then you can customize the brand new user_config.json file to suit your needs.

Let me know tomorrow if you still run into problems!

[NoExplorer]

I am back. Sorry for making you wait so long but I got good news!

The feature works as expected! No errors or warnings. All posts I entered were downloaded and the contents of each post are exactly the same as the original. Took a bit to get working (I am a bit stupid, I kept failing to see the entirely new function[Is that what it is called in Javascript?] that enabled the link list download feature. But got it figured out.) but it was all worth it!!

You may merge the experimental and main branch now and close this issue. So happy that I can finally archive the posts I want with so much ease and contribute to something for once!!

Have to also comment, you are very cooperative and super skilled at programming. Wasn't expecting super quick replies and well-detailed help and tips! Also you are very polite and got a lot of patience.

(The post links above are some random ones, nothing important. Also I should apologize for being a little stupid and not super good to work with. Sorry if I was a bit of a burden)

[josephrcox]

AMAZING! Thanks for testing it out, and you are so welcome!

No problem at all, I am glad that it all works for you, and I appreciate the kind words! 😄

Will merge it now and release the next version.