Closed connecteev closed 10 months ago
Ideally the json file with work ONLY with the course links - the course names aren't really needed. Another file format option (instead of a json file) could be a plain text file, with each line representing a course URL.
Basically option 'Download from a file' means I have the links and etc so don't hit API anymore, but it is misleading as I can see, I am thinking to remove it in order not to confuse user?
Ah I see, yes - that's confusing! I actually think an option to download ONLY the courses that I want from a file that I feed ccdown would be a nice touch! Think of this use case:
Step 1: I ask the tool to get me a text list of all every course available (maybe this can be filtered by courses within a subject only, maybe this list can be sorted by newest / latest course)
Step 2: I manually create a new text file for ONLY the courses I want to download, and feed it to ccdown
Step 3: ccdown has an option to prompt for a file, and ccdown only downloads the course videos (and code, if available) for all courses on that list.
This use-case actually brings everything (#4, #5, #6) together and makes the experience all the better (huge time savings).
In the future I could run step 1 again to get a fresh list of courses, compare with my old list, create a new list with only the new courses I want to download (step 2), and then feed it to ccdown (step 3). It would make the workflow amazing.
Thanks again for your work on this!
Hey @connecteev, regarding your points:
Step 1: I ask the tool to get me a text list of all every course available (maybe this can be filtered by courses within a subject only, maybe this list can be sorted by newest / latest course) This is already done with search-courses.json file, which is either updated by me or the user if he chooses so.
As I understand, the main point here is that I allow multiple/single course(es) download(s), maybe I can add some kind of 'list' for desired courses to be downloaded
I pushed new version as well in order to reduce the noise of logging
Thanks @muhamed-didovic, yes - a 'list' for desired courses to be downloaded would be great.
Btw I pulled the latest version and it gives me this error now:
Also tried uninstalling and re-installing, and see the same problem.
ah, I see, pushed new version, please verify
I'm still seeing the error
Actually it's a different one now after a fresh installation
I pushed some fixes please pull new version and verify
Thanks @muhamed-didovic Still seeing the same error...but there is a workaround. It looks like the error happens because it cannot find the json/search-courses.json file.
~/Code/web/_course_downloaders
$ ccdown
node:internal/modules/cjs/loader:1070
const err = new Error(message);
^
Error: Cannot find module '/Users/kp/Code/web/_course_downloaders/json/search-courses.json'
Require stack:
- /opt/homebrew/lib/node_modules/ccdown/lib/helpers/search.cjs
at Module._resolveFilename (node:internal/modules/cjs/loader:1070:15)
at Module._load (node:internal/modules/cjs/loader:923:27)
at Module.require (node:internal/modules/cjs/loader:1137:19)
at require (node:internal/modules/helpers:121:18)
at Object.<anonymous> (/opt/homebrew/lib/node_modules/ccdown/lib/helpers/search.cjs:6:16)
at Module._compile (node:internal/modules/cjs/loader:1255:14)
at Module._extensions..js (node:internal/modules/cjs/loader:1309:10)
at Module.load (node:internal/modules/cjs/loader:1113:32)
at Module._load (node:internal/modules/cjs/loader:960:12)
at ModuleWrap.<anonymous> (node:internal/modules/esm/translators:165:29)
at ModuleJob.run (node:internal/modules/esm/module_job:192:25)
at async DefaultModuleLoader.import (node:internal/modules/esm/loader:246:24)
at async loadESM (node:internal/process/esm_loader:40:7)
at async handleMainPromise (node:internal/modules/run_main:66:12) {
code: 'MODULE_NOT_FOUND',
requireStack: [ '/opt/homebrew/lib/node_modules/ccdown/lib/helpers/search.cjs' ]
}
Node.js v20.3.0
If I create the json file manually from https://github.com/muhamed-didovic/ccdown/blob/main/json/search-courses.json and put it at json/search-courses.json it runs fine. It's not a big deal but just want to point out that previously the tool did not need the json file to be within the current directory.
2 questions for you:
Just to confirm which version are you running with command: ccdown -v I think I know what is the issue
ccdown -v 0.19.4
ok, thanks.
So to answer you questions:
@muhamed-didovic thanks for clarifying, and sounds amazing, thank you!
On the list functionality, just one thought...I expect to have to download all courses under a topic, like laravel has 187. https://codecourse.com/subjects/laravel/index So, it would probably be best to allow a file to be ingested (versus a UI to select which courses to download..which would make it an unusable experience). Thanks for working on this feature and this great tool.
For example, would be good to get this option to work with this attached file containing 2 courses: search-courses.json.txt
$ ccdown ✔ Do you want all courses? … yes ✔ Do you want download from a file … yes ✔ Enter a file path eg: /opt/homebrew/lib/node_modules/ccdown/json/*.json › search-courses.json ✔ Enter email … valid_email@gmail.com ✔ Enter password … ***** ✔ Enter a directory to save a file (eg: /Users/kp/Code/web/_course_downloaders) … /Users/kp/Code/web/_course_downloaders/videos ✔ Login... completed. MAIN errorr in cli.js: ReferenceError: require is not defined at all (file:///opt/homebrew/lib/node_modules/ccdown/lib/index.js:25:19)
Here is the attached search-courses.json file I used (containing 2 courses): search-courses.json.txt
@connecteev pull new version, I think now it should be ok, also I am almost done with 'list' feature
also pushed autocomplete-multi select for courses :)
@muhamed-didovic Thanks for the quick turnaround!
I upgraded ccdown:
$ npm upgrade -g ccdown
changed 1 package in 3s
233 packages are looking for funding
run `npm fund` for details
$ ccdown -v
0.19.7
This option works okay: $ ccdown ✔ Do you want all courses? … yes ✔ Do you want download from a file … no
But this option errors out: $ ccdown ✔ Do you want all courses? … yes ✔ Do you want download from a file … yes ✔ Enter a file path eg: /opt/homebrew/lib/node_modules/ccdown/json/*.json › my_courses.json ✔ Enter email … valid_email@gmail.com ✔ Enter password … ***** ✔ Enter a directory to save a file (eg: /Users/kp/Code/web/_course_downloaders) … /Users/kp/Code/web/_course_downloaders/videos ✔ Login... completed. MAIN errorr in cli.js: ReferenceError: require is not defined at all (file:///opt/homebrew/lib/node_modules/ccdown/lib/index.js:25:19)
Sorry, forgot to publish new version, which is: 0.20.0 If you want to see new multi-select, so when you are asked: 'Do you want all courses?' choose 'no'
@muhamed-didovic Just tried ccdown version 0.20.0, and multi-select seems to work....
ccdown ✔ Do you want all courses? … no ✔ Choose "Y" if you want to search for a course otherwise choose "N" if you have a link for download … yes ✔ Do you want to search for a courses from a local file (which is faster) … no ? Search for a course › - Space to select. Return to submit Instructions: ↑/↓: Highlight option ←/→/[space]: Toggle selection [a,b,c]/delete: Filter choices enter/return: Complete answer
Filtered results for: Enter something to filter
◯ PHP Security ◯ Cloud Image Upload with Laravel ◯ Flexible flash notifications in Laravel ◯ Realtime chat with Pusher ◯ Database Models with PDO ◯ Social Network ◯ Realtime notifications with Pusher ◉ Send Email With PHPMailer ◉ The SOLID Design Principals ◯ The PHP Reflection API
@muhamed-didovic
While the above UI works, my problem is this: I expect to have to download many (but probably not all) courses under a subject ("laravel" has 187: https://codecourse.com/subjects/laravel/index )
For example, if I have already downloaded, say 20 courses under "laravel", I may want a specific list of courses (say 50 under the subject "laravel", 5 under the subject "vue"). Multi-select makes it very tedious to find and select dozens of courses - it's not very practical.
Can we allow a file to be ingested (versus a UI to select which courses to download..which currently makes it an unusable experience)?
Ideally I could give ccdown a file with (ex: 127) courses, similar to the sample I have attached (easiest would be one line per course URL, though a Json format would also work fine) my_courses_preferred_format.txt
hey @connecteev, made something for you regarding the download over subjects, here is the example: ✔ Do you want all courses? … no ✔ Choose "Y" if you want to search for a course otherwise choose "N" if you have a link for download … no ✔ Enter url for download. … https://codecourse.com/subjects/flutter
@muhamed-didovic thanks! this works great and is going to save a ton of time. I closed out https://github.com/muhamed-didovic/ccdown/issues/4
@muhamed-didovic Any chance you can take a look at this? Thanks for this useful package!
When I select "yes" to the first 2 options ("Do you want all courses?" and "Do you want download from a file"), the script immediately ends with "Downloaded all videos for 'courses' api! ", but doesn't do anything.
Logs:
✔ Do you want all courses? … yes ✔ Do you want download from a file … yes found some files: 4 in folder: /opt/homebrew/lib/node_modules/ccdown/json ✔ Enter a file path eg: /opt/homebrew/lib/node_modules/ccdown/json/*.json › search-courses.json ✔ Enter email … valid_email@gmail.com ✔ Enter password … **** ✔ Enter a directory to save a file (eg: /Users/kp/Code/web/_course_downloaders) … /Users/kp/Code/web/_course_downloaders/videos ✔ Login... completed. ✔ Downloaded all videos for 'courses' api! (total: 17)
Attached is the nuxt.json file (containing all codecourse courses in the "nuxt.js" category: https://codecourse.com/subjects/nuxt-js/index . I placed the file (attached) at /opt/homebrew/lib/node_modules/ccdown/json/nuxt.json: (Note that I have renamed the file to .txt because github won't let me upload json files)
Attached: nuxt.json.txt