ArchiveTeam / grab-site

The archivist's web crawler: WARC output, dashboard for all crawls, dynamic ignore patterns
Other
1.39k stars 135 forks source link

How do you add custom hooks now? #207

Open TheTechRobo opened 2 years ago

TheTechRobo commented 2 years ago

I see they were removed, but I need them (I want wpull's youtube-dl functionality, but more advanced)

TheTechRobo commented 2 years ago

I'll try the method documented in #75

TheTechRobo commented 2 years ago

Update: --python-script no longer seems to exist.

ivan commented 2 years ago

When I removed that (I think because wpull 2.0 changed the APIs too much), the idea was that the user could still make a copy of wpull_hooks.py and then use --wpull-args=--plugin-script=PATH/TO/modified_wpull_hooks.py. I have not tested this.