search

mikwielgus / forum-dl

Scrape posts, threads from forums, news aggregators, mail archives, export to JSONL, mailbox, WARC
MIT License
68 stars 2 forks source link

WARNING:root:Waiting X seconds. #1

Closed peterjschroeder closed 1 year ago

peterjschroeder commented 1 year ago

Every forum I've tried shoots out this warning and just stay there accumulating seconds, never going any further.

mikwielgus commented 1 year ago

Hi, thanks for testing.

This was due to the connection retry code being wrongly called during forum software detection. Should be fixed by ee544ad4074faf1309fe2c62bbc3b911fe4f9721.

If you still have problems (likely, given that this is early alpha software), please give me a list of links you tried to run Forum-dl on so that I can debug.

peterjschroeder commented 1 year ago

Thank you, that got it going further.

Now it stops on: msg["From"] = post.data["author"] KeyError: author

These are the forums I am testing it on: https://www.dragonsfoot.org/forums/ phpbb https://www.thepiazza.org.uk/bb phpbb https://www.teamcanadaonline.net/ Simple Machines http://forum.theonyxpath.com/ Vbulletin

mikwielgus commented 1 year ago

Fixed this in 3713cd19c21d0dd7b1c7ed377b02d50e14b71d31 just now.

peterjschroeder commented 1 year ago

Excellent, thank you.