search

bibanon / BASC-Archiver

Python-based Imageboard (4chan) complete thread archiver.
https://pypi.python.org/pypi/BASC-Archiver/
135 stars 18 forks source link

--follow-children needs to check for dupes #17

Open DanielOaks opened 8 years ago

DanielOaks commented 8 years ago

It seems that threads added by --follow-children get duplicated, so if the same thread is linked three times by different threads it'll be added to our list three times in a row and cause issues.

Alternatively, the child thread that kept getting added was a closed thread, so I wonder whether it just kept seeing that the thread was closed, removing it from our list, and then next time it checked for child threads it just readded it because it didn't seem to exist at all in our list.

Will investigate.

DanielOaks commented 8 years ago

This should be fixed with bbdce2b2c338f0ed0dd8ca4dd2f7b9ae917b86df, but needs to be confirmed