Generating source lists is too slow

ndfred / unifi-pi-hole

A Pi-hole equivalent for the Unifi Security Gateway

31 stars 1 forks source link

Generating source lists is too slow #14

Open ndfred opened 5 years ago

ndfred commented 5 years ago

remove_duplicate_domains is very inefficient and causes CI to time out (see 1b2a725d0c4690be4708164fd7ffa38b3385f97d). Figure out a faster way to match sub-domains of an existing domain.

ndfred commented 5 years ago

Might be worth using the tldextract package to split hostnames and then match the domain + extension part to a set