Currently the crawler does not handle TXT records properly, breaking SPF record entries in cases where the TXT record is split in several parts (e.g. 0.spf0.hubspotemail.net). The crawler currently also handles other TXT records even if they don't start with v=spf1 which is against the specification.
This PR superseeds #19 as we currently use the modified version in production. This modified version also introduces some code cleanups suggested by @tresni in his fork. Our modified version might introduce additional improvements in the future.
Currently the crawler does not handle TXT records properly, breaking SPF record entries in cases where the TXT record is split in several parts (e.g.
0.spf0.hubspotemail.net). The crawler currently also handles other TXT records even if they don't start withv=spf1which is against the specification.This PR superseeds #19 as we currently use the modified version in production. This modified version also introduces some code cleanups suggested by @tresni in his fork. Our modified version might introduce additional improvements in the future.