search

Transparency-Consent-Framework-Research / consent-crawler

Crawlee/Playwright based crawler for collecting collecting TCF signals from websites
MIT License
2 stars 1 forks source link

CombinedPropertyError in make_target_list #8

Open MCristalli opened 3 months ago

MCristalli commented 3 months ago

In main.js when I try to run the crawler on the startUrls I get a CombinedPropertyError.

await crawler.run(startUrls);

After some reading into your code I found that changing line 8 of /src/util/target_list.ts from

    return `http://${domain}`;

to 

    return `http://www.${domain}`;

fixes this error. Apparently the www. is needed in front of the domain name. I encountered this error on a fresh Ubuntu install following your README.

Best regards Maja Cristalli

antoniojtorres commented 3 months ago

@MCristalli Thank you for pointing this out, good catch!