Open HibernantBear opened 3 months ago
@HibernantBear The playground works well because it uses a mix of services and scraping methods, not just one. This setup helps bypass blockers on websites effectively. For your self-hosted setup, using the Firecrawl API key might help, as it includes access to these multiple services.
You can check out the getScrapingFallbackOrder
function in single_url.ts
under apps/api/src/scraper/WebScraper
for more details on how this works.
will, I found out this is because fallback to fetch, if playwright
is available, this should acts alright
this is because of the auth check block available playwright service that should be used
modify the api's relate ts file will let playwright works
Describe the Bug Self Host Service Certain Web Page Scrape Return Wrong Encoding Result on Self Host Service, and Official Online Demo is Totally Fine
To Reproduce Steps to reproduce the issue:
Expected Behavior should be response like the jsonn below, and the weird thing is online test is totally fine, only self host meet this problem