search

dlt-hub / verified-sources

Contribute to dlt verified sources 🔥
https://dlthub.com/docs/walkthroughs/add-a-verified-source
Apache License 2.0
52 stars 40 forks source link

CI: Scraping source idles indefinitely #394

Open sultaniman opened 4 months ago

sultaniman commented 4 months ago

Scraping Scraping source if run by CI hangs indefinitely while attempts to reproduce it on local environment didn't succeed this needs proper investigation and resolution.

To Reproduce

  1. remove pytest.mark.skip from scraping tests,
  2. push to separate PR,
  3. observe idling scraping source once scraping ends.

Expected behavior

CI tests should pass without hanging and should pass for all scraping test suite.

Running environment