search

CredentialEngine / ai-course-crawler

Apache License 2.0
1 stars 0 forks source link

Complete Extraction for Michigan Virtual with no course file #29

Closed rvilsack closed 2 months ago

rvilsack commented 3 months ago

URL tested: https://michiganvirtual.org/courses/professional/ Crawler extraction link: https://master.ai-course-crawler.development.c66.me/extractions/10

Extraction finished but no courses were provided: image

rsaksida commented 2 months ago

The problem here is in the recipe. The model is getting confused when detecting pagination.

I'll add some additional checks to prevent this type of error, but #40 should help as well.