Closed CloudsProgram closed 1 year ago
Hi @CloudsProgram — turning on flow retries enables task result persistence by default. See https://docs.prefect.io/concepts/results/#toggling-persistence for more details.
It looks like the beautiful soup object cannot be pickled so it's not compatible with our default result serializer. You could work around this by either disabling persistence for the specific task (it should just run again on retry) or changing your return value to something we can cache the result of.
Gotcha, thank you much for the response!
First check
Bug summary
When I use beautiful soup, when this line of code takes place
soup = BeautifulSoup(response.content, 'lxml')
<<(I think this parses the entire html from the page)and when there are retries for the flow, it throws an error.
I tried putting the retries in task, which doesn't throw an error. But I'm just wondering if this is a bug or not. If it is I'm wondering what is causing the error.
Reproduction
Error
Versions
Additional context
No response