[webcrawler-source] Expose the allow-non-html-contents parameter

LangStream / langstream

LangStream. Event-Driven Developer Platform for Building and Running LLM AI Apps. Powered by Kubernetes and Kafka.

https://langstream.ai

Apache License 2.0

378 stars 28 forks source link

[webcrawler-source] Expose the allow-non-html-contents parameter #740

Closed eolivelli closed 7 months ago

eolivelli commented 7 months ago

Summary:

allow users to configure "allow-non-html-contents"

eolivelli commented 7 months ago

I am not adding the property in the examples because otherwise people with older versions of LangStream won't be able to run the examples. Usually we wait for a couple of releases