Closed esensar closed 2 weeks ago
Hi @esensar, thanks for suggesting a fix.
since this might be too much to spawn a thread for each lookup.
This might be problematic, could we have a pool or a dedicated thread for this?
@esensar just a heads up that we'll be releasing Vector next week in-case you want to try to fix this before then.
I have implemented just a single dedicated thread for this. I know that is probably not the best idea, but this close to the release, I didn't want to risk breaking stuff by adding a thread pool (which would probably include a crate, since it would be better to rely on something existing already).
I'd like to get @pront's review again too.
I'd like to get @pront's review again too.
This is significantly improved, thanks @esensar. I had a question about the channel capacity which I believe is important to resolve before we merge this.
When testing
dns_lookup
invector
in a real environment, we got this error:Which happens because VRL is called in a tokio managed thread. The solution here was to move the call to another thread, because internally
domain
crate implement blocking calls by still relyting onasync
function implementation.Let me know if you have a better solution for this, since this might be too much to spawn a thread for each lookup.