search

Yelp / elastalert

Easy & Flexible Alerting With ElasticSearch
https://elastalert.readthedocs.org
Apache License 2.0
7.99k stars 1.74k forks source link

Stomp alerter code (stomp.py) slowing elastalert startup time significantly #1791

Open beezz opened 6 years ago

beezz commented 6 years ago

Resolution of all local hostnames in stomp.py utils causes slow elastalert start.

With stomp alerter:

» time python -c "from elastalert import elastalert"
python -c "from elastalert import elastalert"  0.42s user 0.06s system 1% cpu 40.895 total

Without stomp alerter:

» time python -c "from elastalert import elastalert"
python -c "from elastalert import elastalert"  0.32s user 0.05s system 98% cpu 0.382 total

Solution might be to do late import on stomp alerter class instatiation. Let me know what do you think and I can prepare PR. For now I'll have fork with Stomp alerter and stomp.py as dependency removed.

Qmando commented 6 years ago

Sounds like you've got a bad DNS configuration. I'm guessing you've got a large number of DNS search domains or one of the resolvers is unreachable.

If you attempt to resolve other, non-existent domains, does it always return immediately?

I do kind of want to move to optional imports, so I'd be will to accept a PR that does that.

beezz commented 6 years ago

@Qmando Indeed the extremity (40s) was my DNS config. Anyway, doing blocking calls to like stomp.py is doing during import time does not sound like a good idea. Especially when running in lambda where every millisecond counts.

Qmando commented 6 years ago

Yeah, I agree it's a bad thing to do at import time. Doesn't look like this will get fixed upstream either. https://github.com/jasonrbriggs/stomp.py/issues/186

As I said, I'd be happy to move the imports around, so that they are only used if the alerter is being used, like you suggested.