How to load other modules?

[ktpktr0]

I use docker to install elastalert. I try to load the new module, but it fails

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/elastalert", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.10/site-packages/elastalert/elastalert.py", line 2155, in main
    client = ElastAlerter(args)
  File "/usr/local/lib/python3.10/site-packages/elastalert/elastalert.py", line 128, in __init__
    self.rules = self.rules_loader.load(self.conf, self.args)
  File "/usr/local/lib/python3.10/site-packages/elastalert/loaders.py", line 170, in load
    raise EAException('Error loading file %s: %s' % (rule_file, e))
elastalert.util.EAException: Error loading file /home/elastalert/rules/example_frequency.yaml: Error initiating alert ['elastalert_modules.dingtalk_alert.DingTalkAlerter']: Could not import module elastalert_modules.dingtalk_alert.DingTalkAlerter: No module named 'elastalert_modules'

load modules:

# ll /data/elastalert/elastalert_modules/
dingtalk_alert.py  __init__.py 

rules:

alert_text_type: alert_text_only

alert_text_args:
  - host
  - method
  - request
  - status
  - upstream
  - num_hits

alert:
  - "elastalert_modules.dingtalk_alert.DingTalkAlerter"

[nsano-rururu]

yelp / elastalert does not work with python 3.9 and 3.10. Ask the creator about dingtalk_alert.py.

[nsano-rururu]

jertel / elastalert2 is compatible with python 3.9 and 3.10. dingtalk is also supported. https://github.com/jertel/elastalert2

[ktpktr0]

Thank you for your help. I have solved the problem. In addition, I want to know how elasalert uses certificates to connect to es. I tried the following methods, but it doesn't work.

To generate a certificate, I use the existing p12 certificate (This certificate is used to generate the node certificate of the ES cluster)

echo "" | /usr/share/elasticsearch/bin/elasticsearch-certutil cert --silent --pem /etc/elasticsearch/ssl/elastic-stack-ca.p12 --ip 192.168.1.165 --dns es-master2 --out /tmp/1.zip

elastalert.yaml:

#Optional, select whether to connect es with SSL, true or false
use_ ssl: True

#Optional, whether to verify the TLS certificate. Set it to true or false. The default is - true
#verify_ certs: False

#Select certificate
ca_ certs: /home/elastalert/ssl/ca.crt
client_ crt: /home/elastalert/ssl/es-master2.crt
client_ key: /home/elastalert/ssl/es-master2.key

error log:

Traceback (most recent call last):
  File "/usr/local/bin/elastalert-create-index", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.10/site-packages/elastalert/create_index.py", line 278, in main
    create_index_mappings(es_client=es, ea_index=index, recreate=args.recreate, old_ea_index=old_index)
  File "/usr/local/lib/python3.10/site-packages/elastalert/create_index.py", line 23, in create_index_mappings
    esinfo = es_client.info()['version']
  File "/usr/local/lib/python3.10/site-packages/elasticsearch/client/utils.py", line 84, in _wrapped
    return func(*args, params=params, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/elasticsearch/client/__init__.py", line 259, in info
    return self.transport.perform_request("GET", "/", params=params)
  File "/usr/local/lib/python3.10/site-packages/elasticsearch/transport.py", line 318, in perform_request
    status, headers_response, data = connection.perform_request(method, url, params, body, headers=headers, ignore=ignore, timeout=timeout)
  File "/usr/local/lib/python3.10/site-packages/elasticsearch/connection/http_requests.py", line 83, in perform_request
    raise SSLError('N/A', str(e), e)
elasticsearch.exceptions.SSLError: ConnectionError(HTTPSConnectionPool(host='192.168.1.165', port=9200): Max retries exceeded with url: / (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:997)')))) caused by: SSLError(HTTPSConnectionPool(host='192.168.1.165', port=9200): Max retries exceeded with url: / (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:997)'))))
``

[nsano-rururu]

I think it will work if the ssl certificate related files are created correctly and set properly. This is the setting I tried with ssl's self-signed certificate in the past.

# The elasticsearch hostname for metadata writeback
# Note that every rule can have its own elasticsearch host
es_host: es01

# The elasticsearch port
es_port: 9200

# Connect with TLS to elasticsearch
use_ssl: True

# Verify TLS certificates
#verify_certs: True

# Option basic-auth username and password for elasticsearch
es_username: elastic
es_password: xxxxxxxxx

# ssl
ca_certs: /usr/share/elasticsearch/config/certificates/ca/ca.crt
client_cert: /usr/share/elasticsearch/config/certificates/es01/es01.crt
client_key: /usr/share/elasticsearch/config/certificates/es01/es01.key

[ktpktr0]

My es cluster node uses the following command to generate an xpack certificate:

echo "" | /usr/share/elasticsearch/bin/elasticsearch-certutil cert --ca elastic-stack-ca.p12 --ip 192.168.1.165 --dns es-master2 --out es-master2.p12 --pass ""

However, generating the certificate as at the beginning does not seem to work properly. I don't know how elasalert needs to generate a certificate

[nsano-rururu]

This is what I have confirmed in the past by generating a self-signed certificate with elastalert-server. Please refer only to the certificate creation part. https://github.com/johnsusek/praeco/issues/345#issuecomment-822827568

[nsano-rururu]

There is no more information I can provide.

[nsano-rururu]

If the above answers do not help, ask in the discussion of elastalert2. https://github.com/jertel/elastalert2/discussions

[ktpktr0]

I've tried, but it didn't work

[ktpktr0]

Through research, I solved the problem of loading modules. Thank you for your help