search

mistio / mist-ce

Mist is an open source, multicloud management platform
https://mist.io
Apache License 2.0
1.87k stars 315 forks source link

Mist sends invalid requests to SMTP mail server #1028

Open davidrylance opened 2 years ago

davidrylance commented 2 years ago

Dear Mist maintainers,

Thank you for Mist. It's great!

I host my own e-mail server (postfix) and I noticed that after I create an account for Mist and configure Mist to use it, the server is literally flooded by requests that generate a lot of warning logs.

I was thinking maybe there is something wrong the mail server hosted on public internet so I installed Postfix into a container on private LAN and configured it with no authentication to relay e-mails to public mail server and I see the same error logs.

Reproduce

Configure Mist e-mailing - in my case postfix on private LAN:

## Configure email settings.
MAILER_SETTINGS = {
    'mail.host': "192.168.1.25",
    'mail.port': "25",
    'mail.tls': False,
    'mail.starttls': False,
    'mail.username': "",
    'mail.password': "",
}
EMAIL_FROM = "existing@account.com"

Or to use mailserver on public internet

MAILER_SETTINGS = {
    'mail.host': "some.mailserver.com",
    'mail.port': "587",
    'mail.tls': True,
    'mail.starttls': True,
    'mail.username': "existing@account.com",
    'mail.password': "password",
}
EMAIL_FROM = "existing@account.com"

Btw. e-mails work - I tested adding rule to mail myself when a VM is stopped / started and I get e-mails. It just something else Mist does that is wrong.

Thank you.

Kind regards,

David

dimgal1 commented 2 years ago

Hi @davidrylance we looked into this and it seems there are several issues leading to the problem.

To alleviate these issues:

services: vminsert: image: victoriametrics/vminsert:v1.60.0-cluster command:

davidrylance commented 2 years ago

Thank you very much @dimgal1 for resolving the problem.

Looking at variables in https://github.com/mistio/mist.api/blob/master/src/mist/api/config.py

Could you please tell if I also need to override other EMAIL_* variables in settings.py?

Like this:

MAILER_SETTINGS = {
    'mail.host': "192.168.1.25",
    'mail.port': "25",
    'mail.tls': False,
    'mail.starttls': False,
    'mail.username': "",
    'mail.password': "",
}
EMAIL_FROM = "existing@account.com"
EMAIL_ALERTS = "existing@account.com"
EMAIL_REPORTS = "existing@account.com"
EMAIL_INFO = "existing@account.com"
EMAIL_SALES = "existing@account.com"
EMAIL_SUPPORT = "existing@account.com"
EMAIL_NOTIFICATIONS = "existing@account.com"
EMAIL_ALERTS_BCC = ""

NOTIFICATION_EMAIL = {
    'all': "existing@account.com",
    'dev': "existing@account.com",
    'ops': "existing@account.com",
    'sales': "existing@account.com",
    'demo': "existing@account.com",
    'support': "existing@account.com",
}

Thank you.

Kind regards,

David

dimgal1 commented 2 years ago

Yes, you can override these variables too. They are used as sender email accounts and depending on your mail server configuration they may not work with the default values

davidrylance commented 2 years ago

Thank you @dimgal1 . I can confirm that NOTIFICATION_EMAIL in settings.py fixed the e-mail server being flooded by invalid requests.

Now I get a lot of e-mails about "Could not fetch old counter' and "Could not send metering data". docker-compose.override.yml was applied since I can see new containers after compose up:

Creating network "mist_io_default" with the default driver
Pulling vmstorage (victoriametrics/vmstorage:v1.60.0-cluster)...
v1.60.0-cluster: Pulling from victoriametrics/vmstorage
540db60ca938: Pull complete
ecb38ea512d4: Pull complete
26e0ed6f2d85: Pull complete
Digest: sha256:ee8dd29ce74e4790b9695de408e3b01f2582c31821b4bf7ce1167650a8953235
Status: Downloaded newer image for victoriametrics/vmstorage:v1.60.0-cluster
Pulling vmselect (victoriametrics/vmselect:v1.60.0-cluster)...
v1.60.0-cluster: Pulling from victoriametrics/vmselect
540db60ca938: Already exists
ecb38ea512d4: Already exists
7cc5443caa74: Pull complete
Digest: sha256:7977d31a1b272b875e165fc8ff06e2f9a54179bdc6c8edf876fe0456e8503eba
Status: Downloaded newer image for victoriametrics/vmselect:v1.60.0-cluster
Pulling vminsert (victoriametrics/vminsert:v1.60.0-cluster)...
v1.60.0-cluster: Pulling from victoriametrics/vminsert
540db60ca938: Already exists
ecb38ea512d4: Already exists
ce451e1025b8: Pull complete
Digest: sha256:faa5e9d6e891f34459498ca77b0e6fef888225502653b76877dd00ba4774027f
Status: Downloaded newer image for victoriametrics/vminsert:v1.60.0-cluster
Creating mist_io_wsproxy_1       ... done
Creating mist_io_vmselect_1      ... done
Creating mist_io_memcached_1     ... done
Creating mist_io_huproxy_1              ... done
Creating mist_io_mongodb_1       ... done
Creating mist_io_traefik_1       ... done
Creating mist_io_vmstorage_1            ... done
Creating mist_io_influxdb_1             ... done
Creating mist_io_rabbitmq_1      ... done
Creating mist_io_elasticsearch_1 ... done
Creating mist_io_mailmock_1      ... done
Creating mist_io_ui_1            ... done
Creating mist_io_vminsert_1             ... done
Creating mist_io_socat_1         ... done
Creating mist_io_landing_1       ... done
Creating mist_io_swagger_1              ... done
Creating mist_io_gocky_1                ... done
Creating mist_io_dramatiq_1             ... done
Creating mist_io_elasticsearch-manage_1 ... done
Creating mist_io_kibana_1               ... done
Creating mist_io_apply-migrations_1     ... done
Creating mist_io_logstash_1             ... done
Creating mist_io_scheduler_1            ... done
Creating mist_io_api-v2_1               ... done
Creating mist_io_api_1                  ... done
Creating mist_io_sockjs_1               ... done
Creating mist_io_nginx_1                ... done

But the containers don't seem to work properly since I get those error e-mails.

davidrylance commented 2 years ago 
docker-compose ps
  Name                                  Command               State                                                                  Ports
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
mist_io_api-v2_1                 /mist.api/bin/docker-init  ...   Up       8080/tcp
mist_io_api_1                    /mist.api/bin/docker-init  ...   Up       80/tcp
mist_io_apply-migrations_1       /mist.api/bin/docker-init  ...   Exit 0
mist_io_dramatiq_1               /mist.api/bin/docker-init  ...   Up
mist_io_elasticsearch-manage_1   ./scripts/add_templates.py       Exit 0
mist_io_elasticsearch_1          /docker-entrypoint.sh elas ...   Up       9200/tcp, 9300/tcp
mist_io_gocky_1                  /go/src/github.com/mistio/ ...   Up       0.0.0.0:9096->9096/tcp,:::9096->9096/tcp, 0.0.0.0:9097->9097/tcp,:::9097->9097/tcp
mist_io_huproxy_1                /app --listen 0.0.0.0:8086       Up       8086/tcp
mist_io_influxdb_1               /entrypoint.sh influxd           Up       0.0.0.0:8083->8083/tcp,:::8083->8083/tcp, 0.0.0.0:8086->8086/tcp,:::8086->8086/tcp
mist_io_kibana_1                 /docker-entrypoint.sh kibana     Up       5601/tcp
mist_io_landing_1                /entry.sh sh /entry.sh           Up       80/tcp, 8000/tcp
mist_io_logstash_1               /usr/local/bin/docker-entr ...   Up       5044/tcp, 9600/tcp
mist_io_mailmock_1               twistd -n localmail --smtp ...   Up       8025/tcp
mist_io_memcached_1              docker-entrypoint.sh memcached   Up       11211/tcp
mist_io_mongodb_1                docker-entrypoint.sh mongod      Up       27017/tcp
mist_io_nginx_1                  nginx -g daemon off;             Up       0.0.0.0:80->80/tcp,:::80->80/tcp
mist_io_rabbitmq_1               docker-entrypoint.sh rabbi ...   Up       15671/tcp, 15672/tcp, 15691/tcp, 15692/tcp, 25672/tcp, 4369/tcp, 5671/tcp, 5672/tcp
mist_io_scheduler_1              /mist.api/bin/docker-init  ...   Up
mist_io_socat_1                  socat TCP4-LISTEN:2375,for ...   Up       2375/tcp
mist_io_sockjs_1                 /mist.api/bin/docker-init  ...   Up       8081/tcp
mist_io_swagger_1                sh /usr/share/nginx/docker ...   Up       8080/tcp
mist_io_traefik_1                /traefik --accesslog --api ...   Up       0.0.0.0:8040->80/tcp,:::8040->80/tcp, 0.0.0.0:8041->8080/tcp,:::8041->8080/tcp
mist_io_ui_1                     /entry.sh sh /entry.sh           Up       80/tcp, 8000/tcp
mist_io_vminsert_1               /vminsert-prod --influxTri ...   Up       0.0.0.0:49156->8480/tcp,:::49156->8480/tcp
mist_io_vmselect_1               /vmselect-prod --search.la ...   Up       0.0.0.0:49153->8481/tcp,:::49153->8481/tcp
mist_io_vmstorage_1              /vmstorage-prod --retentio ...   Up       0.0.0.0:49157->8400/tcp,:::49157->8400/tcp, 0.0.0.0:49155->8401/tcp,:::49155->8401/tcp, 0.0.0.0:49154->8482/tcp,:::49154->8482/tcp
mist_io_wsproxy_1                python3 wsproxy.py               Up       8764/tcp
cpsaltis commented 2 years ago

Hi @davidrylance, it will probably be easier to investigate this issue in real time. Would it be possible to arrange a quick call and look into it?

If not, we can stick to this ticket.