search

StackStorm-Exchange / stackstorm-email

E-Mail Actions/Sensors for StackStorm
https://exchange.stackstorm.org/
Apache License 2.0
9 stars 45 forks source link

Config schema and length check not matching #2

Closed mickmcgrath13 closed 7 years ago

mickmcgrath13 commented 7 years ago

The schema is: https://github.com/StackStorm-Exchange/stackstorm-email/blob/master/config.schema.yaml#L6

The length check is: https://github.com/StackStorm-Exchange/stackstorm-email/blob/master/actions/send_email.py#L12

change the config schema and then check the length of the dict differently.

Per The docs, the configuration should look like:

imap_mailboxes:
  gmail_main:
    server: "gmail.imap.com"
    port: 993
    username: "james@stackstorm.com"
    password: "superawesomepassword"
    ssl: True
    download_attachments: False
  alternate:
    server: "mail.stackstorm.net"
    port: 143
    username: "stanley"
    password: "esteetew"
    folder: "StackStorm"
    ssl: True
    download_attachments: True
max_attachment_size: 1024
attachment_datastore_ttl: 1800

..however, with a email.yaml like:

imap_mailboxes:
  gmail_main:
    server: "imap.gmail.com"
    port: 993
    username: "my.email.address@gmail.com"
    password: "stackstormFTW"
    ssl: true
    download_attachments: True
max_attachment_size: 1024
attachment_datastore_ttl: 1800
smtp_listen_ip: 'smtp.gmail.com'
smtp_listen_port: 465

..and an action object like:

{
  "account": "gmail_main",
  "email_to": "someone@gmail.com",
  "message": "Test",
  "email_from": "my.email.address@gmail.com",
  "subject": "test"
}

this error is output:

Traceback (most recent call last):
  File "/opt/stackstorm/st2/lib/python2.7/site-packages/st2common/runners/python_action_wrapper.py", line 259, in <module>
    obj.run()
  File "/opt/stackstorm/st2/lib/python2.7/site-packages/st2common/runners/python_action_wrapper.py", line 155, in run
    output = action.run(**self._parameters)
  File "/opt/stackstorm/packs/email/actions/send_email.py", line 13, in run
    raise ValueError('at least one account is required to send email.')
ValueError: at least one account is required to send email.
LindsayHill commented 7 years ago

@warrenvw here's one related to those small cleanups we were talking about. Should be straightforward fix

warrenvw commented 7 years ago

I was able to reproduce this using https://github.com/stackstorm/st2-docker.

git clone https://github.com/stackstorm/st2-docker
cd st2-docker
make env
docker-compose up -d
make exec

You should find yourself in the stackstorm docker container. The following commands are run in the container.

root@e5eefde05459:/opt/stackstorm/configs# st2 pack install email

    [ succeeded ] download pack
    [ succeeded ] make a prerun
    [ succeeded ] install pack dependencies
    [ succeeded ] register pack

+-------------+---------------------------------------+
| Property    | Value                                 |
+-------------+---------------------------------------+
| name        | email                                 |
| description | E-Mail Actions/Sensors for StackStorm |
| version     | 0.2.0                                 |
| author      | James Fryman                          |
+-------------+---------------------------------------+
root@e5eefde05459:/opt/stackstorm/configs# st2 pack config email
smtp_listen_ip [127.0.0.1]: 
smtp_listen_port (integer) [1025]: 
attachment_datastore_ttl (integer) [1800]: 
max_attachment_size (integer) [1024]: 
---
Do you want to preview the config in an editor before saving? [y]: n
---
Do you want me to save it? [y]: y
+----------+----------------------------------------+
| Property | Value                                  |
+----------+----------------------------------------+
| pack     | email                                  |
| values   | {                                      |
|          |     "smtp_listen_ip": "127.0.0.1",     |
|          |     "attachment_datastore_ttl": 1800,  |
|          |     "smtp_listen_port": 1025,          |
|          |     "max_attachment_size": 1024,       |
|          |     "imap_mailboxes": {}               |
|          | }                                      |
+----------+----------------------------------------+

The default configuration is created and registered by st2 pack config email. Let's try sending an email.

root@e5eefde05459:/opt/stackstorm/configs# st2 run email.send_email account=junk subject="Hello" email_to="personal@example.com" message="This is a test email" email_from="junk@example.com"
.
id: 5938f33b8d36d400fe64758b
status: failed
parameters: 
  account: junk
  email_from: junk@example.com
  email_to: personal@example.com
  message: This is a test email
  subject: Hello
result: 
  exit_code: 1
  result: None
  stderr: "Traceback (most recent call last):
  File "/opt/stackstorm/st2/local/lib/python2.7/site-packages/st2common/runners/python_action_wrapper.py", line 259, in <module>
    obj.run()
  File "/opt/stackstorm/st2/local/lib/python2.7/site-packages/st2common/runners/python_action_wrapper.py", line 155, in run
    output = action.run(**self._parameters)
  File "/opt/stackstorm/packs/email/actions/send_email.py", line 13, in run
    raise ValueError('at least one account is required to send email.')
ValueError: at least one account is required to send email.
"
  stdout: ''

This reproduces the same error you encountered. However, this error is expected given the existing configuration. The imap_mailboxes dict has 0 keys. Let's add some by editing /opt/stackstorm/configs/email.yaml so it contains:

imap_mailboxes:
  junk:
    server: "imap.example.com"
    port: 993
    username: "junk@example.com"
    password: "Junk468"
    ssl: true
    download_attachments: True
  personal:
    server: "imap.example.com"
    port: 993
    username: "email@example.com"
    password: "personalpwd"
    ssl: true
    download_attachments: True
max_attachment_size: 1024
attachment_datastore_ttl: 1800
smtp_listen_ip: smtp.example.com
smtp_listen_port: 587

Let's try sending an email with the exact same command:

root@e5eefde05459:/opt/stackstorm/configs# st2 run email.send_email account=junk subject="Hello" email_to="personal@example.com" message="This is a test email" email_from="junk@example.com"
.
id: 5938f5028d36d400fe64758e
status: failed
parameters: 
  account: junk
  email_from: junk@example.com
  email_to: personal@example.com
  message: This is a test email
  subject: Hello
result: 
  exit_code: 1
  result: None
  stderr: "Traceback (most recent call last):
  File "/opt/stackstorm/st2/local/lib/python2.7/site-packages/st2common/runners/python_action_wrapper.py", line 259, in <module>
    obj.run()
  File "/opt/stackstorm/st2/local/lib/python2.7/site-packages/st2common/runners/python_action_wrapper.py", line 155, in run
    output = action.run(**self._parameters)
  File "/opt/stackstorm/packs/email/actions/send_email.py", line 13, in run
    raise ValueError('at least one account is required to send email.')
ValueError: at least one account is required to send email.
"
  stdout: ''

Notice the same error. Now, register the config and try sending an email:

root@e5eefde05459:/opt/stackstorm/configs# st2ctl reload --register-configs
Registering content...[flags = --config-file /etc/st2/st2.conf --register-configs]
2017-06-08 06:58:20,352 INFO [-] Connecting to database "st2" @ "mongo:27017" as user "None".
2017-06-08 06:58:20,477 INFO [-] =========================================================
2017-06-08 06:58:20,478 INFO [-] ############## Registering configs ######################
2017-06-08 06:58:20,478 INFO [-] =========================================================
2017-06-08 06:58:21,199 INFO [-] Registered 1 configs.
##### st2 components status #####
st2actionrunner PID: 136
st2actionrunner PID: 159
st2api PID: 59
st2api PID: 254
st2stream PID: 61
st2stream PID: 264
st2auth PID: 48
st2auth PID: 257
st2garbagecollector PID: 44
st2notifier PID: 54
st2resultstracker PID: 52
st2rulesengine PID: 57
st2sensorcontainer PID: 42
st2chatops is not running.
mistral-server PID: 409
mistral-api PID: 403
mistral-api PID: 434
mistral-api PID: 435
root@e5eefde05459:/opt/stackstorm/configs# st2 run email.send_email account=junk subject="Hello" email_to="personal@example.com" message="This is a test email" email_from="junk@example.com"
.
id: 5938f6398d36d400fe647597
status: failed
parameters: 
  account: junk
  email_from: junk@example.com
  email_to: personal@example.com
  message: This is a test email
  subject: Hello
result: 
  exit_code: 1
  result: None
  stderr: "Traceback (most recent call last):
  File "/opt/stackstorm/st2/local/lib/python2.7/site-packages/st2common/runners/python_action_wrapper.py", line 259, in <module>
    obj.run()
  File "/opt/stackstorm/st2/local/lib/python2.7/site-packages/st2common/runners/python_action_wrapper.py", line 155, in run
    output = action.run(**self._parameters)
  File "/opt/stackstorm/packs/email/actions/send_email.py", line 27, in run
    s = SMTP(account_data['server'], int(account_data['port']), timeout=20)
  File "/usr/lib/python2.7/smtplib.py", line 256, in __init__
    (code, msg) = self.connect(host, port)
  File "/usr/lib/python2.7/smtplib.py", line 316, in connect
    self.sock = self._get_socket(host, port, self.timeout)
  File "/usr/lib/python2.7/smtplib.py", line 291, in _get_socket
    return socket.create_connection((host, port), timeout)
  File "/usr/lib/python2.7/socket.py", line 553, in create_connection
    for res in getaddrinfo(host, port, 0, SOCK_STREAM):
socket.gaierror: [Errno -2] Name or service not known
"
  stdout: ''

The account is now located in the config, and send_email attempts to send the email. Since smtp.example.com doesn't exist, the above error is expected.

So while I can't be 100% certain you have a similar setup, please try running st2ctl reload --register-configs and check if you still see the error ValueError: at least one account is required to send email?

If this solves your problem, fantastic! If not, please provide me detailed steps on how to reproduce. Thanks!

mickmcgrath13 commented 7 years ago

Finally got around to testing this again. I've updated my email.yaml file to look like this:

imap_mailboxes:
  gmail_stackstorm_demo:
    server: "imap.gmail.com"
    port: 993
    username: "example@gmail.com"
    password: "mypass"
    ssl: true
    download_attachments: True
attachment_datastore_ttl: 1800
max_attachment_size: 1024
smtp_listen_ip: smtp.gmail.com
smtp_listen_port: 465

I then register the configs with success:

$ sudo st2ctl reload --register-configs
Registering content...[flags = --config-file /etc/st2/st2.conf --register-configs]
2017-07-21 12:00:32,620 INFO [-] Connecting to database "st2" @ "0.0.0.0:27017" as user "st2".
2017-07-21 12:00:32,749 INFO [-] =========================================================
2017-07-21 12:00:32,749 INFO [-] ############## Registering configs ######################
2017-07-21 12:00:32,749 INFO [-] =========================================================
2017-07-21 12:00:33,512 INFO [-] Registered 4 configs.
##### st2 components status #####
st2actionrunner PID: 16966
st2actionrunner PID: 16987
st2actionrunner PID: 17008
st2actionrunner PID: 17029
st2actionrunner PID: 17050
st2actionrunner PID: 17071
st2actionrunner PID: 17096
st2actionrunner PID: 17123
st2actionrunner PID: 17150
st2actionrunner PID: 17177
st2api PID: 17242
st2api PID: 30405
st2stream PID: 17296
st2stream PID: 17315
st2auth PID: 17345
st2auth PID: 17367
st2garbagecollector PID: 17398
st2notifier PID: 17436
st2resultstracker PID: 17468
st2rulesengine PID: 17500
st2sensorcontainer PID: 17534
st2chatops PID: 17572
st2chatops PID: 17573
mistral-server PID: 18107
mistral-api PID: 18063
mistral-api PID: 18073
mistral-api PID: 18074

Then, when I try sending an email, it does make it beyond the previous error (thank you!!), but now I get this error:

$ st2 run email.send_email account=gmail_stackstorm_demo subject="Hello" email_to="email_to@example.com" message="This is a test email" email_from="email_from@example.com"
......
id: 59724f53271fb276c5380492
status: failed
parameters: 
  account: gmail_stackstorm_demo
  email_from: email_from@example.com
  email_to: email_to@example.com
  message: This is a test email
  subject: Hello
result: 
  exit_code: 1
  result: None
  stderr: "Traceback (most recent call last):
  File "/opt/stackstorm/st2/lib/python2.7/site-packages/st2common/runners/python_action_wrapper.py", line 259, in <module>
    obj.run()
  File "/opt/stackstorm/st2/lib/python2.7/site-packages/st2common/runners/python_action_wrapper.py", line 155, in run
    output = action.run(**self._parameters)
  File "/opt/stackstorm/packs/email/actions/send_email.py", line 27, in run
    s = SMTP(account_data['server'], int(account_data['port']), timeout=20)
  File "/usr/share/python/st2python/lib/python2.7/smtplib.py", line 256, in __init__
    (code, msg) = self.connect(host, port)
  File "/usr/share/python/st2python/lib/python2.7/smtplib.py", line 317, in connect
    (code, msg) = self.getreply()
  File "/usr/share/python/st2python/lib/python2.7/smtplib.py", line 368, in getreply
    raise SMTPServerDisconnected("Connection unexpectedly closed")
smtplib.SMTPServerDisconnected: Connection unexpectedly closed
"
  stdout: ''

Any thoughts or advice on how to debug this?

LindsayHill commented 7 years ago

@mickmcgrath13 I think there's a bit of confusion here about this pack's configuration. It's not your fault, it's more the way the pack is written. My guess is that the pack was originally just an IMAP & SMTP inbound email sensor, and it had the send_email action added, re-using some of the original configuration. The pack should really be re-written.

Anyway, first look at these lines in the config:

smtp_listen_ip: smtp.gmail.com
smtp_listen_port: 465

Note the listen part of the parameter name. Those parameters apply to the SMTP sensor, for receiving email. They are not used for sending email. They are telling the sensor what IP to listen on, and which port to listen on.

Looking at the code for send_email.py https://github.com/StackStorm-Exchange/stackstorm-email/blob/master/actions/send_email.py#L9, we can see that it is retrieving the username/password/server/port from the imap_mailboxes section of the config. Confusing, because we're trying to send email, not retrieve it via IMAP.

If you have configuration like this:

imap_mailboxes:
  gmail_stackstorm_demo:
    server: "imap.gmail.com"
    port: 993
    username: "example@gmail.com"
    password: "mypass"
    ssl: true
    download_attachments: True

and you run st2 run email.send_email account=gmail_stackstorm_demo subject="Hello" email_to="email_to@example.com" message="This is a test email" email_from="email_from@example.com"

then it will be trying to send email using imap.gmail.com on port 993. That won't work.

If you run a tcpdump on your server when running that action, I'm fairly certain you will see outbound tcp/993 traffic.

The initial connection will succeed, but then when you start sending SMTP commands to an IMAP server, it will kick you off, resulting in that smtplib.SMTPServerDisconnected: Connection unexpectedly closed message.

You will need to have configuration that looks something like this:

imap_mailboxes:
  gmail_stackstorm_demo:
    server: "smtp.gmail.com"
    port: 465
    username: "example@gmail.com"
    password: "mypass"
    ssl: true
    download_attachments: True

If you are also using the IMAP mail sensor, you will need another config block that contains the values you already have, with port 993 & imap.gmail.com.

The pack should really be re-written to, at a minimum, change the mailbox configuration block to have separate values for smtp_server and smtp_port.

mickmcgrath13 commented 7 years ago

Okay, so I'm able to get it to send an email, now! ..not with a .gmail account, though. I added a non-gmail configuration in the way you've just described for smtp, and it sent.
That same non-gmail address did not send with the smtp listen ip/port.

Seems like gmail doesn't like the setup for some reason?

I am done with this issue now because our non-gmail address is preferred, anyway, but it would be nice if the gmail setup at least gave a more detailed response.

Anyway, thanks so much for your help!