search

mysociety / alaveteli

Provide a Freedom of Information request system for your jurisdiction
https://alaveteli.org
Other
389 stars 196 forks source link

Mail poller crashes on authentication failure #5325

Open sagepe opened 5 years ago

sagepe commented 5 years ago

If POP authentication fails, the mail poller crashes:

/opt/rbenv/versions/2.3.8/lib/ruby/2.3.0/net/pop.rb:1006:in `check_response_auth': -ERR [AUTH] Authentication failed. (Net::POPAuthenticationError)
        from /opt/rbenv/versions/2.3.8/lib/ruby/2.3.0/net/pop.rb:905:in `auth'
        from /opt/rbenv/versions/2.3.8/lib/ruby/2.3.0/net/pop.rb:567:in `do_start'
        from /opt/rbenv/versions/2.3.8/lib/ruby/2.3.0/net/pop.rb:537:in `start'
        from /data/vhost/eu.alaveteli.org/alaveteli-2019-08-01T12-44-57/lib/alaveteli_mail_poller.rb:113:in `start'
        from /data/vhost/eu.alaveteli.org/alaveteli-2019-08-01T12-44-57/lib/alaveteli_mail_poller.rb:24:in `poll_for_incoming'
        from /data/vhost/eu.alaveteli.org/alaveteli-2019-08-01T12-44-57/lib/alaveteli_mail_poller.rb:41:in `poll_for_incoming_loop'
        from /data/vhost/eu.alaveteli.org/shared/vendor/bundle/ruby/2.3.0/gems/railties-5.0.7.2/lib/rails/commands/runner.rb:63:in `<top (required)>'
        from /data/vhost/eu.alaveteli.org/shared/vendor/bundle/ruby/2.3.0/gems/railties-5.0.7.2/lib/rails/commands/runner.rb:63:in `eval'
        from /data/vhost/eu.alaveteli.org/shared/vendor/bundle/ruby/2.3.0/gems/railties-5.0.7.2/lib/rails/commands/runner.rb:63:in `<top (required)>'
        from /data/vhost/eu.alaveteli.org/shared/vendor/bundle/ruby/2.3.0/gems/railties-5.0.7.2/lib/rails/commands/commands_tasks.rb:138:in `require'
        from /data/vhost/eu.alaveteli.org/shared/vendor/bundle/ruby/2.3.0/gems/railties-5.0.7.2/lib/rails/commands/commands_tasks.rb:138:in `require_command!'
        from /data/vhost/eu.alaveteli.org/shared/vendor/bundle/ruby/2.3.0/gems/railties-5.0.7.2/lib/rails/commands/commands_tasks.rb:104:in `runner'
        from /data/vhost/eu.alaveteli.org/shared/vendor/bundle/ruby/2.3.0/gems/railties-5.0.7.2/lib/rails/commands/commands_tasks.rb:49:in `run_command!'
        from /data/vhost/eu.alaveteli.org/shared/vendor/bundle/ruby/2.3.0/gems/railties-5.0.7.2/lib/rails/commands.rb:18:in `<top (required)>'
        from bin/rails:4:in `require'
        from bin/rails:4:in `<main>'
garethrees commented 5 years ago

Hmm, I was thinking of this as a bug, but actually, isn't this a good thing? What should happen if we get the credentials wrong?

sagepe commented 5 years ago

Well, clearly we need to raise an exception of some form but a crash seems a bit extreme to me - surely it's better to handle known error conditions more gracefully?

In this case, the reason it failed was transient; the system as a whole would have become consistent in due course so subsequent poller runs would have started succeeding - but because the daemon had crashed, they stopped taking place until the daemon was restarted.

I notice there are regular cron jobs for some of the vhosts that use the poller to check and restart the daemon. This workaround may be acceptable for the time being, but we should reduce our reliance on cron and either eliminate the crashes where we can or use a better tool for process management.

Getting to #2523 would offer us another way of dealing with this in terms of more robust process management as this is something systemd handles. There are other options (such as running processes like this in a container) but exploring all these is probably out of scope for this ticket.