search

timja / jenkins-gh-issues-poc-06-18

0 stars 0 forks source link

[JENKINS-16163] NullPointerException when sending post build emails - all triggers appear to be impacted #2763

Closed timja closed 11 years ago

timja commented 11 years ago

NullPointerException when sending post build emails - all triggers appear to be impacted

Seems there may be a conflict between

I upgraded my jenkins instance yesterday from 1.493 to 1.494 and then noticed that builds no longer sent email alerts as expected when status changed.

Am using the Email Extension Plugin to trigger email alerts mainly when build meets the following conditions:

To get things working again, I needed to rollback my jenkins instance to v 1.493 as it appeared that with both ver. 2.25 & 2.24.1 of the mail plugin, mail did not send properly. As soon as I rolled back the jenkins version, mail started working properly again.

Info from build log for a build was triggered as 'fixed' (same type of error seen for build that was triggered as 'first failure')

IRC notifier plugin: Sending notification to: #alerts
Email was triggered for: Fixed
Trigger Success was overridden by another trigger and will not send an email.
Sending email for trigger: Fixed
ERROR: Could not send email as a part of the post-build publishers.
java.lang.NullPointerException
    at hudson.plugins.emailext.ExtendedEmailPublisher.createMail(ExtendedEmailPublisher.java:431)
    at hudson.plugins.emailext.ExtendedEmailPublisher.sendMail(ExtendedEmailPublisher.java:297)
    at hudson.plugins.emailext.ExtendedEmailPublisher._perform(ExtendedEmailPublisher.java:289)
    at hudson.plugins.emailext.ExtendedEmailPublisher.perform(ExtendedEmailPublisher.java:249)
    at hudson.tasks.BuildStepMonitor$3.perform(BuildStepMonitor.java:36)
    at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:804)
    at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:779)
    at hudson.model.Build$BuildExecution.cleanUp(Build.java:192)
    at hudson.model.Run.execute(Run.java:1587)
    at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
    at hudson.model.ResourceController.execute(ResourceController.java:88)
    at hudson.model.Executor.run(Executor.java:236)
Finished: SUCCESS
Originally reported by jyanko, imported from: NullPointerException when sending post build emails - all triggers appear to be impacted
  • assignee: slide_o_mix
  • status: Closed
  • priority: Minor
  • resolution: Not A Defect
  • resolved: 2013-02-06T01:45:30+00:00
  • imported: 2022/01/10
timja commented 11 years ago

slide_o_mix:

This won't be fixed until 1.494 (or something after) becomes the LTS release. email-ext is only guaranteed to work with the latest LTS. There were breaking changes in 1.494 for email session creation.

timja commented 11 years ago

slide_o_mix:

Reducing priority since there were known breaking changes in 1.494.

timja commented 11 years ago

slide_o_mix:

You could also enable debug mode in the global config and then post the output from the build.

timja commented 11 years ago

slide_o_mix:

There were breaking changes in 1.494 and above, email-ext is only guaranteed to work with latest LTS.

timja commented 11 years ago

kutzi:

If there were changes in core which can break plugins in such a way, is there a bug reported against core already?

timja commented 11 years ago

slide_o_mix:

This is because of the split of Mailer out of core. I don't know if an issue was filed against core or not.

timja commented 11 years ago

kutzi:

Then maybe we should reopen this and assign this to core?
The split-out of the Mailer should be transparent to all plugins relying on it. If it's not that's clearly a bug in core.

timja commented 11 years ago

slide_o_mix:

FYI, the line causing the exception is as follows:

msg = new MimeMessage(Mailer.descriptor().createSession());

I am not seeing this issue when I try and run with the latest (1.496) as part of the pom. Perhaps I will try installing into a separate instance of Jenkins and seeing if I see the same issue.

timja commented 11 years ago

slide_o_mix:

Reopening and assigning to core since this was a breaking change in core.

timja commented 11 years ago

richardm_tx:

We're having the same problem sending emails via our corporate smtp host since it requires a valid Sender Email Address (adminAddress).

timja commented 11 years ago

slide_o_mix:

A workaround would be to use "override default settings" in the global config and put the mail server and admin info there instead of relying on the mail config from Mailer.

timja commented 11 years ago

richardm_tx:

I just tried that, but it didn't work. I got the following error message:
javax.mail.SendFailedException: Invalid Addresses; nested exception is: com.sun.mail.smtp.SMTPAddressFailedException: 504 5.5.2 : Sender address rejected: need fully-qualified address

The input box for "System Admin E-mail Address" has a valid address, which is not nobody@nowhere.

timja commented 11 years ago

slide_o_mix:

Can you enable debug mode and post the build log?

timja commented 11 years ago

slide_o_mix:

Actually, nobody@nowhere is what JavaMail uses for an invalid address, can you try something different? Did you use that previously?

timja commented 11 years ago

richardm_tx:

I've tried using several different email address for Sender, but it always comes back with a complaint about nobody@nowhere. I've never tried using nobody@nowhere as the Sender.

As for enabling debug mode and posting the build log, that will have to wait until Monday, as I won't have access to our server over the weekend and am already late for my daughter's soccer practice.

timja commented 11 years ago

jyanko:

Tried updating my installation to 1.496 today - seems that I still get the same issue with this version.

Not sure if you need more log info from my particular installation at this point or not.

Where do I enable debug mode and grab the build log after doing so? Didn't see anything in the main jenkins config page for this.

timja commented 11 years ago

slide_o_mix:

Debug mode is enabled in the global config. The build log is the console log for the job.

timja commented 11 years ago

slide_o_mix:

Also, please take a screenshot of your global config (with the email-ext area shown) and fuzz out anything that shouldn't be shown to the world.

timja commented 11 years ago

slide_o_mix:

Do you have the "Mailer" plugin disabled in your config?

timja commented 11 years ago

richardm_tx:

I have been using the "Test configuration by sending test e-mail" feature. With Jenkins version 1.493 it sends test emails successfully, but with the same settings in versions 1.494 and up (including 1.497) it fails to connect to the smtp host ("java.net.SocketException: Permission denied: connect"). Mailer Plugin 1.4 and Email Extension Plugin 2.25 are both installed and enabled.

timja commented 11 years ago

slide_o_mix:

The test configuration by sending test-email is part of the Mailer plugin, not email-ext.

timja commented 11 years ago

richardm_tx:

Good to know. However, when I run my build it still fails to connect when trying to send the email notification.

timja commented 11 years ago

slide_o_mix:

You have override global settings checked in the extended email configuration and have filled in all the server info?

timja commented 11 years ago

richardm_tx:

Yup. Here's the contents of hudson.plugins.emailext.ExtendedEmailPublisher.xml:

http://myservername:8080/
richard.myname@mycompany.com
smtp.mycompany.com
false
text/plain
$PROJECT_NAME - Build # $BUILD_NUMBER - $BUILD_STATUS!
$PROJECT_NAME - Build # $BUILD_NUMBER - $BUILD_STATUS: Check console output at $BUILD_URL to view the results.
-1


true
false
true
false

timja commented 11 years ago

richardm_tx:

Just for fun, I disabled the Mailer plugin and got NullPointerException. When I have the Mailer enabled and the SMTP server field under Mailer blank, it tries to use localhost instead of the SMTP server specified under Email Extenstion.

timja commented 11 years ago

slide_o_mix:

Can you enable debug mode for the email-ext plugin and then post the build log?

timja commented 11 years ago

richardm_tx:

Sending e-mails to: richard.myname@mycompany.com
ERROR: Could not connect to SMTP host: localhost, port: 25
javax.mail.MessagingException: Could not connect to SMTP host: localhost, port: 25;
nested exception is:
java.net.SocketException: Permission denied: connect
at com.sun.mail.smtp.SMTPTransport.openServer(SMTPTransport.java:1934)
at com.sun.mail.smtp.SMTPTransport.protocolConnect(SMTPTransport.java:638)
at javax.mail.Service.connect(Service.java:295)
at javax.mail.Service.connect(Service.java:176)
at javax.mail.Service.connect(Service.java:125)
at javax.mail.Transport.send0(Transport.java:194)
at javax.mail.Transport.send(Transport.java:124)
at hudson.tasks.MailSender.execute(MailSender.java:116)
at hudson.tasks.Mailer.perform(Mailer.java:117)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:810)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:785)
at hudson.model.Build$BuildExecution.post2(Build.java:183)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:732)
at hudson.model.Run.execute(Run.java:1568)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
at hudson.model.ResourceController.execute(ResourceController.java:88)
at hudson.model.Executor.run(Executor.java:236)
Caused by: java.net.SocketException: Permission denied: connect
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.PlainSocketImpl.doConnect(Unknown Source)
at java.net.PlainSocketImpl.connectToAddress(Unknown Source)
at java.net.PlainSocketImpl.connect(Unknown Source)
at java.net.SocksSocketImpl.connect(Unknown Source)
at java.net.Socket.connect(Unknown Source)
at com.sun.mail.util.SocketFetcher.createSocket(SocketFetcher.java:286)
at com.sun.mail.util.SocketFetcher.getSocket(SocketFetcher.java:231)
at com.sun.mail.smtp.SMTPTransport.openServer(SMTPTransport.java:1900)
... 17 more
Finished: FAILURE

timja commented 11 years ago

slide_o_mix:

Can you please give more of the output? With debug mode enabled there should be a bunch more information before this.

timja commented 11 years ago

richardm_tx:

Nothing related to email, just why the job build crashed. This particular build was expected to fail, so that I could test the email extension.

timja commented 11 years ago

slide_o_mix:

Then debug mode is not enabled, there is much more output from the plugin before it tries to actually send the email.

timja commented 11 years ago

richardm_tx:

It looks to me like the email extensions plugin isn't being run at all. The boxes are checked and the xml file has values of true for both overrideGlobalSettings and debugMode.

timja commented 11 years ago

slide_o_mix:

Can you take a screenshot of the project config (fuzz anything the world shouldn't see). I'm especially interested in the Post-Build Actions.

timja commented 11 years ago

richardm_tx:

Here's the xml (can't fit everything in a screenshot):


- Continuous build for Dummy project
false

-
- - https://server.domain.net:8443/svn/TheCode/trunk/G9
.


false true
false
false
false
- -
#check every hour and launch the build if there was a change. 0 * * * * false
false
-
-
del Build\Dummy\DEBUG_VS2008\FV*.f* codebuild.bat -P Dummy --UDRIVE

-
C:\Jenkins\Test_Scripts\Dummy_Test_Script.bat


-
-
Build\Dummy\DEBUG_VS2008\FV*.full, Build\Dummy\DEBUG_VS2008\FV*.flash
false

-
richard.merrill@hp.com
false
false



timja commented 11 years ago

slide_o_mix:

Ok, it looks like you don't have email-ext configured at all. So, I don't think the issue you are seeing has anything to do with email-ext. email-ext doesn't show up under your list of publishers (only Mailer@1.4).

timja commented 11 years ago

richardm_tx:

And yet it shows up under the Manage Plugins page as being installed, not to mention it shows up on the config page. Also, the hudson.plugins.emailext.ExtendedEmailPublisher.xml file gets updated whenever I change any of the email extension settings.

timja commented 11 years ago

slide_o_mix:

Yes, they both can be installed, but unless you have it configured on the PROJECT page, it won't be used. Your config shows email-ext is not being used. The hudson.plugins.emailext.ExtendedEmailPublisher.xml file is for the GLOBAL configuration.

timja commented 11 years ago

richardm_tx:

Ok, so how do I enable it for the project? I see no checkbox or anything related to email-ext on the project configure page, other than the Email Notification section, which only has a field for recipients and checkboxes for when to send emails.

timja commented 11 years ago

slide_o_mix:

There should be a drop-down for post-build actions. You select the email-ext plugin (Extended Email Publisher) from there and then configure for your needs.

timja commented 11 years ago

richardm_tx:

I'm not seeing the email-ext plugin, only "Editable Email Notification".

timja commented 11 years ago

slide_o_mix:

That's the one...couldn't remember what its labeled as. Please see the wiki page (https://wiki.jenkins-ci.org/display/JENKINS/Email-ext+plugin) for more information on the plugin.

timja commented 11 years ago

richardm_tx:

Still no joy. This was with regular email notification turned off and using the editable email notification.

Build step 'Archive the artifacts' changed build result to FAILURE
Checking for post-build
Performing post-build step
Checking if email needs to be generated
Email was triggered for: Failure
Sending email for trigger: Failure
Overriding default server settings, creating our own session
messageContentType = text/plain; charset=UTF-8
Adding recipients from recipient list
Successfully created MimeMessage
Sending email to: richard.merrill@mycompany.com
ERROR: Could not send email as a part of the post-build publishers.
javax.mail.MessagingException: Could not connect to SMTP host: smtp.mycompany.com, port: 25;
nested exception is:
java.net.SocketException: Permission denied: connect
at com.sun.mail.smtp.SMTPTransport.openServer(SMTPTransport.java:1934)
at com.sun.mail.smtp.SMTPTransport.protocolConnect(SMTPTransport.java:638)
at javax.mail.Service.connect(Service.java:295)
at javax.mail.Service.connect(Service.java:176)
at javax.mail.Service.connect(Service.java:125)
at javax.mail.Transport.send0(Transport.java:194)
at javax.mail.Transport.send(Transport.java:124)
at hudson.plugins.emailext.ExtendedEmailPublisher.sendMail(ExtendedEmailPublisher.java:308)
at hudson.plugins.emailext.ExtendedEmailPublisher._perform(ExtendedEmailPublisher.java:289)
at hudson.plugins.emailext.ExtendedEmailPublisher.perform(ExtendedEmailPublisher.java:249)
at hudson.tasks.BuildStepMonitor$3.perform(BuildStepMonitor.java:36)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:810)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:785)
at hudson.model.Build$BuildExecution.cleanUp(Build.java:192)
at hudson.model.Run.execute(Run.java:1587)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
at hudson.model.ResourceController.execute(ResourceController.java:88)
at hudson.model.Executor.run(Executor.java:236)
Caused by: java.net.SocketException: Permission denied: connect
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.PlainSocketImpl.doConnect(Unknown Source)
at java.net.PlainSocketImpl.connectToAddress(Unknown Source)
at java.net.PlainSocketImpl.connect(Unknown Source)
at java.net.SocksSocketImpl.connect(Unknown Source)
at java.net.Socket.connect(Unknown Source)
at java.net.Socket.connect(Unknown Source)
at com.sun.mail.util.SocketFetcher.createSocket(SocketFetcher.java:288)
at com.sun.mail.util.SocketFetcher.getSocket(SocketFetcher.java:231)
at com.sun.mail.smtp.SMTPTransport.openServer(SMTPTransport.java:1900)
... 17 more
Some error occured trying to send the email...check the Jenkins log
Finished: FAILURE

timja commented 11 years ago

slide_o_mix:

Since it occurs with both plugins, I would try looking at your SMTP server config. Did something change recently? The way email-ext connects is similar but different from the Mailer plugin, so I don't think its an issue with the plugins themselves, it is more likely something else in the environment. If you are using JDK 7, please try what they mention in this page: http://stackoverflow.com/questions/7477712/sending-email-using-jsp/7478027#7478027 (TL;DR Set -Djava.net.preferIPv4Stack=true) and see if that helps with the issue.

timja commented 11 years ago

richardm_tx:

As far as I know, our SMTP server config has not changed. I work for a very large company and would never be allowed to change it anyway. All I know is if I use Jenkins 1.493 and Email Extensions 2.25 I am able to send emails. If I upgrade to Jenkins 1.494 or higher, I can't.

As far as the JDK tip, I have yet to figure out where I should be adding that property.

timja commented 11 years ago

slide_o_mix:

So, you are sure that the email-ext plugin is enabled for the other jobs since it wasn't for the test one you originally posted the XML for. You should set the property for the JVM I believe, I am not certain where to set it.

timja commented 11 years ago

slide_o_mix:

Can you enable JavaMail debug like in the following issue?

https://issues.jenkins-ci.org/browse/JENKINS-16237

timja commented 11 years ago

richardm_tx:

I currently have Jenkins running on two separate Windows Server 2008 R2 servers as version 1.493 and 1.496. I created a new test job on each that does nothing more than try to run a non-existent batch file, triggering a build failure. On Jenkins 1.493, it sends the email via email-ext@2.25 both with Override Global Settings set and without. On Jenkins 1.496, it fails to connect to the smtp server regardless of whether Override Global Settings is set.

Here's the config for my test job:




false


true
false
false
false

false


C:\Jenkins\Test_Scripts\Nonexistent_Test_Script.bat




richard.merrill@MyReallyBigCompany.com




$PROJECT_DEFAULT_SUBJECT
$PROJECT_DEFAULT_CONTENT
false
false
false
true

false






$PROJECT_DEFAULT_SUBJECT
$PROJECT_DEFAULT_CONTENT
false
false
false
true

false




default
test email from ROMBUILD Jenkins
$DEFAULT_CONTENT


false
richard.merrill@MyReallyBigCompany.com



timja commented 11 years ago

richardm_tx:

I have yet to figure out how/where to enable JavaMail debugging...

timja commented 11 years ago

slide_o_mix:

Do you get the same behavior with the Mailer plugin as well with the different versions?

timja commented 11 years ago

slide_o_mix:

Are you running under Tomcat? http://stackoverflow.com/questions/5195154/setting-runtime-property-in-web-application

timja commented 11 years ago

richardm_tx:

We are running the Windows Web Server IIS. The Mailer plugin seems to work with 1.493, but not with 1.496.