"pam_authenticate failed" what drives this message?

[MikeyCarter]

2019-10-30 15:51:03.239 ERROR 20613 --- [nio-8443-exec-4] o.o.a.s.CustomAuthenticationProvider     : PAM authentication failed: pam_authenticate failed : Authentication failure -- org.jvnet.libpam.PAMException: pam_authenticate failed : Authentication failure
    at org.jvnet.libpam.PAM.check(PAM.java:106)
    at org.jvnet.libpam.PAM.authenticate(PAM.java:124)

Getting this in my program. Tryied on three different linux OS same result. Can't seem to figure out what this message is telling me. Did I miss a step somewhere?

pamtester -v login **** authenticate

This works... but fails via the java program running as the same user.

Tried on Fedora 27, Fedora 29, even a Oracle Linux 7 I had kicking around.

[MikeyCarter]

Interesting development. I downloaded your source into my program so I could play with the logging. It seems that if the password is valid... it sets it to null then says access denied. If the password is invalid it's access denied. If the password has special characters it's also denied.

Hard-code the password..... access works.

                LOGGER.info("1pass:"+password);
                password = "testpassword";
                if(password==null)
                    return PAM_CONV_ERR;

                LOGGER.info("2pass:"+password);

[MikeyCarter]

ok finally found the problem. Never fails I search at a problem but the minute I have to document it into an issue like this I find it.

So my problem was this:

            Collection<? extends GrantedAuthority> authorities = Collections.singleton(new SimpleGrantedAuthority("ROLE_USER"));
            return new UsernamePasswordAuthenticationToken(authentication.getPrincipal(), 
                                                           authentication.getCredentials(),
                                                           authorities);

vs

                                                           authentication.getCredentials());

Without the ROLES at the end it authenticates once fine (which I missed in the logs) then blanks out the password. Then tries a second attempt with a null password. Which is what I saw above. When it was a wrong password the thing would fail on first try.

So resolution... pay attention to the logs more closely. Posting here in case anyone else falls into the same trap but the issue can be closed.