TroubleShooting WindowsDesktop SSO in Openam

[OpenAM] WindowsDesktopSSO problem - ERROR: Authentication failed with GSSException

When working on Openam Windows desktop SSO we had the following error " [OpenAM] WindowsDesktopSSO problem - ERROR: Authentication failed with GSSException"

As part of the Kerberos negotiation process, DNS lookups will be done to confirm that the server you're authenticating against is who it says it is.

So if your service is at openam.example.com <http://openam.example.com> then you need to make sure that:

1. It's an A record and not a CNAME and  - Looks good in our case
2. The reverse (PTR) record for that IP also resolves back to openam.example.com <http://openam.example.com> - - Looks good in our case
3. We verified the KVNO (Key version number) in key tab file to match the kvno is AD sp account
4. We also made sure that we are not using aliasing when we created the SP principal name in AD.
5. The service principal name can only exist on one account and is not linked to multiple accounts.
6. Also verified that we are using the load balancer url with full qualified name when in Key tab fil
7. The IDP is part of Our Intranet zone
8. Integrated Windows Authentication in IE is enabled
9. The server is not local to the browser.
10. The clients Kerberos systems is authenticated to a domain controller.
11. Your Active Directory principal and/or SPNs may be setup incorrectly. One gotcha: the SPN can only exist on one account; if you associate it to multiple then you'll have problem.

 

 Even after enabling all debug levels all we can see in Openam  debug.log is "*ERROR: Authentication failed with GSSException.":*

 Here is the exception snippet from debug.log

 (I did not copy the entire SPNEG token here. There are several lines of these hexadecimal numbers)

 ...

 ....

  e2 92 2e e9 a6 c3 4b 40 25 28 22 6c 75 79 41 f5

  fc 41 eb 49 03 4b 55 2a 7c fe 56 7c 80 f9 3c b4

  f1 f4 8e 47 0b 76 68 e2 c5 be e7 d9 cb b7 6c 98

  1c 06 eb 7e 7c 06 da e8 f3 f1 13 63 41 fe 22 15

  ec 71 1a 27 3e 8f f7 44 8a 9a ac 93 14 ca 2b b1

  

This tells that the KVNO (Key version number ) in your key tab file doesn’t match what’s in AD.
Verify Keytab file  kvno  by

ktab -l –k  yourkeytabfilename.keytab

 

You can change the KVNO number in key tab file to match the one in Active directory by using
Lets say if we need to set KVNO to you key tab file to 4
 

ktab -l -a HTTP/xyz@yourdomain.com ******** -n 4 -k yourkeytabfilename.keytab

where ******** is the password

How do I know what the value of KVNO in ActiveDirectory is ? 

Login to AD machine and use  "ADSIEdit.msc" on AD server and open the user account
and verify the  value of attribute "msDS-KeyVersionNumber" . If the  KVNO in keytab and AD accoun don’t match then WINDOWSDESKTOPSSO
within openam will not work. 

 Ok further we found that on Active Directory for a given user there is a flag
"Do not require Kerberos preauthentication" which we turned off.

Once we did that now we have the below error -
" Caused by: java.security.GeneralSecurityException: Checksum failed"

It so happened that our dev environment had AD on windows 2008.
Higher environments had AD on windows 2003.
The keytab that was generated using ktpass command that's on win 2003 was causing  the issue.
So our admin copied ktpass executable from win 2008 to win 2003 machine and regenerated 
the Keytab file on win 2003 machines and windows desktop sso started working.
So if you have Win 2003 there is every chance that you will have the above issues.