We are using Windows 2016 based Active Directory Domain Controller (2 of them) in our organization & have configured [powershell based scripts] email alerts on any account locked-out which occurs dueto 3 incorrect login attempts. We have also enabled audit on failed/success login under group policy.
Since yesterday I was receiving frequent email alerts for a user account locked-out which is used on four different oracle servers & the headache was that it had not any CALLING COMPUTER , I tried various tools to track the culprit but failed. Tested all services / task schedulers / saved credentials but no use. I also tried TCPVIEW on all four servers but did not found any nu-usual activities.
1 2 3 4 5 6 7 8 9 10 11 | Security ID: S-1-5-18Account Name: DC01$Account Domain: MYDOMAINLogon ID: 0x3e7Account That Was Locked Out:Security ID: S-1-5-21-664357565-1371172752-1124750213-14679Account Name: USERXAdditional Information:Caller Computer Name: . |
First lets see Possible causes of account locked-out …
- Mapped drives using old credentials
- Systems using old cached credentials
- Applications using old credentials
- Windows Services using expired credentials
- Scheduled Tasks
- Persistent drive mappings
- Mobile devices using domain services like Exchange mailbox
- Service Accounts using cached passwords
- Scheduled tasks
- Programs using stored credentials
- Misconfigured domain policy settings issues
- Disconnected Terminal Server sessions
- Active Directory delayed or failed replication
I sorted it by first enabling the NETLOGON debug on both Domain Controllers, and then examine both logs side by side closely for an hour using WINTAIL.
Finally I collected the following entries …
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 | 03/29 09:52:57 [LOGON] [5076] MYDOMAIN: SamLogon: Transitive Network logon of (null)\VMWARE from (via MYADMINPC) Entered03/29 09:52:57 [LOGON] [5076] MYDOMAIN: SamLogon: Transitive Network logon of (null)\VMWARE from (via MYADMINPC) Returns 0xC000006403/29 09:52:57 [LOGON] [5076] MYDOMAIN: SamLogon: Transitive Network logon of (null)\PRAXIS from (via MYADMINPC) Entered03/29 09:52:57 [LOGON] [5076] MYDOMAIN: SamLogon: Transitive Network logon of (null)\PRAXIS from (via MYADMINPC) Returns 0xC000006403/29 09:53:01 [LOGON] [5076] MYDOMAIN: SamLogon: Transitive Network logon of (null)\NAS from (via MYADMINPC) Entered03/29 09:53:01 [LOGON] [5076] MYDOMAIN: SamLogon: Transitive Network logon of (null)\NAS from (via MYADMINPC) Returns 0xC000006403/29 09:53:01 [LOGON] [2136] MYDOMAIN: SamLogon: Transitive Network logon of (null)\VEEAMSERVER from (via MYADMINPC) Entered03/29 09:53:01 [LOGON] [2136] MYDOMAIN: SamLogon: Transitive Network logon of (null)\VEEAMSERVER from (via MYADMINPC) Returns 0xC000006403/29 09:53:10 [LOGON] [4796] MYDOMAIN: SamLogon: Transitive Network logon of (null)\LENOVO from (via MYADMINPC) Entered03/29 09:53:10 [LOGON] [4796] MYDOMAIN: NlPickDomainWithAccount: LENOVO: Algorithm entered. UPN:0 Sam:1 Exp:0 Cross: 0 Root:1 DC:003/29 09:53:14 [LOGON] [2136] MYDOMAIN: SamLogon: Transitive Network logon of (null)\DBADMIN from (via MYADMINPC) Entered03/29 09:53:14 [LOGON] [2136] MYDOMAIN: NlPickDomainWithAccount: DBADMIN: Algorithm entered. UPN:0 Sam:1 Exp:0 Cross: 0 Root:1 DC:003/29 09:53:14 [LOGON] [2136] MYDOMAIN: SamLogon: Transitive Network logon of (null)\DBADMIN from (via MYADMINPC) Returns 0xC000006403/29 09:53:16 [LOGON] [4796] MYDOMAIN: SamLogon: Transitive Network logon of (null)\HARDADMIN from (via MYADMINPC) Entered03/29 09:53:16 [LOGON] [4796] MYDOMAIN: NlPickDomainWithAccount: HARDADMIN: Algorithm entered. UPN:0 Sam:1 Exp:0 Cross: 0 Root:1 DC:003/29 09:53:16 [LOGON] [4796] MYDOMAIN: SamLogon: Transitive Network logon of (null)\HARDADMIN from (via MYADMINPC) Returns 0xC000006403/29 09:53:19 [LOGON] [4796] MYDOMAIN: SamLogon: Transitive Network logon of (null)\ADMIN01 from (via MYADMINPC) Entered03/29 09:53:19 [LOGON] [4796] MYDOMAIN: NlPickDomainWithAccount: ADMIN01: Algorithm entered. UPN:0 Sam:1 Exp:0 Cross: 0 Root:1 DC:003/29 09:53:19 [LOGON] [4796] MYDOMAIN: SamLogon: Transitive Network logon of (null)\ADMIN01 from (via MYADMINPC) Returns 0xC000006403/29 09:53:19 [LOGON] [4796] MYDOMAIN: SamLogon: Transitive Network logon of (null)\NETWORK from (via MYADMINPC) Entered03/29 09:53:19 [LOGON] [4796] MYDOMAIN: NlPickDomainWithAccount: NETWORK: Algorithm entered. UPN:0 Sam:1 Exp:0 Cross: 0 Root:1 DC:003/29 09:53:19 [LOGON] [4796] MYDOMAIN: SamLogon: Transitive Network logon of (null)\NETWORK from (via MYADMINPC) Returns 0xC000006403/29 09:53:22 [LOGON] [2136] MYDOMAIN: SamLogon: Transitive Network logon of (null)\BACKUP from (via MYADMINPC) Entered03/29 09:53:22 [LOGON] [2136] MYDOMAIN: NlPickDomainWithAccount: BACKUP: Algorithm entered. UPN:0 Sam:1 Exp:0 Cross: 0 Root:1 DC:003/29 09:53:22 [LOGON] [2136] MYDOMAIN: SamLogon: Transitive Network logon of (null)\BACKUP from (via MYADMINPC) Returns 0xC000006403/29 09:53:22 [LOGON] [2136] MYDOMAIN: SamLogon: Transitive Network logon of (null)\COMMON from (via MYADMINPC) Entered03/29 09:53:22 [LOGON] [2136] MYDOMAIN: NlPickDomainWithAccount: COMMON: Algorithm entered. UPN:0 Sam:1 Exp:0 Cross: 0 Root:1 DC:003/29 09:53:22 [LOGON] [2136] MYDOMAIN: SamLogon: Transitive Network logon of (null)\COMMON from (via MYADMINPC) Returns 0xC0000064 |
so it found out that my admin PC was the culprit and then I scanned my admin pc with Malware-Bytes and although it did not detected any critical threat (except for some normal low level cookies etc) but the problem went right away.
I will keep monitoring the logs for next few days & will update here.
In short NETLOGON Debugging saved my Day !
Command to enable NETLOGON DEBUG
netlogon.log Log Location:
C:\Windows\debug\netlogon.log
To enable LOG, issue following CMD on on Domain Controller CMD
1 | nltest /dbflag:0x2080ffff |
It will start logging the file right away (at least in server 2016 I saw it happened immediately without needing of netlogon service restart)
When your task is finished, disable NetLogon Logging with below command:
1 | nltest /dbflag:0x0 |
Note: You may disable NETLOGON logging after resolved the issues to avoid server performance issue
You may want to look into Microsoft message analyzer as well. read it here
