Real-time Auditing of changes in Active Directory

[Chuck Chopp]

I'm interested in identifying the programming interfaces used for real-time
auditing of changes to objects in Active Directory. The LDAP uSNChanged and
DirSync ADSI control are not sufficient for what I'm doing, and modifying
the object security and DC security policies to turn on security auditing is
not a viable alternative, either.

There are existing products, such as Quest's "Quest Change Manager for
Active Directory" that appear to be collecting the same time of real-time
information that I'm looking for, and Quest claims to be doing so w/o using
the native audit log features of Windows when doing so.

In a nutshell, I need to receive notifications on a DC whenever an event of
interest happens within the domain or any of its child containers that the
DC contains in its replica of its portion of the tree. I would prefer to
register to receive notification of only the events I'm interested in, but
if I have to receive all events and evaluate them that's OK, too.

Object creation
Object deletion
Object modification [excluding DN changes]
Object rename/move

In the case of object modification, I need to know what attribute was
changed, what the previous value was and what the new value is, or, if it is
a multi-valued attribute, I need to know what the particular value is that
was added to or removed from the list along with the actual add/remove value
operation being identified.

I've gone over the Platform SDK docs [updated for Windows Server 2003 SP1]
and I'm not seeing *anything* even remotely close to what I'm looking for.
However, since there are commercial products on the market that seem to be
obtaining the same type of information, there's got to be some sort of
programming interface with which to obtain the desired information. In the
Novell environment eDirectory [f.k.a. NDS] has a very comprehensive event
monitoring API that can be used to achieve a fine degree of granularity in
terms of the events that can be monitored, and the event notifications can
be delivered via an async callback mechanism.

Is there something obvious or less than obvious that I'm missing? Or, are
these products making use of undocumented interfaces to perform their tasks?

Any assistance would be appreciated.


TIA,

Chuck
--
Chuck Chopp

ChuckChopp (at) rtfmcsi (dot) com http://www.rtfmcsi.com

RTFM Consulting Services Inc. 864 801 2795 voice & voicemail
103 Autumn Hill Road 864 801 2774 fax
Greer, SC 29651

Do not send me unsolicited commercial email.

[Joe Kaplan (MVP - ADSI)]

If you don't want to do change polling, then the only option I'm aware of
are LDAP change notifications. They are documented here:

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/ad/ad/change_notifications_in_active_directory.asp?frame=true

Note that MS warns against using these on an entire naming context due to
performance problems, especially on big DCs. It appears that they really
want you to use one of the polling-based approaches (dirsync or usnChanged),
even though you have ruled that out.

If anyone else has heard of anything, I'd be interested in hearing about it.

Joe K.

Post by Chuck Chopp
I'm interested in identifying the programming interfaces used for
real-time auditing of changes to objects in Active Directory. The LDAP
uSNChanged and DirSync ADSI control are not sufficient for what I'm doing,
and modifying the object security and DC security policies to turn on
security auditing is not a viable alternative, either.
There are existing products, such as Quest's "Quest Change Manager for
Active Directory" that appear to be collecting the same time of real-time
information that I'm looking for, and Quest claims to be doing so w/o
using the native audit log features of Windows when doing so.
In a nutshell, I need to receive notifications on a DC whenever an event
of interest happens within the domain or any of its child containers that
the DC contains in its replica of its portion of the tree. I would prefer
to register to receive notification of only the events I'm interested in,
but if I have to receive all events and evaluate them that's OK, too.
Object creation
Object deletion
Object modification [excluding DN changes]
Object rename/move
In the case of object modification, I need to know what attribute was
changed, what the previous value was and what the new value is, or, if it
is a multi-valued attribute, I need to know what the particular value is
that was added to or removed from the list along with the actual
add/remove value operation being identified.
I've gone over the Platform SDK docs [updated for Windows Server 2003 SP1]
and I'm not seeing *anything* even remotely close to what I'm looking for.
However, since there are commercial products on the market that seem to be
obtaining the same type of information, there's got to be some sort of
programming interface with which to obtain the desired information. In
the Novell environment eDirectory [f.k.a. NDS] has a very comprehensive
event monitoring API that can be used to achieve a fine degree of
granularity in terms of the events that can be monitored, and the event
notifications can be delivered via an async callback mechanism.
Is there something obvious or less than obvious that I'm missing? Or, are
these products making use of undocumented interfaces to perform their tasks?
Any assistance would be appreciated.
TIA,
Chuck
--
Chuck Chopp
ChuckChopp (at) rtfmcsi (dot) com http://www.rtfmcsi.com
RTFM Consulting Services Inc. 864 801 2795 voice & voicemail
103 Autumn Hill Road 864 801 2774 fax
Greer, SC 29651
Do not send me unsolicited commercial email.

[Chuck Chopp]

Post by Joe Kaplan (MVP - ADSI)
If you don't want to do change polling, then the only option I'm aware of
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/ad/ad/change_notifications_in_active_directory.asp?frame=true

Yes, that's the stuff straight out of the MSDN Platform SDK... and it's very
much lacking in terms of what I'm wanting to do. Perhaps I'm spoiled with
what can be done w/respect to event notification in eDirectory, but somehow
someway AD has got to have a "native interface" that exposes better
functionality than what's available via LDAP. Even the LDAP control
extensions that AD implements are weak in comparison to the ones implemented
by eDirectory.

Post by Joe Kaplan (MVP - ADSI)
Note that MS warns against using these on an entire naming context due to
performance problems, especially on big DCs. It appears that they really
want you to use one of the polling-based approaches (dirsync or usnChanged),
even though you have ruled that out.

I've come to the conclusion that Microsoft has very little faith in its own
directory services product compared to what Novell does with eDirectory.
It's kind of like the difference between a Fisher Price toddler's piano and
a Steinway baby grand piano... one is a toy for children and the other is a
finely tuned professional instrument. That's not said to start a flame-war,
it's simply an observation and vented in frustration at the lack of
documented & supported functionality. For small tasks, AD works just fine,
but for large scale industrial-strength directory-enabled applications, MS
seems to be hesitant in terms of what AD will be capable of doing.

Post by Joe Kaplan (MVP - ADSI)
If anyone else has heard of anything, I'd be interested in hearing about it.

The LDAP method mentioned in the Platform SDK doesn't provide the
granularity I'm looking for, nor do either of the polling methods.
Specifically, I need to know if the change is due to object creation,
deletion, rename, move or is just a generic modification of the object's
attributes. And, if it's modified attributes, I need to know the before &
after attribute values for single valued attributes, and, for multi-valued
attributes, I need to know the individual value in the list that was
modified and whether the value was added to or removed from the list.

Take a look at these links:

http://wm.quest.com/Library/getDocument.asp?target=cmadpds
http://www.bi101.net/products/solutions/netpro/
http://www.netpro.com/products/changeauditor/index.cfm

These products are all making claims of auditing AD events and offering a
fine level of granularity in the changes w/o making use of any of the
built-in auditing mechanisms. The functionality they describe cannot be
achieved using DirSync or LDAP as far as I know, so that leaves me with the
thought that they are using some *other* interface into Active Directory.
It's that *other* interface that I'm interested in learning about. Given
that there's more than one product doing this, I'm guessing that they all
work in a similar manner using the same interface into AD. The alternative
is that they're maintaining private replicas of AD information, and that's a
grossly inefficient method that wouldn't perform nearly as well how these
products are supposed to be performing.

Maybe there's a means of hooking into the replication interfaces in AD. If
I could reliably hook into AD in that manner then I could intercept every
single piece of replication traffic and *that* would allow me to obtain the
desired information in real-time, or at least as close to real-time as the
replication schedule allows for. It would eliminate polling and it would
certainly allow me to directly observe in very fine detail exactly what is
going on in AD.

Another possibility is the thinly documented event tracing facility.
There's a very vague reference to in in the Platform SDK in connection with
AD, but there's nothing of substance in the docs to indicate if I'm heading
in the right direction with the thoughts of trying to track down an event
source that will provide the desired event information.
--
Chuck Chopp

ChuckChopp (at) rtfmcsi (dot) com http://www.rtfmcsi.com

RTFM Consulting Services Inc. 864 801 2795 voice & voicemail
103 Autumn Hill Road 864 801 2774 fax
Greer, SC 29651

Do not send me unsolicited commercial email.

[Joe Kaplan (MVP - ADSI)]

Like I said, I can't tell you any more nor do I have any idea how those
other products are doing this. I think you need someone from MS to weigh
in.

You might want to get one of those registered no-spam aliases and try
posting again to see f you can coax an answer out of them.

http://msdn.microsoft.com/newsgroups/managed/

Joe K.

Post by Chuck Chopp

Post by Joe Kaplan (MVP - ADSI)
If you don't want to do change polling, then the only option I'm aware of
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/ad/ad/change_notifications_in_active_directory.asp?frame=true

Yes, that's the stuff straight out of the MSDN Platform SDK... and it's
very much lacking in terms of what I'm wanting to do. Perhaps I'm spoiled
with what can be done w/respect to event notification in eDirectory, but
somehow someway AD has got to have a "native interface" that exposes
better functionality than what's available via LDAP. Even the LDAP
control extensions that AD implements are weak in comparison to the ones
implemented by eDirectory.

Post by Joe Kaplan (MVP - ADSI)
Note that MS warns against using these on an entire naming context due to
performance problems, especially on big DCs. It appears that they really
want you to use one of the polling-based approaches (dirsync or
usnChanged), even though you have ruled that out.

I've come to the conclusion that Microsoft has very little faith in its
own directory services product compared to what Novell does with
eDirectory. It's kind of like the difference between a Fisher Price
toddler's piano and a Steinway baby grand piano... one is a toy for
children and the other is a finely tuned professional instrument. That's
not said to start a flame-war, it's simply an observation and vented in
frustration at the lack of documented & supported functionality. For
small tasks, AD works just fine, but for large scale industrial-strength
directory-enabled applications, MS seems to be hesitant in terms of what
AD will be capable of doing.

Post by Joe Kaplan (MVP - ADSI)
If anyone else has heard of anything, I'd be interested in hearing about it.

The LDAP method mentioned in the Platform SDK doesn't provide the
granularity I'm looking for, nor do either of the polling methods.
Specifically, I need to know if the change is due to object creation,
deletion, rename, move or is just a generic modification of the object's
attributes. And, if it's modified attributes, I need to know the before &
after attribute values for single valued attributes, and, for multi-valued
attributes, I need to know the individual value in the list that was
modified and whether the value was added to or removed from the list.
http://wm.quest.com/Library/getDocument.asp?target=cmadpds
http://www.bi101.net/products/solutions/netpro/
http://www.netpro.com/products/changeauditor/index.cfm
These products are all making claims of auditing AD events and offering a
fine level of granularity in the changes w/o making use of any of the
built-in auditing mechanisms. The functionality they describe cannot be
achieved using DirSync or LDAP as far as I know, so that leaves me with
the thought that they are using some *other* interface into Active
Directory. It's that *other* interface that I'm interested in learning
about. Given that there's more than one product doing this, I'm guessing
that they all work in a similar manner using the same interface into AD.
The alternative is that they're maintaining private replicas of AD
information, and that's a grossly inefficient method that wouldn't perform
nearly as well how these products are supposed to be performing.
Maybe there's a means of hooking into the replication interfaces in AD.
If I could reliably hook into AD in that manner then I could intercept
every single piece of replication traffic and *that* would allow me to
obtain the desired information in real-time, or at least as close to
real-time as the replication schedule allows for. It would eliminate
polling and it would certainly allow me to directly observe in very fine
detail exactly what is going on in AD.
Another possibility is the thinly documented event tracing facility.
There's a very vague reference to in in the Platform SDK in connection
with AD, but there's nothing of substance in the docs to indicate if I'm
heading in the right direction with the thoughts of trying to track down
an event source that will provide the desired event information.
--
Chuck Chopp
ChuckChopp (at) rtfmcsi (dot) com http://www.rtfmcsi.com
RTFM Consulting Services Inc. 864 801 2795 voice & voicemail
103 Autumn Hill Road 864 801 2774 fax
Greer, SC 29651
Do not send me unsolicited commercial email.

[Chuck Chopp]

Post by Joe Kaplan (MVP - ADSI)
Like I said, I can't tell you any more nor do I have any idea how those
other products are doing this. I think you need someone from MS to weigh
in.
You might want to get one of those registered no-spam aliases and try
posting again to see f you can coax an answer out of them.
http://msdn.microsoft.com/newsgroups/managed/

It'd take opening a support incident, I think, as the depth of knowledge I'm
looking for is turning out not be likely to be found out on Usenet or the
web. What I've learned about the commercial auditing & change reporting
products that I referenced is that they are using unsupported methods to
directly tap into AD through the use of hooks that allow internal AD
functions to be intercepted. They do not make use of any of the documented
& supported methods for obtaining AD change notification.
--
Chuck Chopp

ChuckChopp (at) rtfmcsi (dot) com http://www.rtfmcsi.com

RTFM Consulting Services Inc. 864 801 2795 voice & voicemail
103 Autumn Hill Road 864 801 2774 fax
Greer, SC 29651

Do not send me unsolicited commercial email.

[Rhett Gong [MSFT]]

Hi Chuck,
Sorry for the delay.
For this issue, I have confirmed with our AD guys, current AD interface
doesn't have an attribute type granularity - like whether it is single /
multi-value. If your product needs small granularity monitoring, I suggest
you submit an advisory case for further assistance.

Thanks,
Rhett Gong [MSFT]
Microsoft Online Partner Support
Get Secure! - www.microsoft.com/security
http://support.microsoft.com/default.aspx?scid=/servicedesks/msdn/nospam.asp
&SD=msdn

This posting is provided "AS IS" with no warranties and confers no rights.

[Joe Richards [MVP]]

Actually the dirsync control is a replication based control. It will not show
you all changes, only changes that would replicate. With it you sip from the
firehose and sort out what you need from it and realize that local
non-replicating changes will not be available through it.

The LDAP event notification is really not meant for monitoring all of AD but for
watching changes on specific pieces of AD just like the registry event
notification is really for watching specific pieces of the registry versus the
entire thing.

Last time I talked to the NetPro guys they were doing at least some of the work
with Event Tracing so your best bet would be to dig into that more. It isn't a
popular subject so you aren't likely to find much info. Those who have done it
are those who are selling products and will obviously be a bit slow to provide
source code or details. It wouldn't make sense if they spent money figuring it
out and then just handed it over to anyone asking.

I can't really speak to the what eDir can do versus what AD can do. It really
isn't relavent, we are talking about AD, not eDir. All of the complaints about
what one has over the other isn't going to change either nor make anything work.
If you need specific functionality out of AD, the mechanism is to submit a DCR
to Microsoft for the change through PSS. Expect that if there isn't a good
number of similar requests, it will most likely be dropped.

joe


--
Joe Richards Microsoft MVP Windows Server Directory Services
www.joeware.net

Post by Chuck Chopp

Post by Joe Kaplan (MVP - ADSI)
If you don't want to do change polling, then the only option I'm aware
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/ad/ad/change_notifications_in_active_directory.asp?frame=true

Yes, that's the stuff straight out of the MSDN Platform SDK... and it's
very much lacking in terms of what I'm wanting to do. Perhaps I'm
spoiled with what can be done w/respect to event notification in
eDirectory, but somehow someway AD has got to have a "native interface"
that exposes better functionality than what's available via LDAP. Even
the LDAP control extensions that AD implements are weak in comparison to
the ones implemented by eDirectory.

Post by Joe Kaplan (MVP - ADSI)
Note that MS warns against using these on an entire naming context due
to performance problems, especially on big DCs. It appears that they
really want you to use one of the polling-based approaches (dirsync or
usnChanged), even though you have ruled that out.

I've come to the conclusion that Microsoft has very little faith in its
own directory services product compared to what Novell does with
eDirectory. It's kind of like the difference between a Fisher Price
toddler's piano and a Steinway baby grand piano... one is a toy for
children and the other is a finely tuned professional instrument.
That's not said to start a flame-war, it's simply an observation and
vented in frustration at the lack of documented & supported
functionality. For small tasks, AD works just fine, but for large scale
industrial-strength directory-enabled applications, MS seems to be
hesitant in terms of what AD will be capable of doing.

Post by Joe Kaplan (MVP - ADSI)
If anyone else has heard of anything, I'd be interested in hearing about it.

The LDAP method mentioned in the Platform SDK doesn't provide the
granularity I'm looking for, nor do either of the polling methods.
Specifically, I need to know if the change is due to object creation,
deletion, rename, move or is just a generic modification of the object's
attributes. And, if it's modified attributes, I need to know the before
& after attribute values for single valued attributes, and, for
multi-valued attributes, I need to know the individual value in the list
that was modified and whether the value was added to or removed from the
list.
http://wm.quest.com/Library/getDocument.asp?target=cmadpds
http://www.bi101.net/products/solutions/netpro/
http://www.netpro.com/products/changeauditor/index.cfm
These products are all making claims of auditing AD events and offering
a fine level of granularity in the changes w/o making use of any of the
built-in auditing mechanisms. The functionality they describe cannot be
achieved using DirSync or LDAP as far as I know, so that leaves me with
the thought that they are using some *other* interface into Active
Directory. It's that *other* interface that I'm interested in learning
about. Given that there's more than one product doing this, I'm
guessing that they all work in a similar manner using the same interface
into AD. The alternative is that they're maintaining private replicas
of AD information, and that's a grossly inefficient method that wouldn't
perform nearly as well how these products are supposed to be performing.
Maybe there's a means of hooking into the replication interfaces in AD.
If I could reliably hook into AD in that manner then I could intercept
every single piece of replication traffic and *that* would allow me to
obtain the desired information in real-time, or at least as close to
real-time as the replication schedule allows for. It would eliminate
polling and it would certainly allow me to directly observe in very fine
detail exactly what is going on in AD.
Another possibility is the thinly documented event tracing facility.
There's a very vague reference to in in the Platform SDK in connection
with AD, but there's nothing of substance in the docs to indicate if I'm
heading in the right direction with the thoughts of trying to track down
an event source that will provide the desired event information.

[Chuck Chopp]

Post by Joe Richards [MVP]
Last time I talked to the NetPro guys they were doing at least some of
the work with Event Tracing so your best bet would be to dig into that
more. It isn't a popular subject so you aren't likely to find much info.
Those who have done it are those who are selling products and will
obviously be a bit slow to provide source code or details. It wouldn't
make sense if they spent money figuring it out and then just handed it
over to anyone asking.

LOL - I know - At this point I'm simply trying to gather information for
purposes of doing further research into the subject. I've been able to rule
out a significant number of possible of avenues of research, so the scope is
being narrowed down to something manageable.

Post by Joe Richards [MVP]
I can't really speak to the what eDir can do versus what AD can do. It
really isn't relavent, we are talking about AD, not eDir. All of the

The relevance is relative, perhaps? In this case... I'm dealing with a port
of code that was originally written in the Novell NDS/eDir environment and
I'm trying to find equivalent functionality in AD in terms of event
monitoring. I'd say the differences between the two directory services are
very relevant in terms of the feasibility of making the port successful.
And so I keep on with the research....
--
Chuck Chopp

ChuckChopp (at) rtfmcsi (dot) com http://www.rtfmcsi.com

RTFM Consulting Services Inc. 864 801 2795 voice & voicemail
103 Autumn Hill Road 864 801 2774 fax
Greer, SC 29651

Do not send me unsolicited commercial email.

[r***@gmail.com]

Hi , Please have a look at the third party tool such a

Lepide Auditor for Active Directory. This tool will enable you to give you a customize real time report and moreover letting you know who all has accessed the active directory at what time.

This tool will help your time and moreover help you to a get a customize report fpr the same.

Please download the tool from the given link and hope it helps you a lot.

http://www.lepide.com/active-directory-audit/

Thanks.

Post by Chuck Chopp
I'm interested in identifying the programming interfaces used for real-time
auditing of changes to objects in Active Directory. The LDAP uSNChanged and
DirSync ADSI control are not sufficient for what I'm doing, and modifying
the object security and DC security policies to turn on security auditing is
not a viable alternative, either.
There are existing products, such as Quest's "Quest Change Manager for
Active Directory" that appear to be collecting the same time of real-time
information that I'm looking for, and Quest claims to be doing so w/o using
the native audit log features of Windows when doing so.
In a nutshell, I need to receive notifications on a DC whenever an event of
interest happens within the domain or any of its child containers that the
DC contains in its replica of its portion of the tree. I would prefer to
register to receive notification of only the events I'm interested in, but
if I have to receive all events and evaluate them that's OK, too.
Object creation
Object deletion
Object modification [excluding DN changes]
Object rename/move
In the case of object modification, I need to know what attribute was
changed, what the previous value was and what the new value is, or, if it is
a multi-valued attribute, I need to know what the particular value is that
was added to or removed from the list along with the actual add/remove value
operation being identified.
I've gone over the Platform SDK docs [updated for Windows Server 2003 SP1]
and I'm not seeing *anything* even remotely close to what I'm looking for.
However, since there are commercial products on the market that seem to be
obtaining the same type of information, there's got to be some sort of
programming interface with which to obtain the desired information. In the
Novell environment eDirectory [f.k.a. NDS] has a very comprehensive event
monitoring API that can be used to achieve a fine degree of granularity in
terms of the events that can be monitored, and the event notifications can
be delivered via an async callback mechanism.
Is there something obvious or less than obvious that I'm missing? Or, are
these products making use of undocumented interfaces to perform their tasks?
Any assistance would be appreciated.
TIA,
Chuck
--
Chuck Chopp
ChuckChopp (at) rtfmcsi (dot) com http://www.rtfmcsi.com
RTFM Consulting Services Inc. 864 801 2795 voice & voicemail
103 Autumn Hill Road 864 801 2774 fax
Greer, SC 29651
Do not send me unsolicited commercial email.