More than 1000 objects?

Discussion:

(too old to reply)

Timo

2004-12-26 11:51:02 UTC

Hi there,

I try to get all users or groups out of directory. I'am using c# for that.
But we have more than 1000 objects. How can I query all my objects in c#? Can
someone give me a hint or better a code snipe for the property I need to set.

Thanks
Timo

Florian Frommherz

2004-12-26 15:12:16 UTC

Permalink

Hi Timo,

Post by Timo
I try to get all users or groups out of directory. I'am using c# for that.
But we have more than 1000 objects. How can I query all my objects in c#? Can

Using the LDAP-Prodiver, it would work like this:

<snip>

using System;
using System.DirectoryServices;
[...]
DirectoryEntry myDEntry = new DirectoryEntry("LDAP://DC=yourdomain,DC=tld");
DirectorySearcher mySearcher = new DirectorySearcher(myDEntry);
mySearcher.SearchScope = SearchScope.Subtree;
mySearcher.Filter="(objectClass=User)"; // or: (objectClass=Group)
foreach(SearchResult res in mySearcher.FindAll())
{
Console.WriteLine(res.Path.ToString());
}
mySearcher.Close();

<snap>

cheers,

Florian

--
Nachwuchsadmin aus dem Süddeutschen.
Mitglied der Frickel-Fraktion.
eMail: Mein Vorname [bei] frickelsoft [Punkt] net.

Timo

2004-12-26 17:31:02 UTC

Permalink

Sorry, this do not work

try with more than 1000 objects and you will see.

Server 2003 is I think with 1300 objects. There is a special property what
is necessary to set but I did not found.

Greetings
Timo

Post by Florian Frommherz
Hi Timo,

Post by Timo
I try to get all users or groups out of directory. I'am using c# for that.
But we have more than 1000 objects. How can I query all my objects in c#? Can

<snip>
using System;
using System.DirectoryServices;
[...]
DirectoryEntry myDEntry = new DirectoryEntry("LDAP://DC=yourdomain,DC=tld");
DirectorySearcher mySearcher = new DirectorySearcher(myDEntry);
mySearcher.SearchScope = SearchScope.Subtree;
mySearcher.Filter="(objectClass=User)"; // or: (objectClass=Group)
foreach(SearchResult res in mySearcher.FindAll())
{
Console.WriteLine(res.Path.ToString());
}
mySearcher.Close();
<snap>
cheers,
Florian
--
Nachwuchsadmin aus dem Süddeutschen.
Mitglied der Frickel-Fraktion.
eMail: Mein Vorname [bei] frickelsoft [Punkt] net.

Lee Flight

2004-12-27 00:07:10 UTC

Permalink

Hi

you need to set the PageSize property of your DirectorySearcher
to a valid value (1 to 1000) to return all of the results.

Lee Flight

Post by Timo
Sorry, this do not work
try with more than 1000 objects and you will see.
Server 2003 is I think with 1300 objects. There is a special property what
is necessary to set but I did not found.
Greetings
Timo

Post by Florian Frommherz
Hi Timo,

Post by Timo
I try to get all users or groups out of directory. I'am using c# for that.
But we have more than 1000 objects. How can I query all my objects in c#? Can

<snip>
using System;
using System.DirectoryServices;
[...]
DirectoryEntry myDEntry = new
DirectoryEntry("LDAP://DC=yourdomain,DC=tld");
DirectorySearcher mySearcher = new DirectorySearcher(myDEntry);
mySearcher.SearchScope = SearchScope.Subtree;
mySearcher.Filter="(objectClass=User)"; // or: (objectClass=Group)
foreach(SearchResult res in mySearcher.FindAll())
{
Console.WriteLine(res.Path.ToString());
}
mySearcher.Close();
<snap>
cheers,
Florian
--
Nachwuchsadmin aus dem Süddeutschen.
Mitglied der Frickel-Fraktion.
eMail: Mein Vorname [bei] frickelsoft [Punkt] net.

Timo

2004-12-27 09:07:01 UTC

Permalink

Well, but when I set to 1000 than I get only 1000 objects back.
But what do I do with 2000 objects? And I know if I set a higher value to
the pagesize property than the standard value of 1000 will be used.
You see my trouble?

Greetings
Timo

Post by Florian Frommherz
Hi Timo,

Post by Timo
I try to get all users or groups out of directory. I'am using c# for that.
But we have more than 1000 objects. How can I query all my objects in c#? Can

Rohit Sinha

2004-12-27 14:39:18 UTC

Permalink

It seems that you are getting incomplete recordset with Directory Searcher
class
It looks like the same issue we faced before i.e. Microsoft .NET Framework
1.1 Lightweight Directory Access Protocol (LDAP) Client that uses the .NET
DirectorySearcher class to query the Active Directory directory service may
receive an incomplete result set . If this is the problem you are facing
then it is a Known Issue with Directory Searcher class.
Following Microsoft Knowledge Base article gives you more details about
this <http://support.microsoft.com/default.aspx?scid=kb;en-us;833789>.
In order to get the complete results with your current .net installation,
user should catch this exception(refer to the above KB article) and call
MoveNext to retrieve the remaining results. A code snippet looks like:

bool moreResult = false;
SearchResultCollection results = srch.FindAll();
IEnumerator enumerator = results.GetEnumerator();

while(true)
{
try
{
moreResult = enumerator.MoveNext();
}
catch(COMException e)
{
if(e.ErrorCode == unchecked((int)0x800700EA))
continue;
else
throw e;
}

if(moreResult)
{
SearchResult result = (SearchResult) enumerator.Current;
Console.WriteLine(result.Properties["name"][0]);
}
else
break;

}

We also provide a configuration file setting option, that is to say, if
user specifies the following configuration either in app config or machine
config, it will reissue the request and retrieve the results for the user,
so user could continue using the foreach syntax.

<?xml version="1.0" encoding="UTF-8" ?>
<configuration>
<configSections>
<section name="system.directoryservices"
type="System.DirectoryServices.SearchWaitHandler, System.DirectoryServices,
Version=1.0.5000.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" />
</configSections>
<system.directoryservices>
<DirectorySearcher waitForPagedSearchData="true" /> // the value
for
waitForPagedSearchData could be either true or false
</system.directoryservices>
</configuration>

Please see if it helps?

Rohit Sinha[MSFT]
Microsoft Developer Support

Disclaimer: This posting is provided "AS IS" with no warranties, and
confers no rights. You assume all risk for your use.

Timo

2005-01-05 19:51:07 UTC

Permalink

I tried the code part but I get 1000 objects back :(. I do really not
understand why.

Timo

Post by Rohit Sinha
It seems that you are getting incomplete recordset with Directory Searcher
class
It looks like the same issue we faced before i.e. Microsoft .NET Framework
1.1 Lightweight Directory Access Protocol (LDAP) Client that uses the .NET
DirectorySearcher class to query the Active Directory directory service may
receive an incomplete result set . If this is the problem you are facing
then it is a Known Issue with Directory Searcher class.
Following Microsoft Knowledge Base article gives you more details about
this <http://support.microsoft.com/default.aspx?scid=kb;en-us;833789>.
In order to get the complete results with your current .net installation,
user should catch this exception(refer to the above KB article) and call
bool moreResult = false;
SearchResultCollection results = srch.FindAll();
IEnumerator enumerator = results.GetEnumerator();
while(true)
{
try
{
moreResult = enumerator.MoveNext();
}
catch(COMException e)
{
if(e.ErrorCode == unchecked((int)0x800700EA))
continue;
else
throw e;
}
if(moreResult)
{
SearchResult result = (SearchResult) enumerator.Current;
Console.WriteLine(result.Properties["name"][0]);
}
else
break;
}
We also provide a configuration file setting option, that is to say, if
user specifies the following configuration either in app config or machine
config, it will reissue the request and retrieve the results for the user,
so user could continue using the foreach syntax.
<?xml version="1.0" encoding="UTF-8" ?>
<configuration>
<configSections>
<section name="system.directoryservices"
type="System.DirectoryServices.SearchWaitHandler, System.DirectoryServices,
Version=1.0.5000.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" />
</configSections>
<system.directoryservices>
<DirectorySearcher waitForPagedSearchData="true" /> // the value
for
waitForPagedSearchData could be either “true” or “false”
</system.directoryservices>
</configuration>
Please see if it helps?
Rohit Sinha[MSFT]
Microsoft Developer Support
Disclaimer: This posting is provided "AS IS" with no warranties, and
confers no rights. You assume all risk for your use.

Joe Kaplan (MVP - ADSI)

2005-01-05 20:39:48 UTC

Permalink

What does your code look like now?

Joe K.

Post by Timo
I tried the code part but I get 1000 objects back :(. I do really not
understand why.
Timo

Timo

2005-01-05 21:29:24 UTC

Permalink

Really simple

try
{
DirectoryEntry dirEntry = new DirectoryEntry("LDAP://" + strRootOU);
DirectorySearcher dirSearcher = new DirectorySearcher(dirEntry);
dirSearcher.Filter = "(&(objectClass=user)(objectCategory=person))";
dirSearcher.PropertiesToLoad.Add("distinguishedName");
dirSearcher.SearchScope = scope;
dirSearcher.Sort.Direction = sortDir;

SearchResultCollection collResult = dirSearcher.FindAll();

if (collResult != null)
{
foreach(SearchResult result in collResult)
{
Console.WriteLine(result.Properties["distinguishedName"][0].ToString());
}
}

dirEntry.Close();
}
catch (COMException e)
{
throw e;
}
catch (Exception e)
{
throw e;
}

Greetings
Timo

Post by Joe Kaplan (MVP - ADSI)
What does your code look like now?
Joe K.

Post by Timo
I tried the code part but I get 1000 objects back :(. I do really not
understand why.
Timo

Joe Kaplan (MVP - ADSI)

2005-01-05 21:34:56 UTC

Permalink

If you don't set the PageSize to something other than 0 (such as 1000), you
won't do a paged search and won't get more than 1000 results.

dirSearcher.PageSize = 1000

The post you replied to was showing a special setting you can use with the
more recent service packs for .NET that helps fix an issue with paged
searches that occurs pretty rarely in some weird timeout situations. You
mostly likely won't need that.

Joe K.

Post by Timo
Really simple
try
{
DirectoryEntry dirEntry = new DirectoryEntry("LDAP://" + strRootOU);
DirectorySearcher dirSearcher = new DirectorySearcher(dirEntry);
dirSearcher.Filter = "(&(objectClass=user)(objectCategory=person))";
dirSearcher.PropertiesToLoad.Add("distinguishedName");
dirSearcher.SearchScope = scope;
dirSearcher.Sort.Direction = sortDir;
SearchResultCollection collResult = dirSearcher.FindAll();
if (collResult != null)
{
foreach(SearchResult result in collResult)
{
Console.WriteLine(result.Properties["distinguishedName"][0].ToString());
}
}
dirEntry.Close();
}
catch (COMException e)
{
throw e;
}
catch (Exception e)
{
throw e;
}
Greetings
Timo

Post by Joe Kaplan (MVP - ADSI)
What does your code look like now?
Joe K.

Post by Timo
I tried the code part but I get 1000 objects back :(. I do really not
understand why.
Timo

Timo

2005-01-05 22:43:03 UTC

Permalink

Thanks a lot. This works. I knowed about this property in general but I never
used ;). What do suprise me that is necessary to set to 1000 if I want to get
more than 1000 elements. This is a strange construct.
I do understand to limit the query. But why is it not possible to set for
example 5000 or whatever to maximize the query. For example -1 for querying
all elements.

Whatever thank you a lot.

Greetings
Timo

Post by Joe Kaplan (MVP - ADSI)
If you don't set the PageSize to something other than 0 (such as 1000), you
won't do a paged search and won't get more than 1000 results.
dirSearcher.PageSize = 1000
The post you replied to was showing a special setting you can use with the
more recent service packs for .NET that helps fix an issue with paged
searches that occurs pretty rarely in some weird timeout situations. You
mostly likely won't need that.
Joe K.

Post by Joe Kaplan (MVP - ADSI)
What does your code look like now?
Joe K.

Post by Timo
I tried the code part but I get 1000 objects back :(. I do really not
understand why.
Timo

Joe Kaplan (MVP - ADSI)

2005-01-05 22:51:51 UTC

Permalink

Excellent question.

Actually, by setting it to 1000, you are setting it to the maximum page size
for your domain. That can be changed using ntdsutil and is actually 1500 by
default for Windows 2003 native AD.

If it is set to 0, a non-paged search is executed. You can actually set the
page size to anything between 1 and the max page size. 1000 is pretty
standard.

AD implements paging to help with resource utilization on the DC. By
returning only 1000 results by default, huge numbers of server resources are
not accidentally consumed by naive searches. The net result is that you
have to do a little more work, but your AD infrastructure is hopefully more
reliable this way.

I agree that the API design is kind of weird and non-intuitive. It is what
it is and we are stuck with it. At least we have an API that supports paged
searches though. Some LDAP stacks don't and they are SOL unless they change
the maxPageSize in AD which is generally regarded as a bad idea.

Joe K.

Post by Timo
Thanks a lot. This works. I knowed about this property in general but I never
used ;). What do suprise me that is necessary to set to 1000 if I want to get
more than 1000 elements. This is a strange construct.
I do understand to limit the query. But why is it not possible to set for
example 5000 or whatever to maximize the query. For example -1 for querying
all elements.
Whatever thank you a lot.
Greetings
Timo

Post by Joe Kaplan (MVP - ADSI)
What does your code look like now?
Joe K.

Post by Timo
I tried the code part but I get 1000 objects back :(. I do really not
understand why.
Timo

Timo

2005-01-05 23:19:02 UTC

Permalink

You wrote...
Actually, by setting it to 1000, you are setting it to the maximum page size
for your domain. That can be changed using ntdsutil and is actually 1500 by
default for Windows 2003 native AD.
....

What does this mean. I need to reset with ntdsutil the property to maximize
my query. We are using Windows 2003 and in general we do not want to exchange
the setting. We need that only for special quering within the domain.
So if I set the pagesize property to 1000 I get the maximum value which is
set within the domain? Or do I get really all elements for what I make my
query.

Well, ADSI is not every time intuitive :D.

Greetings
Timo

Post by Joe Kaplan (MVP - ADSI)
Excellent question.
Actually, by setting it to 1000, you are setting it to the maximum page size
for your domain. That can be changed using ntdsutil and is actually 1500 by
default for Windows 2003 native AD.
If it is set to 0, a non-paged search is executed. You can actually set the
page size to anything between 1 and the max page size. 1000 is pretty
standard.
AD implements paging to help with resource utilization on the DC. By
returning only 1000 results by default, huge numbers of server resources are
not accidentally consumed by naive searches. The net result is that you
have to do a little more work, but your AD infrastructure is hopefully more
reliable this way.
I agree that the API design is kind of weird and non-intuitive. It is what
it is and we are stuck with it. At least we have an API that supports paged
searches though. Some LDAP stacks don't and they are SOL unless they change
the maxPageSize in AD which is generally regarded as a bad idea.
Joe K.

Post by Joe Kaplan (MVP - ADSI)
What does your code look like now?
Joe K.

Post by Timo
I tried the code part but I get 1000 objects back :(. I do really not
understand why.
Timo

Joe Kaplan (MVP - ADSI)

2005-01-06 03:27:56 UTC

Permalink

Let me try this again. I don't think I was very clear.

Normally, when you do an LDAP search against AD, you don't do a paged
search. In that case, AD will return up to the maxPageSize number of
results. When you request a paged search, AD will return search results in
pages of a size you specify up to the maxPageSize until all matching results
are returned.

With the LDAP API, paged searches are manual in that you have to keep
requesting additional pages. However, with ADSI/S.DS, you basically just
"turn on" paging by specifying a page size. ADSI does all the extra for you
and returns all the matching results.

So essentially, you don't need to change the maxPageSize. If you need more
results than what it is set to, you just need to do a paged search as we
already discussed. I was just trying to explain where the numbers came from
and why AD implements paging.

Joe K.

Post by Timo
You wrote...
Actually, by setting it to 1000, you are setting it to the maximum page size
for your domain. That can be changed using ntdsutil and is actually 1500 by
default for Windows 2003 native AD.
....
What does this mean. I need to reset with ntdsutil the property to maximize
my query. We are using Windows 2003 and in general we do not want to exchange
the setting. We need that only for special quering within the domain.
So if I set the pagesize property to 1000 I get the maximum value which is
set within the domain? Or do I get really all elements for what I make my
query.
Well, ADSI is not every time intuitive :D.
Greetings
Timo

Post by Timo
Thanks a lot. This works. I knowed about this property in general but I never
used ;). What do suprise me that is necessary to set to 1000 if I want
to
get
more than 1000 elements. This is a strange construct.
I do understand to limit the query. But why is it not possible to set for
example 5000 or whatever to maximize the query. For example -1 for querying
all elements.
Whatever thank you a lot.
Greetings
Timo

Post by Joe Kaplan (MVP - ADSI)
If you don't set the PageSize to something other than 0 (such as
1000),
you
won't do a paged search and won't get more than 1000 results.
dirSearcher.PageSize = 1000
The post you replied to was showing a special setting you can use with the
more recent service packs for .NET that helps fix an issue with paged
searches that occurs pretty rarely in some weird timeout situations.
You
mostly likely won't need that.
Joe K.

Post by Joe Kaplan (MVP - ADSI)
What does your code look like now?
Joe K.

Post by Timo
I tried the code part but I get 1000 objects back :(. I do really not
understand why.
Timo

Timo

2005-01-06 08:15:02 UTC

Permalink

Thanks again,

this was really clear.

Greetings
Timo

Post by Joe Kaplan (MVP - ADSI)
Let me try this again. I don't think I was very clear.
Normally, when you do an LDAP search against AD, you don't do a paged
search. In that case, AD will return up to the maxPageSize number of
results. When you request a paged search, AD will return search results in
pages of a size you specify up to the maxPageSize until all matching results
are returned.
With the LDAP API, paged searches are manual in that you have to keep
requesting additional pages. However, with ADSI/S.DS, you basically just
"turn on" paging by specifying a page size. ADSI does all the extra for you
and returns all the matching results.
So essentially, you don't need to change the maxPageSize. If you need more
results than what it is set to, you just need to do a paged search as we
already discussed. I was just trying to explain where the numbers came from
and why AD implements paging.
Joe K.

Post by Timo
Thanks a lot. This works. I knowed about this property in general but I never
used ;). What do suprise me that is necessary to set to 1000 if I want
to
get
more than 1000 elements. This is a strange construct.
I do understand to limit the query. But why is it not possible to set for
example 5000 or whatever to maximize the query. For example -1 for querying
all elements.
Whatever thank you a lot.
Greetings
Timo

Post by Joe Kaplan (MVP - ADSI)
If you don't set the PageSize to something other than 0 (such as
1000),
you
won't do a paged search and won't get more than 1000 results.
dirSearcher.PageSize = 1000
The post you replied to was showing a special setting you can use with the
more recent service packs for .NET that helps fix an issue with paged
searches that occurs pretty rarely in some weird timeout situations.
You
mostly likely won't need that.
Joe K.

Post by Joe Kaplan (MVP - ADSI)
What does your code look like now?
Joe K.

Post by Timo
I tried the code part but I get 1000 objects back :(. I do really not
understand why.
Timo

Marc Scheuner [MVP ADSI]

2004-12-28 06:55:36 UTC

Permalink

Post by Timo
Well, but when I set to 1000 than I get only 1000 objects back.

No - you will get back PACKETS of 1000 objects - as many as needed.

If you don't set the page size, you'll hit a limit at about 1000
objects - but if you set the page size, to make AD do "paged
searches", then you'll get back ALL your objects requested.

Marc
________________________________________________________________
Marc Scheuner ** mscheuner -at- mvps.org ** http://adsi.mvps.org
Microsoft MVP for Directory Services Programming
http://www.dirteam.com/blogs/mscheuner/default.aspx
Come see http://groups.yahoo.com/group/ADSIANDDirectoryServices/

Timo

2005-01-05 22:38:37 UTC

Permalink

Thanks a lot

Timo

Post by Marc Scheuner [MVP ADSI]

Post by Timo
Well, but when I set to 1000 than I get only 1000 objects back.

No - you will get back PACKETS of 1000 objects - as many as needed.
If you don't set the page size, you'll hit a limit at about 1000
objects - but if you set the page size, to make AD do "paged
searches", then you'll get back ALL your objects requested.
Marc
________________________________________________________________
Marc Scheuner ** mscheuner -at- mvps.org ** http://adsi.mvps.org
Microsoft MVP for Directory Services Programming
http://www.dirteam.com/blogs/mscheuner/default.aspx
Come see http://groups.yahoo.com/group/ADSIANDDirectoryServices/

Continue reading on narkive:

Search results for 'More than 1000 objects?' (Questions and Answers)

replies

Object moves away from stationary object at speed of light?

started 2006-07-20 17:13:59 UTC

physics

replies

Where is the Object in the Hello World program?

started 2012-07-05 17:47:43 UTC

programming & design

replies

object oriented DBMS?