Sitemap collection stops inexplicably

[GoogleCodeExporter]

What steps will reproduce the problem?

1. Normal install on https psswd protected
2. set for web filter only, speed up collection and save settings, nothing
clever here include everything for now
3. wait

What is the expected output? What do you see instead?

The sitemap grows and the grows some more, but after ~44,000 urls it stops
inexplicably.  There appears to be no obvious error.  Tried to delete and
start again (probably not the right way?) ...

What version of the product are you using? On what operating system?

Latest

Please provide any additional information below.

Large drupal based site with over 250k product pages and then the rest.

Original issue reported on code.google.com by philip.immirzi@gmail.com on 20 Nov 2009 at 5:39

[GoogleCodeExporter]

Installed on Debian Etch with Plesk 9 skulking in the background.

Original comment by philip.immirzi@gmail.com on 20 Nov 2009 at 5:45

[GoogleCodeExporter]

Working correctly.  Somehow I thought the web filter option parsed the site 
content
recursively ... clearly not. :(

Original comment by phi...@gmail.com on 23 Nov 2009 at 7:33