search

samibnjd / sitemap-generators

Automatically exported from code.google.com/p/sitemap-generators
0 stars 0 forks source link

Migrated bug: "MemoryError. sitemap_gen-1.4" reported anonymously on 2006-05-31 #5

Open GoogleCodeExporter opened 9 years ago

GoogleCodeExporter commented 9 years ago 
Original bug listed here:
http://sourceforge.net/tracker/index.php?func=detail&aid=1498163&group_id=137793
&atid=739383

# ./sitemap_gen.py --config=forex-rdc.ru.xml
Reading configuration file: forex-rdc.ru.xml
Opened URLLIST file: forex-rdc.ru.txt
Walking DIRECTORY "/usr/local/www/forex-rdc.ru/images/"
Walking DIRECTORY "/usr/local/www/forex-rdc.ru/demobook/"
Walking DIRECTORY "/usr/local/www/forex-rdc.ru/"
Opened ACCESSLOG file:
/var/log/httpd/access_forex-rdc.ru.log
[WARNING] Discarded URL for not starting with the
base_url: http://admin/
[WARNING] Discarded URL for not starting with the
base_url: http://styles.css
[WARNING] Discarded URL for not starting with the
base_url: http://images/predictions_small.gif
[WARNING] Discarded URL for not starting with the
base_url: http://www.example.com
[WARNING] Empty URL
Sorting and normalizing collected URLs.
Writing Sitemap file
"/usr/local/www/forex-rdc.ru/sitemap.xml.gz" with 50000
URLs
Sorting and normalizing collected URLs.
Writing Sitemap file
"/usr/local/www/forex-rdc.ru/sitemap1.xml.gz" with
50000 URLs
Sorting and normalizing collected URLs.
Writing Sitemap file
"/usr/local/www/forex-rdc.ru/sitemap2.xml.gz" with
50000 URLs
Sorting and normalizing collected URLs.
Writing Sitemap file
"/usr/local/www/forex-rdc.ru/sitemap3.xml.gz" with
50000 URLs
Sorting and normalizing collected URLs.
Writing Sitemap file
"/usr/local/www/forex-rdc.ru/sitemap4.xml.gz" with
50000 URLs
Sorting and normalizing collected URLs.
Writing Sitemap file
"/usr/local/www/forex-rdc.ru/sitemap5.xml.gz" with
50000 URLs
Sorting and normalizing collected URLs.
Writing Sitemap file
"/usr/local/www/forex-rdc.ru/sitemap6.xml.gz" with
50000 URLs
Sorting and normalizing collected URLs.
Writing Sitemap file
"/usr/local/www/forex-rdc.ru/sitemap7.xml.gz" with
50000 URLs
Sorting and normalizing collected URLs.
Writing Sitemap file
"/usr/local/www/forex-rdc.ru/sitemap8.xml.gz" with
50000 URLs
Sorting and normalizing collected URLs.
Writing Sitemap file
"/usr/local/www/forex-rdc.ru/sitemap9.xml.gz" with
50000 URLs
Sorting and normalizing collected URLs.
Traceback (most recent call last):
File "./sitemap_gen.py", line 2203, in ?
sitemap.Generate()
File "./sitemap_gen.py", line 1775, in Generate
input.ProduceURLs(self.ConsumeURL)
File "./sitemap_gen.py", line 1137, in ProduceURLs
consumer(url, True)
File "./sitemap_gen.py", line 1843, in ConsumeURL
self.FlushSet()
File "./sitemap_gen.py", line 1862, in FlushSet
url.priority = '%.4f' % (float(dup) /
float(self._dup_max))
MemoryError

Contact me via e-mail: suntechnic@gmail.com

Original issue reported on code.google.com by api.ma...@gmail.com on 13 Aug 2007 at 7:30

GoogleCodeExporter commented 9 years ago 
[deleted comment]
GoogleCodeExporter commented 9 years ago 
Given that you have over 50000 URLs I assume that your accesslog files are big. 
I
have fixed sitemap_gen.py so it now does not consume enormous amount of memory 
and
python does not die with MemoryError. Look here
http://www.bashkirtsev.com/2009/05/14/sitemap/

Original comment by vbashkir...@gmail.com on 14 May 2009 at 3:44

GoogleCodeExporter commented 9 years ago 
http://www.bestpropertyinfo.com

Original comment by bpropert...@gmail.com on 19 May 2011 at 8:57