issues
search
c4software
/
python-sitemap
Mini website crawler to make sitemap from a website.
GNU General Public License v3.0
362
stars
110
forks
source link
issues
Newest
Newest
Most commented
Recently updated
Oldest
Least commented
Least recently updated
--parserobots doesn't seem to work
#93
varna9000
opened
1 month ago
0
Include iframe contents
#92
marshvee
opened
4 months ago
0
Client-side rendered SPAs don't work
#91
Garrett-R
opened
4 months ago
0
iframe contents ignored
#90
Garrett-R
opened
5 months ago
0
Add option to specify bot (user-agent) for robots.txt
#89
marshvee
closed
5 months ago
1
Sort URLs alphabetically
#88
marshvee
closed
5 months ago
3
Option to specify bot for robots.txt
#87
Garrett-R
closed
5 months ago
2
How to exclude noindex page
#86
damarkuzz
opened
9 months ago
0
Alphabetize URLs
#85
Garrett-R
closed
5 months ago
1
crawling depth setting
#84
styxiik
closed
1 year ago
0
How to add hreflang tags
#83
crossmania337
opened
2 years ago
2
Windows and/or Python 3.7.2?
#82
VVilku
closed
2 years ago
2
Updated the string format method
#81
cyai
closed
2 years ago
1
urls not saved to sitemap.xml
#80
crossmania337
closed
2 years ago
2
URL UnicodeEncodeError
#79
wkingnet
opened
2 years ago
0
Exclude canonicalized pages
#78
Spidle
opened
2 years ago
0
fix(crawler): add condition for not parseable content
#77
ChenKuanSun
closed
2 years ago
0
Python 3.9.6 support? SyntaxError
#76
PeterWoelfel
opened
3 years ago
5
AttributeError: 'NoneType' object has no attribute 'geturl'
#75
devopsenko
opened
3 years ago
4
Exclude redirects from sitemap
#74
Garrett-R
closed
3 years ago
2
Fixed indentation in crawler.py
#73
rstular
closed
3 years ago
1
Feature Request: Limit per category/section the number of URLs to parse
#72
Veilkrand
opened
3 years ago
0
Update README.md
#71
Rajeev-Kumar-DSA
closed
3 years ago
3
Fixes skipping pages accessed with ?p=
#70
reuning
closed
3 years ago
2
Improve Docs
#69
raghavkumarbhatia53
closed
3 years ago
1
Update LICENSE.txt
#68
singharsh0
closed
3 years ago
0
RuntimeError: Event loop is closed - with > 1 workers
#66
dpatz
closed
3 years ago
6
Add support for sitemap index
#65
jswilson
closed
3 years ago
4
Add package to PyPI
#64
Garrett-R
opened
4 years ago
5
BUG: remove race condition in multithreading
#63
Garrett-R
closed
4 years ago
0
Only limit to same domain, not same subdomain
#62
Garrett-R
closed
3 years ago
0
add basic auth to enable crawling of password protected sites
#61
LoveBootCaptain
closed
4 years ago
3
Handling more than 50,000 URLs
#59
jswilson
opened
4 years ago
4
Stop and continue
#58
ishandutta2007
opened
4 years ago
1
No URLs found
#57
exportio
opened
5 years ago
7
Fixed handling of relative URLs
#56
mnlipp
closed
5 years ago
3
Fix space and tabs with pycharm software
#55
cavazquez
closed
4 years ago
0
Change name project
#54
cavazquez
closed
5 years ago
2
sintax error
#53
francisco-baptista
closed
5 years ago
1
Added a rate limiter for load reduction on the website
#52
Bash-
opened
5 years ago
7
Fix print bug in crawler report generator
#51
todpole3
closed
5 years ago
0
Limit search to path instead of domain?
#50
1kastner
opened
5 years ago
5
Add multithread option
#49
Garrett-R
closed
5 years ago
3
Relative URLs are parsed incorrectly
#48
ghost
opened
6 years ago
2
Please move the project away from GitHub
#47
ghost
closed
6 years ago
2
Error: No space left on device
#46
FranciscoPaixao
closed
5 years ago
3
Video sitemap
#45
FranciscoPaixao
closed
6 years ago
2
Working with Angular sites?
#44
tspicer
closed
5 years ago
1
Adding trailing '/' to all URLs
#43
dgursh
closed
6 years ago
1
Ignore possible errors in UTF-8 encoding
#42
ghost
closed
6 years ago
1
Next