vezaynk Sitemap-Generator-Crawler issues

vezaynk / Sitemap-Generator-Crawler

PHP script to recursively crawl websites and generate a sitemap. Zero dependencies.

https://www.bbss.dev

MIT License

241 stars 92 forks source link

issues

Newest

Newest Most commented Recently updated Oldest Least commented Least recently updated

How to setup page I need ?

#101 nethiker76 closed 1 month ago
1
Does not work with the following website address...

#100 duongkyvuong closed 7 months ago
1
Add a log with links, contains url returns 404

#99 oim37 closed 11 months ago
2
Does not work with Real Estate FLOWFACT Wordpress Plugin generated Object Pages

#98 Zorast opened 2 years ago
3
Update README.md

#97 AndCycle closed 2 years ago
1
[!] Reformatted site from https://example.com/folder/#with_index.php# to https://example.com/

#96 martianmaikel opened 3 years ago
8
What if my website has more then 50,000 pages

#95 Jaydeep-01 closed 3 years ago
1
Specify for IP local sitemap generation

#94 vezaynk opened 3 years ago
0
Could not find files for the given pattern(s)

#93 Jaydeep-01 closed 3 years ago
2
Pls help me

#92 sirstevemedia closed 3 years ago
1
Fails for Maximum allowed length

#91 canuck-sailor opened 3 years ago
5
Is there support for static sites defined locally

#90 KommuSoft closed 3 years ago
4
Reports empty error after finishing

#89 mylselgan closed 3 years ago
1
"mmap failed... cannot allocate memory"

#88 tomjennings closed 3 years ago
2
[Feature Request] List all kinds of files & add an option to not try to curl some extensions

#87 jean-christophe-manciot closed 4 years ago
1
Remove WINNT check for argument processing

#86 jamesjohnmcguire opened 4 years ago
2
only creating one URL where I have 700+

#85 tejalbaria5 closed 4 years ago
7
[logger] set typo to lowercase

#84 akshaydodkade closed 4 years ago
1
Fixed issue 82 - added noindex functionality

#83 wcmohler opened 5 years ago
6
"noindex" URL are listed in sitemap

#82 stephanros opened 5 years ago
12
sitemap.xml access denied can't open

#81 Ganofins closed 5 years ago
4
Empty sitemap close

#80 MattMski closed 5 years ago
0
Option to not crawl Frames

#79 kewh closed 3 years ago
3
<iframe> tags ignored

#78 kewh opened 5 years ago
1
fix for lastmod Issue #76

#77 kewh closed 5 years ago
3
Last modified date always set to current date/time

#76 kewh closed 5 years ago
0
Little problems with &

#75 webapteka closed 6 years ago
1
Please move the project away from GitHub

#74 ghost closed 6 years ago
1
Some things added

#73 brunoabcn closed 6 years ago
2
Crawling the root website instead of sub-root

#72 yashitgarg closed 6 years ago
4
Adds only the main page :(

#71 webapteka closed 6 years ago
1
wrap Class

#70 akiraz2 closed 5 years ago
2
Don't mark redirects as scanned before scanning them

#69 pronobis opened 6 years ago
0
Switch from arrays to hashtables

#68 vezaynk closed 6 years ago
1
Tracking deferred scans

#67 vezaynk closed 6 years ago
1
Blacklist not working

#66 Kristiansky closed 6 years ago
9
Sitemap.xml access denied can't open generated xml file

#65 gomathyfollowon closed 6 years ago
9
banned pagination url

#64 jazuly1 closed 6 years ago
1
INFO: Could not find files for the given pattern(s).

#63 blacksmoke26 closed 6 years ago
2
Memory leak reduction

#62 erandelax closed 6 years ago
1
Reduced memory leak

#61 erandelax closed 6 years ago
0
Output file permissions

#60 vezaynk closed 6 years ago
1
Retrieve and parse header before requesting the full page

#59 vezaynk opened 6 years ago
20
allow to validate non ascii urls (fixes #57)

#58 francisek opened 6 years ago
3
Non-ascii urls fail to validate

#57 mazux opened 6 years ago
18
change way scanned urls are stored

#56 jandanielcz closed 6 years ago
3
Added versioning. Close #45

#55 vezaynk closed 6 years ago
0
Removed cli colors from Windows as it just throws garbage

#54 vezaynk closed 6 years ago
2
fix bug in is_scanned, wierd explode

#53 jandanielcz closed 6 years ago
1
Add the first unit tests

#52 villfa closed 6 years ago
2