issues
search
BuilderIO
/
gpt-crawler
Crawl a site to generate knowledge files to create your own custom GPT from a URL
https://www.builder.io/blog/custom-gpt
ISC License
18.15k
stars
1.88k
forks
source link
issues
Newest
Newest
Most commented
Recently updated
Oldest
Least commented
Least recently updated
Change LICENSE from MIT to ISC
#68
justindhillon
closed
7 months ago
2
Fix for Issue #66 | Docker run fails due to "Cannot find module '/home/myuser/dist/main.js'" | Error in Docker container
#67
Daethyra
closed
7 months ago
2
Fix for "Cannot find module '/home/myuser/dist/main.js'" Error in Docker Container
#66
Daethyra
closed
7 months ago
3
Scope of the crawler (limits)
#65
PizBernina
opened
7 months ago
0
Use new `crawler.exportData` helper
#64
B4nan
opened
7 months ago
0
chore: add `!package-lock.json` and `!tsconfig.json` to `.gitignore`
#63
nilwurtz
closed
7 months ago
2
Add gpt-tokenizer package and implement size and token limits
#62
guillermoscript
closed
7 months ago
1
Is it possible to crawl discord messages (via a browser) ?
#61
heyitsmedev
opened
7 months ago
0
add `cross-env` package to avoid Node errors
#60
lelemathrin
closed
7 months ago
2
chore: add `.DS_Store` and `!package.json` to `.gitignore`
#59
nilwurtz
closed
7 months ago
1
Add .gitignore entries for pnpm-lock.yaml
#58
luissuil
closed
7 months ago
1
Error: Object with guid frame@da17bfa5502b55fac55ab6dcc355fabe was not bound in the connection
#57
kyiree
opened
7 months ago
1
feat: CLI release workflow
#56
marcelovicentegc
closed
7 months ago
1
craw openai --get nothing
#55
zyxcambridge
opened
7 months ago
1
Config validation
#54
iperzic
closed
7 months ago
2
How to Choose a Suitable CSS Selector for a Website
#53
kongjining
opened
7 months ago
1
feat: create crawler api server
#52
adityak74
closed
6 months ago
7
Add help file to crawl github repos
#51
zackees
opened
7 months ago
5
modified config.ts to fix containerized execution
#50
Umar-Azam
closed
7 months ago
2
Pr/48
#49
Cooky420
closed
5 months ago
1
feat: add ci for prettier check and build
#48
kunal00000
closed
7 months ago
6
Add CI
#47
steve8708
closed
7 months ago
3
Hi
#46
Kamran010
closed
7 months ago
0
feat: use Xpath as selector
#45
LeonKohli
closed
7 months ago
2
[Feature Request] A Way to Split a Knowledge File into Multiple Files
#44
VoxAndrews
closed
7 months ago
12
Expose the service as a REST API
#43
databill86
closed
6 months ago
6
[Feature Request]Will gpt-crawler support puppeteer ?
#42
yujinqiu
closed
3 months ago
2
chore: remove .DS_Store
#41
vagusX
closed
7 months ago
3
chore: support multiple globs in config
#40
86
closed
7 months ago
2
Adding proxy support?
#39
victorx98
opened
7 months ago
2
Make GPT Crawler a CLI
#38
marcelovicentegc
closed
7 months ago
5
Are you interested in packaging it as a CLI?
#37
marcelovicentegc
closed
7 months ago
3
Added option for simple containerized execution
#36
Umar-Azam
closed
7 months ago
2
Support `startsWith` selector
#35
nikitavoloboev
opened
7 months ago
1
Add a bit more log output for Crawling
#34
gummipunkt
closed
7 months ago
3
Selector scrape by type
#33
LcCompany
opened
7 months ago
0
How to crawl sites which need login
#32
025nju
opened
7 months ago
5
How does the selector work?
#31
objectiveSee
closed
7 months ago
0
Adds for autoScroll for crawling the multi pages?
#30
SOSONAGI
opened
7 months ago
3
Scrap website
#29
manashan
opened
7 months ago
0
Multiple Selectors
#28
gummipunkt
closed
7 months ago
7
Added Directory Exclusion
#27
bleachedsleet
opened
7 months ago
1
Refactor getPageHtml function to handle selector not found case, using body as fallback. Add support for downloading URLs from sitemap.xml. Update comments to let know that sitemap is supported
#26
guillermoscript
closed
7 months ago
7
Add gpt-crawler subproject
#25
Neihouse
closed
7 months ago
2
added something
#24
belal87
closed
7 months ago
3
Possiblity to crawl a generic and entire xml without selector (for instance a schema)
#23
Tudor44
opened
7 months ago
0
Crawling duplicated url
#22
hbakhtiyor
opened
7 months ago
1
Turning a website into json data doesn't make the GPT more useful.
#21
sudo888samewick
closed
7 months ago
4
add cookie reference
#20
ashudevcodes
closed
7 months ago
2
Selector help
#19
nexuslux
opened
7 months ago
1
Previous
Next