BuilderIO gpt-crawler issues

BuilderIO / gpt-crawler

Crawl a site to generate knowledge files to create your own custom GPT from a URL

https://www.builder.io/blog/custom-gpt

ISC License

18.13k stars 1.88k forks source link

issues

Newest

Newest Most commented Recently updated Oldest Least commented Least recently updated

GPT Crawler cli Drop-in config

#168 zerofill opened 2 days ago
1
Remove references to CLI

#167 bovlb closed 3 weeks ago
1
Script Not Crawling Subdirectories During Website Scraping

#166 AhsanAk opened 3 weeks ago
0
Type Error

#165 JoelWekesa opened 1 month ago
0
Doesnt Extract Texts present in dropboxes

#164 KumarSampurn opened 2 months ago
0
memory usage need to be optimized

#163 banditsmile opened 2 months ago
0
PlaywrightCrawler memory problem and errors

#162 lipstk opened 2 months ago
0
sh: cross-env: command not found

#161 Vickie-Liu closed 3 months ago
0
add a username and password？

#160 GentleLemon closed 3 months ago
0
how to crawler this site?match not work

#159 tom6q6 opened 3 months ago
1
cookie example

#158 cmbcbe opened 3 months ago
0
extracting text in hidden div blocks

#157 udgithub opened 3 months ago
0
How do I keep links to pictures and videos in my web pages?

#156 he1771129657 opened 3 months ago
0
How to supply read-able code to the GPT?

#155 lucastobrazil closed 3 months ago
0
feat: Customizing selector can sometimes cause web crawlers to fail

#154 kukuze opened 3 months ago
0
feat: Customizing CSS selectors can sometimes cause web crawlers to fail

#153 kukuze closed 3 months ago
1
Output.json file not created

#152 bourgeda opened 3 months ago
1
error TS2322

#151 sc0h0 opened 4 months ago
6
Can i use Gemini model by google?

#150 amrpyt opened 4 months ago
0
How to crawl Single Page Application(SPA)

#149 ouyh1111 opened 4 months ago
0
fix: #147 Set `purgeOnStart: true` to process multiple sites as a server

#148 kaibadash closed 4 months ago
0
Does gpt-crawler server always return same site?

#147 kaibadash closed 4 months ago
0
Multiple Selectors not Reflected in Output

#146 mahdii0908 opened 4 months ago
0
Cookies not accepted

#145 stevenbaert opened 4 months ago
1
chore: update README.md

#144 eltociear closed 4 months ago
1
add a method to evaluate the quality of the retrieved context

#143 shrjain1312 opened 4 months ago
0
feat: git clone depth limit in docker

#142 fer-ri opened 4 months ago
0
how to add userName & passwd to gpt-crawler

#141 DorakuCN opened 5 months ago
0
ERROR PlaywrightCrawler: Request failed and reached maximum retries. Navigation timed out after 60 seconds

#140 Mytraas opened 5 months ago
0
Trying to Crawl site nothing working

#139 upup666 opened 5 months ago
1
feat: add server api readme docs

#138 adityak74 closed 5 months ago
3
WARN PlaywrightCrawler: Reclaiming failed request back to the list or queue. Request blocked - received 429 status code.

#137 Voyager3D opened 5 months ago
2
Crawl websites protected by username and password?

#136 alzh666 opened 5 months ago
1
Multiple websites at once?

#135 heyfletch opened 5 months ago
11
Add for a selector to exclude elements in the site

#134 razaanstha opened 5 months ago
1
feat: account for other sitemap names

#133 Skeyelab closed 5 months ago
1
chore: add release notes generator

#132 marcelovicentegc closed 5 months ago
1
Extracting Articles from Sequential Pages of a Website

#131 nicofierrov opened 5 months ago
0
Disallowed Special Token

#130 MrAshRhodes opened 5 months ago
2
Temp

#129 zshnb closed 5 months ago
0
Only one tag html for all the page

#128 Th3Heavy opened 5 months ago
2
npm start issue

#127 diandian0420 opened 5 months ago
3
fix: add run swagger js when start server

#126 zshnb closed 5 months ago
6
aisa gpt

#125 AZURALIF06 opened 5 months ago
0
How to crawl https://zod.dev/ ?

#124 MontiL opened 5 months ago
2
'Zod' package not found |

#123 Daethyra opened 5 months ago
1
feat: add exclude pattern for links in config

#122 cipherCOM closed 5 months ago
4
Custom JSON name and latest dependencies

#121 miguelgargallo opened 6 months ago
0
Add support for concurrent invocations to crawl

#120 adityak74 opened 6 months ago
3
How to paginate for large JSON files?

#119 chnsh opened 6 months ago
0