feat: add exclude pattern for links in config

BuilderIO / gpt-crawler

Crawl a site to generate knowledge files to create your own custom GPT from a URL

https://www.builder.io/blog/custom-gpt

ISC License

18.14k stars 1.88k forks source link

feat: add exclude pattern for links in config #122

Closed cipherCOM closed 5 months ago

cipherCOM commented 6 months ago

See #9 for context. Simple addition to exclude with globs and regex.

cipherCOM commented 6 months ago

@marcelovicentegc my prettier run has many more changes (only trailing commas) in the whole core.ts. So can't help with the prettier_check fail. Or I make the change bigger than it is, by reformatting the core.ts and config.ts, if this is okay with you.

marcelovicentegc commented 5 months ago

Hey @cipherCOM! Happy new years! Would you mind just rebasing this PR 🤗? This will make sure that all tests pass 😉

cipherCOM commented 5 months ago

@marcelovicentegc rebased and pushed, everything okay now? :)

Happy new years to you too and have a great start into the year!

github-actions[bot] commented 5 months ago

:tada: This PR is included in version 1.3.0 :tada:

The release is available on:

Your semantic-release bot :package::rocket: