Smart/Agentic Crawler (Invite Collaboration)

[unclecode]

I'm planning to add a smart crawler that takes a set of user-defined objectives and continues crawling to satisfy them. Objectives can be a query requiring a sufficient amount of information to answer, a set of actions with specific goals, or other custom tasks. I also plan to incorporate reinforcement learning into this process later. I've created an issue for discussion and invite interested individuals to share ideas and comments. If you're enthusiastic about contributing, please share your email address, and I'll invite you to our Discord channels to collaborate.

Happy Crawling

[zhixiongtan]

zhixiongtan94@gmail.com THANKS FOR YOUR WORK

[unclecode]

@zhixiongtan sent :)