Closed skks1212 closed 1 year ago
@skks1212 What is meant by web links? Are we going change the upsert function to process direct text and scrape website from the given link? Can you explain user the flow for this issue
@ishanExtreme Yes, you will have to scrape from the URL.
For ex. If we want to scrape data from React docs, we can do that by entering the URL of the page, one page at a time (maybe we can automate that at a later stage)
Direct text is a different thing, where we can just enter text directly to a textarea and upsert
Currently we are only processing pdf and txt files. Diversify this to take up web links and direct text. Alter the models to store this data as well.