As someone building the chat database, I'd like to be able to run a single command and download all source material to a local cache directory.
Approach
[x] Create a command, chathelper that can be used for all phases of this work.
[x] Add a group of cache commands
[x] Define a configuration file yaml that can store the paper information.
[x] Implement langchain downloaders to fill the cache.
[x] Update the readme with instructions on how to use the download cache command
Eventually, we will probably want to do things like indico as well. And also have a backup cache to speed things up and so we don't have to constantly re-hit the archive.
As someone building the chat database, I'd like to be able to run a single command and download all source material to a local
cachedirectory.Approach
chathelperthat can be used for all phases of this work.cachecommandsyamlthat can store the paper information.langchaindownloaders to fill the cache.download cachecommandEventually, we will probably want to do things like
indicoas well. And also have a backup cache to speed things up and so we don't have to constantly re-hit the archive.