Answer quality

[jarulraj]

Hey @Dundric. This would be very useful in schools. But, are you able to get good answers on the HTTR PDF document?

[Dundric]

Thankyou for your interest! Yes, I am able to get good answers to questions on the HTTR documents. This is due to Instructor embeddings providing functions that are capable of specifying a topic for those embedded documents, which I have set up to allow the user to type in a topic during the embedding proces. Here are some examples of questions I asked and answers that I got from just using the provided pdfs about nuclear material:

Question: What are some of the parts of a HTTR?

Answer: The major design features of the HTTR include the reactor core, a reactor cooling system, engineered safety systems, instrumentation and control system, and a containment vessel. The reactor core consists of low-enriched uranium fuel elements and moderator rods, while the reactor cooling system includes pumps, heat exchangers, and other equipment to circulate water through the core. The engineered safety systems include measures such as passive decay heat removal and emergency core cooling, while the instrumentation and control system monitors and controls the reactor's operation. Finally, the containment vessel provides a barrier between the environment and the radioactive material in case of an accident or release of fission products.

Question: What is passive decay heat removal?

Answer: Passive decay heat removal refers to a method of removing decay heat from a nuclear reactor without using any active systems or electrical power. It involves natural convection, conduction and radiation to transfer heat away from the core to the atmosphere or to other parts of the plant for cooling. This system is designed to work in case of a loss of primary coolant circulation system. Passive decay heat removal systems are widely used in advanced nuclear power plants to provide safety and reliability features.

I'm working on optimizing the project right now. Seeing what models work best and making the thing run faster. Additonally, right now every time you ingest documents it has to create a whole new vector database which I will have fixed by tomorrow so it only adds new documents to the already existing vector database. This will likely allow even better embedding as every time you ingest new documents you can input a really specific topic for those embeddings. Any feedback is appreciated and I am open to having discussion about any part of this project.

[jarulraj]

Thanks for the clarifications, @Dundric. That's neat! Does it capture global document-level details as we break the document into small chunks and run the feature extractor on each chunk? How long does it take for data ingestion?

We are building the EvaDB database system for further simplifying the data loading and reducing overall query time -- https://github.com/georgia-tech-db/eva/tree/master/apps/privategpt. That's why we are interested in real-world applications like this one.

[Dundric]

Your research is very interesting Professor @jarulraj. I'm creating this project as an undergraduate research assistant at the University of Illinois. We are trying to create a portable gpt capable of giving responses on hyper-specific topics. It also has to be completly local as we inted to use it to handle export controlled documents.

In regards to your questions:

1. I don't believe it is capturing document level details as it breaks it down into chunks (even though that would be nice and theoretically possible if you create an embedding of the whole document with a topic, and then create embeddings out of chunks of that document, but I don't know if that would work well with a traditional vector database). Instead what it is doing right now is using a user defined task( in the form of a string) to apply modifications to the vector generated by the embedding of each text chunk to make it seem more similar to text chunks with a similar task. The effectiveness of this method is clearer when applying more specific tasks to embedding of different documents, as it can better differentiate between two different but very similar ideas (I.E HTGR reactor versus HTTR reactor).

2. To ingest 3 documents of 214 pages total in length running on my M1 macbook 16 gigabytes of ram: a. To split into 1614 text chunks (using pool parallel processing) took 5.38 seconds. b. To generate embeddings using Base Instruct Embedding model took 52.08 seconds. c. To generate embeddings using Large Instruct Embedding model took 3 minutes 10 seconds. d. To generate embedding using XL Instruct Embedding model took 11 minutes 13 seconds.