WASP-AI / AI-Powered-Data-Catalog

Bachelor Thesis with Equinor
3 stars 0 forks source link

Create hamse, data description quality checker #3

Open Hamsehs opened 1 year ago

Hamsehs commented 1 year ago

The code imports the required libraries, such as Transformers and Streamlit. It pulls a summarization model that has already been trained from the Hugging Face Transformers library. It specifically loads the tokenizer and the facebook/bart-large-cnn model. The user interface is configured by the code using Streamlit. Together with a file uploader and a text input box for the user to provide a description, it creates a title and a description. When a text file is uploaded by the user, the program scans its contents and then uses the pre-trained model to produce a summary of the file. It uses the tokenizer to encrypt the contents of the file, the model's generate() method to provide a summary, and the tokenizer to decrypt the summary. The Streamlit app shows the resulting summary. The function checks to see if the description matches the generated summary when a user uploads a file and adds a description. If so, a message stating that the description matches the file is displayed; otherwise, a message stating that the description does not match the file is displayed.

The code shows the contents of a file when it is uploaded in a code block. The code shows a description in a text box if one is provided. Overall, the code offers a straightforward interface for determining whether the contents of a text file fit a given description.