QuivrHQ / quivr

Opiniated RAG for integrating GenAI in your apps 🧠 Focus on your product rather than the RAG. Easy integration in existing products with customisation! Any LLM: GPT4, Groq, Llama. Any Vectorstore: PGVector, Faiss. Any Files. Anyway you want.
https://core.quivr.com
Other
36.66k stars 3.58k forks source link

[Bug]: pnly one file show up as loaded for ingestion even when multiple files are dragged for addition #3139

Open atulgrover opened 2 months ago

atulgrover commented 2 months ago

What happened?

A bug happened! image

Relevant log output

No response

Twitter / LinkedIn details

No response

linear[bot] commented 2 months ago

CORE-189 [Bug]: pnly one file show up as loaded for ingestion even when multiple files are dragged for addition

dosubot[bot] commented 2 months ago

The issue of only one file showing up as loaded for ingestion even when multiple files are dragged for addition in the "Add Knowledge" UI modal could be due to the following code logic in the onDrop function:

const files: File[] = (
  knowledgeToFeed.filter((c) => c.source === "upload") as FeedItemUploadType[]
).map((c) => c.file);

const onDrop = (acceptedFiles: File[], fileRejections: FileRejection[]) => {
  if (!isBrainCreationModalOpened) {
    setShouldDisplayFeedCard(true);
  }
  if (fileRejections.length > 0) {
    const firstRejection = fileRejections[0];

    if (firstRejection.errors[0].code === "file-invalid-type") {
      publish({ variant: "danger", text: t("invalidFileType") });
    } else {
      publish({
        variant: "danger",
        text: t("maxSizeError", { ns: "upload" }),
      });
    }

    return;
  }
  for (const file of acceptedFiles) {
    const isAlreadyInFiles =
      files.filter((f) => f.name === file.name && f.size === file.size).length > 0;
    if (isAlreadyInFiles) {
      publish({
        variant: "warning",
        text: t("alreadyAdded", { fileName: file.name, ns: "upload" }),
      });
    } else {
      if (isOnboarding) {
        void trackOnboardingEvent("FILE_UPLOADED");
      } else {
        void track("FILE_UPLOADED");
      }

      addKnowledgeToFeed({
        source: "upload",
        file: cloneFileWithSanitizedName(file),
      });
    }
  }
};

The onDrop function processes each file in the acceptedFiles array. However, if a file is already present in the files array (checked by name and size), it will not be added again, and a warning will be published instead. This could be why only one file appears to be loaded if the other files are considered duplicates based on their name and size [1].

To continue talking to Dosu, mention @dosu.