kubeagi / arcadia

A diverse, simple, and secure one-stop LLMOps platform
http://www.kubeagi.com/
Apache License 2.0
63 stars 20 forks source link

Couple fixes for RAG evaluation #979

Closed bjwswang closed 3 months ago

bjwswang commented 3 months ago

What type of PR is this?

/kind fix

What this PR does / why we need it

Which issue(s) this PR fixes

Fixes #978

Special notes for your reviewer

The reason to make sure all typed reference have apigroup,namespace set due to below error:

Error: node llm-node init failed: can't convert to langchain llm: an empty namespace may not be set when a resource name is provided
Usage:
  arctl eval gen_test_dataset [flags]

Flags:
      --ground-truths-column string   The column name which provides the answers (default "a")
  -h, --help                          help for gen_test_dataset
      --input-dir string              The input directory where to load original dataset files
      --max-context-length int        The maximum length of the context (default 512)
      --merge                         Whether to merge all generated test data into a single file
      --merge-file string             name of the merged document (default "ragas.csv")
      --output string                 The way to output the generated dataset rows.We support two ways:
                                       - stdout: print row
                                       - csv: save row to csv file
      --question-column string        The column name which provides questions (default "q")

Global Flags:
      --application string   The application to be evaluated
      --home string          home directory to use (default "/root/.arcadia")
  -n, --namespace string     namespace to use (default "default")

panic: node llm-node init failed: can't convert to langchain llm: an empty namespace may not be set when a resource name is provided

goroutine 1 [running]:
main.main()
    /go/src/cmd/arctl/main.go:60 +0x34