[Question]:Could't use Trulens on FuzzyCitationEnginePack to RAG evaluation

[Sudar278]

Question Validation

• [x] I have searched both the documentation and discord for an answer.

Question

`# Method_5

Fuzzy Query Engine

app_id='Method_5'

from llama_index.llama_pack import download_llama_pack FuzzyCitationEnginePack = download_llama_pack( "FuzzyCitationEnginePack", "./fuzzy_citation_pack" ) nest_asyncio.apply() os.environ['OPENAI_API_KEY'] = "" llm = OpenAI(temperature=0.1, model="gpt-3.5-turbo") pinecone.init(api_key='', environment='gcp-starter') pinecone_index=pinecone.Index('') vector_store = PineconeVectorStore(pinecone_index=pinecone_index) service_context = ServiceContext.from_defaults() index = VectorStoreIndex.from_vector_store(vector_store=vector_store) query_engine = index.as_query_engine() fuzzy_engine = FuzzyCitationEnginePack(query_engine, threshold=75)`

the above is my code for fuzzy query engine from llamahub.

I can generate answer from this query engine ,please refer to below image

while running a evaluation on this query engine using Trulens getting issue

Evaluation

tru_recorder = TruLlama( fuzzy_engine, app_id=app_id, feedbacks=[ f_qa_relevance, f_qs_relevance, f_groundedness, f_groundtruth ] )

Facing the below error

---

ModuleNotFoundError Traceback (most recent call last) Cell In[18], line 3 1 # Evaluation ----> 3 tru_recorder = TruLlama( 4 fuzzy_engine, 5 app_id=app_id, 6 feedbacks=[ 7 f_qa_relevance, 8 f_qs_relevance, 9 f_groundedness, 10 f_groundtruth 11 ] 12 )

File c:\Users\ADMIN\AppData\Local\Programs\Python\Python311\Lib\site-packages\trulens_eval\tru_llama.py:244, in TruLlama.init(self, app, kwargs) 241 def init(self, app: Union[BaseQueryEngine, BaseChatEngine], kwargs): 242 # TruLlama specific: 243 kwargs['app'] = app --> 244 kwargs['root_class'] = Class.of_object(app) # TODO: make class property 245 kwargs['instrument'] = LlamaInstrument(app=self) 247 super().init(**kwargs)

File c:\Users\ADMIN\AppData\Local\Programs\Python\Python311\Lib\site-packages\trulens_eval\utils\pyschema.py:247, in Class.of_object(obj, with_bases, loadable) 243 @staticmethod 244 def of_object( 245 obj: object, with_bases: bool = False, loadable: bool = False 246 ): --> 247 return Class.of_class( 248 cls=obj.class, with_bases=with_bases, loadable=loadable 249 )

File c:\Users\ADMIN\AppData\Local\Programs\Python\Python311\Lib\site-packages\trulens_eval\utils\pyschema.py:229, in Class.of_class(cls, with_bases, loadable) 223 @staticmethod 224 def of_class( 225 cls: type, with_bases: bool = False, loadable: bool = False 226 ) -> 'Class': 227 ret = Class( 228 name=cls.name, --> 229 module=Module.of_module_name(object_module(cls), loadable=loadable), 230 bases=list(map(lambda base: Class.of_class(cls=base), cls.mro)) 231 if with_bases else None 232 ) 234 if loadable: 235 if "" in repr( 236 cls): # TODO: figure out a better way to check this

File c:\Users\ADMIN\AppData\Local\Programs\Python\Python311\Lib\site-packages\trulens_eval\utils\pyschema.py:168, in Module.of_module_name(module_name, loadable) 164 if loadable and module_name == "main": 165 # running in notebook 166 raise ImportError(f"Module {module_name} is not importable.") --> 168 mod = importlib.import_module(module_name) 169 package_name = mod.package 170 return Module(package_name=package_name, module_name=module_name)

File c:\Users\ADMIN\AppData\Local\Programs\Python\Python311\Lib\importlib__init__.py:126, in import_module(name, package) 124 break 125 level += 1 --> 126 return _bootstrap._gcd_import(name[level:], package, level)

File :1204, in _gcd_import(name, package, level)

File :1176, in _find_andload(name, import)

File :1140, in _find_and_loadunlocked(name, import)