an error occurred in Retriever inference

[juyoung228]

I finished Retriever training and Retriever inference phases and tried to do Retriever validation, but the error was occurred. Can anyone figure out this problem..?

The command was

python dense_retriever.py model_file='/home/work/DPR/outputs/2022-03-16/08-12-08/checkpoints/dpr_biencoder.9' qa_dataset=nq_test ctx_datatsets=[dpr_wiki] encoded_ctx_files='/home/work/DPR/outputs/2022-03-17/07-19-03/out_0' out_file='/home/work/DPR/outputs/results.json'

log was like this:

[2022-03-18 01:12:18,585][root][INFO] - Loading saved model state ... [2022-03-18 01:12:18,786][root][INFO] - Selecting standard question encoder "main1" 00:54 18-Mar-22 [2022-03-18 01:12:21,019][root][INFO] - Encoder vector_size=768 [2022-03-18 01:12:21,019][root][INFO] - qa_dataset: nq_test [2022-03-18 01:12:21,022][dpr.data.download_data][INFO] - Requested resource from https://dl.fbaipublicfiles.com/dpr/data/retriever/nq-test.qa.csv [2022-03-18 01:12:21,022][dpr.data.download_data][INFO] - Download root_dir /home/work/DPR [2022-03-18 01:12:21,023][dpr.data.download_data][INFO] - File to be downloaded as /home/work/DPR/downloads/data/retriever/qas/nq-test.csv [2022-03-18 01:12:21,023][dpr.data.download_data][INFO] - File already exist /home/work/DPR/downloads/data/retriever/qas/nq-test.csv [2022-03-18 01:12:21,061][root][INFO] - questions len 3610 [2022-03-18 01:12:21,061][root][INFO] - questions_text len 0 [2022-03-18 01:12:21,061][root][INFO] - Local Index class <class 'dpr.indexer.faiss_indexers.DenseFlatIndexer'> [2022-03-18 01:12:21,062][root][INFO] - Using special token None [2022-03-18 01:12:26,879][root][INFO] - Encoded queries 3200 [2022-03-18 01:12:27,574][root][INFO] - Total encoded queries tensor torch.Size([3610, 768]) [2022-03-18 01:12:27,576][root][INFO] - ctx_sources: <class 'dpr.data.retriever_data.CsvCtxSrc'> [2022-03-18 01:12:27,576][root][INFO] - id_prefixes per dataset: ['dpr_wiki:'] [2022-03-18 01:12:27,576][root][INFO] - ctx_files_patterns: /home/work/DPR/outputs/2022-03-17/07-19-03/out_0 Error executing job with overrides: ['model_file=/home/work/DPR/outputs/2022-03-16/08-12-08/checkpoints/dpr_biencoder.9', 'qa_dataset=nq_test', 'ctx_datatsets=[dpr_wiki]', 'encoded_ctx_files=/home/work/DPR/outputs/2022-03-17/07-19-03/out_0', 'out_file=/home/work/DPR/outputs/results.json'] Traceback (most recent call last): File "dense_retriever.py", line 578, in main assert len(ctx_files_patterns) == len(id_prefixes), "ctx len={} pref leb={}".format( AssertionError: ctx len=48 pref leb=1

[ZiluLii]

Hi, did you figure out how to solve this? Got the same error :(

[eazaa]

aware :encoded_ctx_files='/home/work/DPR/outputs/2022-03-17/07-19-03/out_0' is should be a list,so try encoded_ctx_files=['/home/work/DPR/outputs/2022-03-17/07-19-03/out_0'] instead i had trouble with this same problem for week,and finally figure out by add "["