search

facebookresearch / DrQA

Reading Wikipedia to Answer Open-Domain Questions
Other
4.48k stars 898 forks source link

Memory Error #190

Open j07nikita opened 5 years ago

j07nikita commented 5 years ago

$ python3 scripts/pipeline/interactive.py 12/08/2018 06:03:16 AM: [ Running on CPU only. ] 12/08/2018 06:03:16 AM: [ Initializing pipeline... ] 12/08/2018 06:03:16 AM: [ Initializing document ranker... ] 12/08/2018 06:03:16 AM: [ Loading /scratch/nikita/DrQA/data/wikipedia/docs-tfidf-ngram=2-hash=16777216-tokenizer=simple.npz ] 12/08/2018 06:03:48 AM: [ Initializing document reader... ] 12/08/2018 06:03:48 AM: [ Loading model /scratch/nikita/DrQA/data/reader/multitask.mdl ] 12/08/2018 06:03:52 AM: [ Initializing tokenizers and document retrievers... ]

Interactive DrQA

process(question, candidates=None, top_n=1, n_docs=5) usage()

salloc: Exceeded job memory limit

getting memory error using 20 cpu (20gb+)

ajfisch commented 5 years ago

Are you running on SLURM? I'm not to sure how the resource manager works there. Is overcommit enabled?

j07nikita commented 5 years ago

Yeah i am running on SLURM. How can i reduce memory usage (because using more threads can cause this issue) ?