search

NCIOCPL / bestbets-api

0 stars 4 forks source link

indexer error for GetTokenCount #9

Closed zhuomingao closed 5 years ago

zhuomingao commented 7 years ago

Unhandled Exception: System.Exception: We should really have a logger for this error. ---> NCI.OCPL.Api.BestBets.APIErrorException: Errors Occurred. at NCI.OCPL.Api.BestBets.Services.ESTokenAnalyzerService.GetTokenCount(String term) at NCI.OCPL.Api.BestBets.Indexer.BestBetSynonymMapper.d3.MoveNext() at NCI.OCPL.Api.BestBets.Indexer.Program.Worker.d10.MoveNext() at MoreLinq.MoreEnumerable.d__132.MoveNext() at NCI.OCPL.Api.BestBets.Indexer.Services.ESBBIndexerService.IndexBestBetsMatches(String indexName, IEnumerable1 matches, Int32 batchSize) at NCI.OCPL.Api.BestBets.Indexer.Program.Worker.Run() --- End of inner exception stack trace --- at NCI.OCPL.Api.BestBets.Indexer.Program.Worker.Run() at NCI.OCPL.Api.BestBets.Indexer.Program.Main(String[] args)

We need to add logging and include the actual synonym that caused the error (ES side slow log and error log does not cover _analyze endpoint)

zhuomingao commented 7 years ago

Added debug information shows the indexer is processing xml files in order of the filename. I ran 3 times in Linux and it is the same XML file causing error 36687.xml, I deleted 36687.xml from Blue publishecontent folder. And now the error happens for 36688.xml

I manually examined the xml files 36687, 36688 ( a Spanish one) can't find anything wrong.

The fact that the processing always failed at 460th XML file two times in a row with different XML file puzzles me, I deleted 36687, 36688 along with 4 of 35xxx XML, and now it failed at 459th XML (36694.XML BRCA2). It seems it is more related to the number of xml processed or synonyms processed than the actual content of xml files

It seems more related to the actual Linux box the indexer runs on. I just did a release and ran the indexer through a docker container, and it finished successfully.

bryanpizzillo commented 7 years ago

@zhuomingao did the error specify the synonym text that was having issues analyzing?

zhuomingao commented 7 years ago

Yes, the error specify the synonym text that was having error.

bryanpizzillo commented 7 years ago

@zhuomingao, sorry for not being clearer in my question. Can you please supply the exact text that was having issues?

zhuomingao commented 7 years ago

Error Is like below: Fetching Category: 36687.xml dbug: NCI.OCPL.Api.BestBets.Indexer.BestBetsIndexer[0] Mapping Category: CT Scan fail: NCI.OCPL.Api.BestBets.Services.ESTokenAnalyzerService[0] Elasticsearch Response for GetTokenCount is Not Valid. Term 'CT'

Synonym in this case is CT

zhuomingao commented 7 years ago

more examples of this error inside container, caused by different term every time:

: System.Exception: We should really have a logger for this error. ---> System.Exception: Error in getting token count for: Cáncer de intestino delgado, Synonym: ,Cáncer de intestino delgado ---> Elasticsearch.Net.UnexpectedElasticsearchClientException: A task was canceled. ---> System.Threading.Tasks.TaskCanceledException: A task was canceled. : System.Exception: We should really have a logger for this error. ---> System.Exception: Error in getting token count for: Pemetrexed Disodium (Alimta®), Synonym: ,pemetrexed ---> Elasticsearch.Net.UnexpectedElasticsearchClientException: A task was canceled. ---> System.Threading.Tasks.TaskCanceledException: A task was canceled. : System.Exception: We should really have a logger for this error. ---> System.Exception: Error in getting token count for: Fellowships, Synonym: ,FDA-NCI ---> Elasticsearch.Net.UnexpectedElasticsearchClientException: A task was canceled. ---> System.Threading.Tasks.TaskCanceledException: A task was canceled. : System.Exception: We should really have a logger for this error. ---> System.Exception: Error in getting token count for: Tumor Markers, Synonym: ,breast ---> Elasticsearch.Net.UnexpectedElasticsearchClientException: A task was canceled. ---> System.Threading.Tasks.TaskCanceledException: A task was canceled. : System.Exception: We should really have a logger for this error. ---> System.Exception: Error in getting token count for: Tratamiento, Synonym: ,Tratamiento ---> Elasticsearch.Net.UnexpectedElasticsearchClientException: A task was canceled. ---> System.Threading.Tasks.TaskCanceledException: A task was canceled. : System.Exception: We should really have a logger for this error. ---> System.Exception: Error in getting token count for: Cancer Research Ideas, Synonym: ,Clinical Trial Ideas ---> Elasticsearch.Net.UnexpectedElasticsearchClientException: A task was canceled. ---> System.Threading.Tasks.TaskCanceledException: A task was canceled. : System.Exception: We should really have a logger for this error. ---> System.Exception: Error in getting token count for: Colorectal Cancer Screening, Synonym: ,colon screening ---> Elasticsearch.Net.UnexpectedElasticsearchClientException: A task was canceled. ---> System.Threading.Tasks.TaskCanceledException: A task was canceled. : System.Exception: We should really have a logger for this error. ---> System.Exception: Error in getting token count for: Cancer Research Ideas, Synonym: ,Cancer Research Ideas ---> Elasticsearch.Net.PipelineException: Failed sniffing cluster state. ---> System.AggregateException: One or more errors occurred. (A task was canceled.) ---> System.Threading.Tasks.TaskCanceledException: A task was canceled. : System.Exception: We should really have a logger for this error. ---> System.Exception: Error in getting token count for: Health Disparities, Synonym: ,ethnic ---> Elasticsearch.Net.UnexpectedElasticsearchClientException: A task was canceled. ---> System.Threading.Tasks.TaskCanceledException: A task was canceled. : System.Exception: We should really have a logger for this error. ---> System.Exception: Error in getting token count for: Clinical Trials: Patient Care Costs, Synonym: ,cost ---> Elasticsearch.Net.UnexpectedElasticsearchClientException: A task was canceled. ---> System.Threading.Tasks.TaskCanceledException: A task was canceled. : System.Exception: We should really have a logger for this error. ---> System.Exception: Error in getting token count for: Epidemiology, Synonym: ,Branch ---> Elasticsearch.Net.UnexpectedElasticsearchClientException: A task was canceled. ---> System.Threading.Tasks.TaskCanceledException: A task was canceled. : System.Exception: We should really have a logger for this error. ---> System.Exception: Error in getting token count for: Childhood Cancers, Synonym: ,aml childhood ---> Elasticsearch.Net.UnexpectedElasticsearchClientException: A task was canceled. ---> System.Threading.Tasks.TaskCanceledException: A task was canceled. : System.Exception: We should really have a logger for this error. ---> System.Exception: Error in getting token count for: Clinical Trials, Synonym: ,APC ---> Elasticsearch.Net.UnexpectedElasticsearchClientException: A task was canceled. ---> System.Threading.Tasks.TaskCanceledException: A task was canceled. : System.Exception: We should really have a logger for this error. ---> System.Exception: Error in getting token count for: Linfoma, Synonym: ,cancer linfatico ---> Elasticsearch.Net.UnexpectedElasticsearchClientException: A task was canceled. ---> System.Threading.Tasks.TaskCanceledException: A task was canceled. : System.Exception: We should really have a logger for this error. ---> System.Exception: Error in getting token count for: Cáncer de colon y recto, Synonym: ,population ---> Elasticsearch.Net.UnexpectedElasticsearchClientException: A task was canceled. ---> System.Threading.Tasks.TaskCanceledException: A task was canceled. : System.Exception: We should really have a logger for this error. ---> System.Exception: Error in getting token count for: Estudios clínicos, Synonym: ,Red Nacional de Estudios Clínicos ---> Elasticsearch.Net.UnexpectedElasticsearchClientException: A task was canceled. ---> System.Threading.Tasks.TaskCanceledException: A task was canceled. : System.Exception: We should really have a logger for this error. ---> System.Exception: Error in getting token count for: LiveHelp, Synonym: ,Live Help ---> Elasticsearch.Net.UnexpectedElasticsearchClientException: A task was canceled. ---> System.Threading.Tasks.TaskCanceledException: A task was canceled. : System.Exception: We should really have a logger for this error. ---> System.Exception: Error in getting token count for: Dietary Fat, Synonym: ,breast cancer ---> Elasticsearch.Net.UnexpectedElasticsearchClientException: A task was canceled. ---> System.Threading.Tasks.TaskCanceledException: A task was canceled. : System.Exception: We should really have a logger for this error. ---> System.Exception: Error in getting token count for: Statistics, Synonym: ,bone ---> Elasticsearch.Net.UnexpectedElasticsearchClientException: A task was canceled. ---> System.Threading.Tasks.TaskCanceledException: A task was canceled. : System.Exception: We should really have a logger for this error. ---> System.Exception: Error in getting token count for: Cáncer de mama, Synonym: ,mamario ---> Elasticsearch.Net.UnexpectedElasticsearchClientException: A task was canceled. ---> System.Threading.Tasks.TaskCanceledException: A task was canceled. : System.Exception: We should really have a logger for this error. ---> System.Exception: Error in getting token count for: Ductal Carcinoma In Situ (DCIS), Synonym: ,DCSI ---> Elasticsearch.Net.UnexpectedElasticsearchClientException: A task was canceled. ---> System.Threading.Tasks.TaskCanceledException: A task was canceled. : System.Exception: We should really have a logger for this error. ---> System.Exception: Error in getting token count for: Grants, Synonym: ,Grants ---> Elasticsearch.Net.UnexpectedElasticsearchClientException: A task was canceled. ---> System.Threading.Tasks.TaskCanceledException: A task was canceled. : System.Exception: We should really have a logger for this error. ---> System.Exception: Error in getting token count for: Cáncer de ovario, Synonym: ,ovario ---> Elasticsearch.Net.UnexpectedElasticsearchClientException: A task was canceled. ---> System.Threading.Tasks.TaskCanceledException: A task was canceled. : System.Exception: We should really have a logger for this error. ---> System.Exception: Error in getting token count for: PAs, Synonym: ,PAs ---> Elasticsearch.Net.UnexpectedElasticsearchClientException: A task was canceled. ---> System.Threading.Tasks.TaskCanceledException: A task was canceled. : System.Exception: We should really have a logger for this error. ---> System.Exception: Error in getting token count for: Primitive neuroectodermal tumors (PNET), Synonym: ,neuroectodermal ---> Elasticsearch.Net.UnexpectedElasticsearchClientException: A task was canceled. ---> System.Threading.Tasks.TaskCanceledException: A task was canceled. : System.Exception: We should really have a logger for this error. ---> System.Exception: Error in getting token count for: Bone Marrow Cancer, Synonym: ,biopsy ---> Elasticsearch.Net.UnexpectedElasticsearchClientException: A task was canceled. ---> System.Threading.Tasks.TaskCanceledException: A task was canceled. : System.Exception: We should really have a logger for this error. ---> System.Exception: Error in getting token count for: Adverse Events, Synonym: ,NCICTC ---> Elasticsearch.Net.UnexpectedElasticsearchClientException: A task was canceled. ---> System.Threading.Tasks.TaskCanceledException: A task was canceled. : System.Exception: We should really have a logger for this error. ---> System.Exception: Error in getting token count for: Chemotherapy Side Effects, Synonym: ,chemotherapy affects ---> Elasticsearch.Net.UnexpectedElasticsearchClientException: A task was canceled. ---> System.Threading.Tasks.TaskCanceledException: A task was canceled. : System.Exception: We should really have a logger for this error. ---> System.Exception: Error in getting token count for: Interferon, Synonym: ,inferon ---> Elasticsearch.Net.UnexpectedElasticsearchClientException: A task was canceled. ---> System.Threading.Tasks.TaskCanceledException: A task was canceled. : System.Exception: We should really have a logger for this error. ---> System.Exception: Error in getting token count for: Summer Students, Synonym: ,summer ---> Elasticsearch.Net.UnexpectedElasticsearchClientException: A task was canceled. ---> System.Threading.Tasks.TaskCanceledException: A task was canceled. : System.Exception: We should really have a logger for this error. ---> System.Exception: Error in getting token count for: Neuropathy, Synonym: ,treatment ---> Elasticsearch.Net.UnexpectedElasticsearchClientException: A task was canceled. ---> System.Threading.Tasks.TaskCanceledException: A task was canceled. : System.Exception: We should really have a logger for this error. ---> System.Exception: Error in getting token count for: Tamoxifen (Nolvadex®), Synonym: ,Nolvadex ---> Elasticsearch.Net.UnexpectedElasticsearchClientException: A task was canceled. ---> System.Threading.Tasks.TaskCanceledException: A task was canceled. : System.Exception: We should really have a logger for this error. ---> System.Exception: Error in getting token count for: Cáncer de pene, Synonym: ,Cáncer de pene ---> Elasticsearch.Net.UnexpectedElasticsearchClientException: A task was canceled. ---> System.Threading.Tasks.TaskCanceledException: A task was canceled. : System.Exception: We should really have a logger for this error. ---> System.Exception: Error in getting token count for: Statistics, Synonym: ,clinical trial ---> Elasticsearch.Net.UnexpectedElasticsearchClientException: A task was canceled. ---> System.Threading.Tasks.TaskCanceledException: A task was canceled. : System.Exception: We should really have a logger for this error. ---> System.Exception: Error in getting token count for: Health Disparities, Synonym: ,Network ---> Elasticsearch.Net.UnexpectedElasticsearchClientException: A task was canceled. ---> System.Threading.Tasks.TaskCanceledException: A task was canceled. : System.Exception: We should really have a logger for this error. ---> System.Exception: Error in getting token count for: Leukemia, Synonym: ,leukemia trials ---> Elasticsearch.Net.UnexpectedElasticsearchClientException: A task was canceled. ---> System.Threading.Tasks.TaskCanceledException: A task was canceled. : System.Exception: We should really have a logger for this error. ---> System.Exception: Error in getting token count for: Cryosurgery, Synonym: ,cryotherapy ---> Elasticsearch.Net.UnexpectedElasticsearchClientException: A task was canceled. ---> System.Threading.Tasks.TaskCanceledException: A task was canceled. : System.Exception: We should really have a logger for this error. ---> System.Exception: Error in getting token count for: Cancer Centers, Synonym: ,transplant ---> Elasticsearch.Net.UnexpectedElasticsearchClientException: A task was canceled. ---> System.Threading.Tasks.TaskCanceledException: A task was canceled. : System.Exception: We should really have a logger for this error. ---> System.Exception: Error in getting token count for: EPEC-O, Synonym: ,EPEC ---> Elasticsearch.Net.UnexpectedElasticsearchClientException: A task was canceled. ---> System.Threading.Tasks.TaskCanceledException: A task was canceled. : System.Exception: We should really have a logger for this error. ---> System.Exception: Error in getting token count for: Prevención, Synonym: ,prevencion ---> Elasticsearch.Net.UnexpectedElasticsearchClientException: A task was canceled. ---> System.Threading.Tasks.TaskCanceledException: A task was canceled. : System.Exception: We should really have a logger for this error. ---> System.Exception: Error in getting token count for: Ductal Carcinoma In Situ (DCIS), Synonym: ,infiltrante ---> Elasticsearch.Net.UnexpectedElasticsearchClientException: A task was canceled. ---> System.Threading.Tasks.TaskCanceledException: A task was canceled. : System.Exception: We should really have a logger for this error. ---> System.Exception: Error in getting token count for: Partnership and Collaboration, Synonym: ,partnering ---> Elasticsearch.Net.UnexpectedElasticsearchClientException: A task was canceled. ---> System.Threading.Tasks.TaskCanceledException: A task was canceled. : System.Exception: We should really have a logger for this error. ---> System.Exception: Error in getting token count for: Cancer Centers, Synonym: ,office ---> Elasticsearch.Net.UnexpectedElasticsearchClientException: A task was canceled. ---> System.Threading.Tasks.TaskCanceledException: A task was canceled. : System.Exception: We should really have a logger for this error. ---> System.Exception: Error in getting token count for: President's Cancer Panel, Synonym: ,Presidents Panel ---> Elasticsearch.Net.UnexpectedElasticsearchClientException: A task was canceled. ---> System.Threading.Tasks.TaskCanceledException: A task was canceled. : System.Exception: We should really have a logger for this error. ---> System.Exception: Error in getting token count for: Training and Employment, Synonym: ,investigator training ---> Elasticsearch.Net.UnexpectedElasticsearchClientException: A task was canceled. ---> System.Threading.Tasks.TaskCanceledException: A task was canceled. : System.Exception: We should really have a logger for this error. ---> System.Exception: Error in getting token count for: CT Scan, Synonym: ,positron ---> Elasticsearch.Net.UnexpectedElasticsearchClientException: A task was canceled. ---> System.Threading.Tasks.TaskCanceledException: A task was canceled. : System.Exception: We should really have a logger for this error. ---> System.Exception: Error in getting token count for: Cáncer de páncreas, Synonym: ,neuroendocrine ---> Elasticsearch.Net.UnexpectedElasticsearchClientException: A task was canceled. ---> System.Threading.Tasks.TaskCanceledException: A task was canceled. : System.Exception: We should really have a logger for this error. ---> System.Exception: Error in getting token count for: Finding a Doctor or Treatment Facility, Synonym: ,cancer clinic ---> Elasticsearch.Net.UnexpectedElasticsearchClientException: A task was canceled. ---> System.Threading.Tasks.TaskCanceledException: A task was canceled. : System.Exception: We should really have a logger for this error. ---> System.Exception: Error in getting token count for: Lung Cancer Symptoms, Synonym: ,lung cancer symtoms ---> Elasticsearch.Net.UnexpectedElasticsearchClientException: A task was canceled. ---> System.Threading.Tasks.TaskCanceledException: A task was canceled. : System.Exception: We should really have a logger for this error. ---> System.Exception: Error in getting token count for: LEEP, Synonym: ,loop ---> Elasticsearch.Net.UnexpectedElasticsearchClientException: A task was canceled. ---> System.Threading.Tasks.TaskCanceledException: A task was canceled. : System.Exception: We should really have a logger for this error. ---> System.Exception: Error in getting token count for: Health Promotion, Synonym: ,Health Promotion ---> Elasticsearch.Net.UnexpectedElasticsearchClientException: A task was canceled. ---> System.Threading.Tasks.TaskCanceledException: A task was canceled. : System.Exception: We should really have a logger for this error. ---> System.Exception: Error in getting token count for: Division of Cancer Treatment and Diagnosis (DCTD), Synonym: ,DCDT ---> Elasticsearch.Net.UnexpectedElasticsearchClientException: A task was canceled. ---> System.Threading.Tasks.TaskCanceledException: A task was canceled. : System.Exception: We should really have a logger for this error. ---> System.Exception: Error in getting token count for: Linfedema, Synonym: ,Linfedema ---> Elasticsearch.Net.UnexpectedElasticsearchClientException: A task was canceled. ---> System.Threading.Tasks.TaskCanceledException: A task was canceled. : System.Exception: We should really have a logger for this error. ---> System.Exception: Error in getting token count for: Letter of Intent for Investigational Drug Trial, Synonym: ,Letter of Intent for Investigational Drug Trial ---> Elasticsearch.Net.UnexpectedElasticsearchClientException: A task was canceled. ---> System.Threading.Tasks.TaskCanceledException: A task was canceled. : System.Exception: We should really have a logger for this error. ---> System.Exception: Error in getting token count for: Cáncer de esófago, Synonym: ,esófago ---> Elasticsearch.Net.UnexpectedElasticsearchClientException: A task was canceled. ---> System.Threading.Tasks.TaskCanceledException: A task was canceled. : System.Exception: We should really have a logger for this error. ---> System.Exception: Error in getting token count for: Requests for Applications, Synonym: ,Requests for Applications ---> Elasticsearch.Net.UnexpectedElasticsearchClientException: A task was canceled. ---> System.Threading.Tasks.TaskCanceledException: A task was canceled. : System.Exception: We should really have a logger for this error. ---> System.Exception: Error in getting token count for: Non-Hodgkin Lymphoma, Synonym: ,no ---> Elasticsearch.Net.UnexpectedElasticsearchClientException: A task was canceled. ---> System.Threading.Tasks.TaskCanceledException: A task was canceled. : System.Exception: We should really have a logger for this error. ---> System.Exception: Error in getting token count for: Estadísticas, Synonym: ,estadística ---> Elasticsearch.Net.UnexpectedElasticsearchClientException: A task was canceled. ---> System.Threading.Tasks.TaskCanceledException: A task was canceled. A task was canceled. : System.Exception: We should really have a logger for this error. ---> System.Exception: Error in getting token count for: Second Opinion, Synonym: ,getting a second opinion ---> Elasticsearch.Net.UnexpectedElasticsearchClientException: A task was canceled. ---> System.Threading.Tasks.TaskCanceledException: A task was canceled. : System.Exception: We should really have a logger for this error. ---> System.Exception: Error in getting token count for: Mammogram, Synonym: ,day ---> Elasticsearch.Net.UnexpectedElasticsearchClientException: A task was canceled. ---> System.Threading.Tasks.TaskCanceledException: A task was canceled. : System.Exception: We should really have a logger for this error. ---> System.Exception: Error in getting token count for: Frederick National Laboratory, Synonym: ,Fredrick ---> Elasticsearch.Net.UnexpectedElasticsearchClientException: A task was canceled. ---> System.Threading.Tasks.TaskCanceledException: A task was canceled. : System.Exception: We should really have a logger for this error. ---> System.Exception: Error in getting token count for: BRCA1 and BRCA2, Synonym: ,BRAC1 ---> Elasticsearch.Net.UnexpectedElasticsearchClientException: A task was canceled. ---> System.Threading.Tasks.TaskCanceledException: A task was canceled. : System.Exception: We should really have a logger for this error. ---> System.Exception: Error in getting token count for: Finding a Doctor or Treatment Facility, Synonym: ,treatment facilities ---> Elasticsearch.Net.UnexpectedElasticsearchClientException: A task was canceled. ---> System.Threading.Tasks.TaskCanceledException: A task was canceled. : System.Exception: We should really have a logger for this error. ---> System.Exception: Error in getting token count for: Women's Health, Synonym: ,Women's Health ---> Elasticsearch.Net.UnexpectedElasticsearchClientException: A task was canceled. ---> System.Threading.Tasks.TaskCanceledException: A task was canceled. : System.Exception: We should really have a logger for this error. ---> System.Exception: Error in getting token count for: Tumor Markers, Synonym: ,biomarkers ---> Elasticsearch.Net.UnexpectedElasticsearchClientException: A task was canceled. ---> System.Threading.Tasks.TaskCanceledException: A task was canceled. : System.Exception: We should really have a logger for this error. ---> System.Exception: Error in getting token count for: Training and Employment, Synonym: ,cancer training ---> Elasticsearch.Net.UnexpectedElasticsearchClientException: A task was canceled. ---> System.Threading.Tasks.TaskCanceledException: A task was canceled. : System.Exception: We should really have a logger for this error. ---> System.Exception: Error in getting token count for: Malignant Peripheral Nerve Sheath Tumor, Synonym: ,peripheral nerve ---> Elasticsearch.Net.UnexpectedElasticsearchClientException: A task was canceled. ---> System.Threading.Tasks.TaskCanceledException: A task was canceled. : System.Exception: We should really have a logger for this error. ---> System.Exception: Error in getting token count for: Human Participant Protections Course, Synonym: ,ethics training ---> Elasticsearch.Net.UnexpectedElasticsearchClientException: A task was canceled. ---> System.Threading.Tasks.TaskCanceledException: A task was canceled. : System.Exception: We should really have a logger for this error. ---> System.Exception: Error in getting token count for: PET Scan, Synonym: ,tomography ---> Elasticsearch.Net.UnexpectedElasticsearchClientException: A task was canceled. ---> System.Threading.Tasks.TaskCanceledException: A task was canceled. : System.Exception: We should really have a logger for this error. ---> System.Exception: Error in getting token count for: Chronic Lymphocytic Leukemia (CLL), Synonym: ,chronic lymphocytic leukaemia ---> Elasticsearch.Net.UnexpectedElasticsearchClientException: A task was canceled. ---> System.Threading.Tasks.TaskCanceledException: A task was canceled. : System.Exception: We should really have a logger for this error. ---> System.Exception: Error in getting token count for: Oral Complications/Care, Synonym: ,mouth sores ---> Elasticsearch.Net.UnexpectedElasticsearchClientException: A task was canceled. ---> System.Threading.Tasks.TaskCanceledException: A task was canceled. : System.Exception: We should really have a logger for this error. ---> System.Exception: Error in getting token count for: Temozolomide (Temodar®), Synonym: ,temadar ---> Elasticsearch.Net.UnexpectedElasticsearchClientException: A task was canceled. ---> System.Threading.Tasks.TaskCanceledException: A task was canceled. : System.Exception: We should really have a logger for this error. ---> System.Exception: Error in getting token count for: CCR, Synonym: ,CCR ---> Elasticsearch.Net.UnexpectedElasticsearchClientException: A task was canceled. ---> System.Threading.Tasks.TaskCanceledException: A task was canceled. : System.Exception: We should really have a logger for this error. ---> System.Exception: Error in getting token count for: Oral Complications/Care, Synonym: ,Oral Complications/Care ---> Elasticsearch.Net.UnexpectedElasticsearchClientException: A task was canceled. ---> System.Threading.Tasks.TaskCanceledException: A task was canceled. : System.Exception: We should really have a logger for this error. ---> System.Exception: Error in getting token count for: Diagnóstico, Synonym: ,Diagnóstico ---> Elasticsearch.Net.UnexpectedElasticsearchClientException: A task was canceled. ---> System.Threading.Tasks.TaskCanceledException: A task was canceled. : System.Exception: We should really have a logger for this error. ---> System.Exception: Error in getting token count for: Cancer Research Ideas, Synonym: ,Cancer Research Ideas ---> NCI.OCPL.Api.BestBets.APIErrorException: Errors Occurred. : System.Exception: We should really have a logger for this error. ---> System.Exception: Error in getting token count for: Cáncer de cérvix, Synonym: ,care ---> Elasticsearch.Net.UnexpectedElasticsearchClientException: A task was canceled. ---> System.Threading.Tasks.TaskCanceledException: A task was canceled. : System.Exception: We should really have a logger for this error. ---> System.Exception: Error in getting token count for: Cáncer de colon y recto, Synonym: ,ca ---> Elasticsearch.Net.UnexpectedElasticsearchClientException: A task was canceled. ---> System.Threading.Tasks.TaskCanceledException: A task was canceled. : System.Exception: We should really have a logger for this error. ---> System.Exception: Error in getting token count for: Hodgkin Lymphoma, Synonym: ,symtoms ---> Elasticsearch.Net.UnexpectedElasticsearchClientException: A task was canceled. ---> System.Threading.Tasks.TaskCanceledException: A task was canceled. : System.Exception: We should really have a logger for this error. ---> System.Exception: Error in getting token count for: Gemcitabine Hydrochloride (Gemzar®), Synonym: ,Gemcitabine Hydrochloride (Gemzar®) ---> Elasticsearch.Net.UnexpectedElasticsearchClientException: A task was canceled. ---> System.Threading.Tasks.TaskCanceledException: A task was canceled.

blairlearn commented 7 years ago

Added try/catch/retry to ESTokenAnalyzerService.GetTokenCount() in be549c727b1370e6cbdfa7a1b5663b2d4093aaa8

This appears to be a network issue rather than an actual problem with the code.

bryanpizzillo commented 7 years ago

Ok, this actually appears to be a problem with the number of connections the ElasticClient uses. (It will use them all) It looks like this is fixed in 2.5.1 as part of https://github.com/elastic/elasticsearch-net/pull/2484.

Our next issue with this item is that we need an upgraded libCurl > 7.30 in order to use the limit, or else the client blows up.

bryanpizzillo commented 5 years ago

Yeah, this was annoying. We now use Node.js to index and the code is in https://github.com/NCIOCPL/bestbets-loader.