Erro no check do SciELO

[anakatiacamilo]

G4 serverofi5:/bases/fiadmin2/DeDup/tpl $ ./01_Dedup_proc.sh sci_201710 SciELO iso-8859-1 http://serverofi5.bireme.br:8180/DeDup/services lilacs_Sas LILACS_Sas_Seven [TIME-STAMP] 2017.10.09 16:17:56 [:INI:] Processa ./01_Dedup_proc.sh sci_201710 SciELO iso-8859-1 http://serverofi5.bireme.br:8180/DeDup/services

• Verificacoes iniciais Gera Arquivo para checagem de Duplicacao! Diretorio de processamento /home/javaapps/sbt-projects/DCDup/sh Diretorio de trabalho e arquivo /bases/fiadmin2/DeDup/wrk/sci_201710/in_sci_201710.txt Formato utilizado pft=if v5.1='S' then s1:=(if p(v10) then (	//@//	+v10^*) else (	//@//	+v11^) fi), s3:=(v30^),(if p(v12) then 'LILACS_Sas	',v2[1],'	',replace(v12^*,'	',''),'	',s3,'	',v65[1].4,'	',v31[1],'	',v32[1]'	',s1,'	',if v14[1]^>'' then v14[1]^ fi,/ fi),(if p(v13) then 'LILACS_Sas	',v2[1],'	',replace(v13^*,'	',''),'	',s3,'	',v65[1].4,'	',v31[1],'	',v32[1]'	',s1,'	',if v14[1]^>'' then v14[1]^ fi,/ fi),fi /bases/lilG4/lxp.sci/aux/gpipe
  /usr/local/bireme/cisis/5.7c/linux64/ffiG_1024K-256/mx /bases/fiadmin2/DeDup/wrk/sci_201710/sci_201710_LILACS gizmo=/bases/lilG4/lxp.sci/aux//gpipe,10 "pft=if v706:'h' then s1:=(if p(v10) then (	//@//	+v10^s	,	v10^n) else (	//@//	+v11^) fi), s3:=(v30^),(if p(v12) then 'LILACS_Sas	',v2[1],'	',replace(v12^*,'	',''),'	',s3,'	',v65[1].4,'	',v31[1],'	',v32[1]'	',s1,'	',if v14[1]^>'' then v14[1]^ fi,/ fi),(if p(v13) then 'LILACS_Sas	',v2[1],'	',replace(v13^*,'	',''),'	',s3,'	',v65[1].4,'	',v31[1],'	',v32[1]'	',s1,'	',if v14[1]^>'' then v14[1]^ fi,/ fi),fi" lw=0 tell=50000 now > /bases/fiadmin2/DeDup/wrk/sci_201710/in_sci_201710_SciELO.txt

  +++ 50000 +++ 100000 +++ 150000 +++ 200000 +++ 250000 Comando para check de DeDup /home/javaapps/sbt-projects/DCDup/sh/DoubleCheckDuplicated.sh /bases/fiadmin2/DeDup/wrk/sci_201710/in_sci_201710_SciELO.txt iso-8859-1 http://serverofi5.bireme.br:8180/DeDup/services lilacs_Sas LILACS_Sas_Seven /bases/fiadmin2/DeDup/outs/sci_201710/out1_sci_201710_SciELO.txt /bases/fiadmin2/DeDup/outs/sci_201710/out2_sci_201710_SciELO.txt /bases/fiadmin2/DeDup/outs/sci_201710/out_ok_sci_201710_SciELO.txt iso-8859-1 [info] Set current project to DCDup (in build file:/home/javaapps/sbt-projects/DCDup/) [info] Running org.bireme.dcdup.WebDoubleCheckDuplicated /bases/fiadmin2/DeDup/wrk/sci_201710/in_sci_201710_SciELO.txt iso-8859-1 http://serverofi5.bireme.br:8180/DeDup/services lilacs_Sas LILACS_Sas_Seven /bases/fiadmin2/DeDup/outs/sci_201710/out1_sci_201710_SciELO.txt /bases/fiadmin2/DeDup/outs/sci_201710/out2_sci_201710_SciELO.txt /bases/fiadmin2/DeDup/outs/sci_201710/out_ok_sci_201710_SciELO.txt iso-8859-1

Self check <<< 0 <<< 1000 <<< 2000 <<< 3000 <<< 4000 <<< 5000 <<< 6000

<<< 455000 <<< 456000 <<< 457000 <<< 458000 <<< 459000 [error] (run-main-0) java.io.IOException: id java.io.IOException: id at br.bireme.ngrams.NGrams.indexDocument(NGrams.java:203) at org.bireme.dcdup.WebDoubleCheckDuplicated.$anonfun$localCheck$1(WebDoubleCheckDuplicated.scala:160) at org.bireme.dcdup.WebDoubleCheckDuplicated.$anonfun$localCheck$1$adapted(WebDoubleCheckDuplicated.scala:152) at scala.collection.Iterator.foreach(Iterator.scala:929) at scala.collection.Iterator.foreach$(Iterator.scala:929) at scala.collection.AbstractIterator.foreach(Iterator.scala:1417) at org.bireme.dcdup.WebDoubleCheckDuplicated.localCheck(WebDoubleCheckDuplicated.scala:152) at org.bireme.dcdup.WebDoubleCheckDuplicated.doubleCheck(WebDoubleCheckDuplicated.scala:94) at org.bireme.dcdup.WebDoubleCheckDuplicated$.delayedEndpoint$org$bireme$dcdup$WebDoubleCheckDuplicated$1(WebDoubleCheckDuplicated.scala:67) at org.bireme.dcdup.WebDoubleCheckDuplicated$delayedInit$body.apply(WebDoubleCheckDuplicated.scala:46) at scala.Function0.apply$mcV$sp(Function0.scala:34) at scala.Function0.apply$mcV$sp$(Function0.scala:34) at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:12) at scala.App.$anonfun$main$1$adapted(App.scala:76) at scala.collection.immutable.List.foreach(List.scala:389) at scala.App.main(App.scala:76) at scala.App.main$(App.scala:74) at org.bireme.dcdup.WebDoubleCheckDuplicated$.main(WebDoubleCheckDuplicated.scala:46) at org.bireme.dcdup.WebDoubleCheckDuplicated.main(WebDoubleCheckDuplicated.scala) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) [trace] Stack trace suppressed: run last compile:runMain for the full output. java.lang.RuntimeException: Nonzero exit code: 1 at scala.sys.package$.error(package.scala:27) [trace] Stack trace suppressed: run last compile:runMain for the full output. [error] (compile:runMain) Nonzero exit code: 1 [error] Total time: 23660 s, completed Oct 9, 2017 10:52:39 PM /bases/fiadmin2/DeDup/tpl

Fim de processamento

DURACAO DE PROCESSAMENTO

• Inicio: 2017.10.09 16:17:56

• Termino: 2017.10.09 22:52:39

  Tempo de execucao: 23683 [s] Ou 6h 34m 43s

[TIME-STAMP] 2017.10.09 22:52:39 [:FIM:] Processa ./01_Dedup_proc.sh sci_201710 SciELO iso-8859-1 http://serverofi5.bireme.br:8180/DeDup/services

[heitorbarbieri]

O erro reportado acima deveu-se ao fato de que o arquivo pipe gerado apresentava linha onde o ítem obrigatório 'id' não estava presente. Este erro gerou a criação do tíquete #7 para que se verique a integridade do arquivo pipe de entrada antes de se iniciar o processo de checagem de duplicados.