search

lareferencia / dspace-stats-collector

Sends repository usage statistics events to Matomo
Other
13 stars 2 forks source link

ERROR: A fatal exception ocurred processing events !!!! 'ip' #13

Closed arieljlira closed 3 years ago

arieljlira commented 3 years ago

Al parecer cuando se recuperan eventos de solr sin el campo ip, la aplicación falla y se cierra. Como retoma desde el mismo punto en el que dejó, queda trabada y no procesa más ítems.

En los logs: 

INFO: Finished 'http://my_solr_url:8080/solr/statistics/select/?q=%2A+%2Btime%3A%7B%222020-11-16T11%3A57%3A16.387Z%22+TO+%2A%5D&sort=time+asc&start=0&wt=json&fq=%2Bstatistics_type%3A%22view%22+%2Btype%3A%280+OR+2%29&fl=id%2Cip%2CowningItem%2Creferrer%2Ctime%2Ctype%2CuserAgent&rows=1000' (get) with body '' in 53.403 seconds, with status 200
ERROR: A fatal exception ocurred processing events !!!! 'ip'
termina la ejecución (mar oct  5 08:46:27 -03 2021)
INFO: Finished 'http://my_solr_url:8080/solr/statistics/select/?q=%2A+%2Btime%3A%7B%222020-11-16T11%3A57%3A16.387Z%22+TO+%2A%5D&sort=time+asc&start=0&wt=json&fq=%2Bstatistics_type%3A%22view%22+%2Btype%3A%280+OR+2%29&fl=id%2Cip%2CowningItem%2Creferrer%2Ctime%2Ctype%2CuserAgent&rows=1000' (get) with body '' in 54.192 seconds, with status 200
ERROR: A fatal exception ocurred processing events !!!! 'ip'
termina la ejecución (mar oct  5 09:01:22 -03 2021)
INFO: Finished 'http://my_solr_url:8080/solr/statistics/select/?q=%2A+%2Btime%3A%7B%222020-11-16T11%3A57%3A16.387Z%22+TO+%2A%5D&sort=time+asc&start=0&wt=json&fq=%2Bstatistics_type%3A%22view%22+%2Btype%3A%280+OR+2%29&fl=id%2Cip%2CowningItem%2Creferrer%2Ctime%2Ctype%2CuserAgent&rows=1000' (get) with body '' in 54.719 seconds, with status 200
ERROR: A fatal exception ocurred processing events !!!! 'ip'
termina la ejecución (mar oct  5 09:46:32 -03 2021)
INFO: Finished 'http://my_solr_url:8080/solr/statistics/select/?q=%2A+%2Btime%3A%7B%222020-11-16T11%3A57%3A16.387Z%22+TO+%2A%5D&sort=time+asc&start=0&wt=json&fq=%2Bstatistics_type%3A%22view%22+%2Btype%3A%280+OR+2%29&fl=id%2Cip%2CowningItem%2Creferrer%2Ctime%2Ctype%2CuserAgent&rows=1000' (get) with body '' in 60.873 seconds, with status 200
ERROR: A fatal exception ocurred processing events !!!! 'ip'
termina la ejecución (mar oct  5 10:01:13 -03 2021)

En los datos recuperados de solr hay casos normales, como por ejemplo: {"ip":"78.234.228.111","referrer":"https://scholar.google.com/","userAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.198 Safari/537.36","id":69532,"type":2,"time":"2020-11-16T12:28:13.003Z"}, y algunos (150) sin campo "ip":

{"id":103030,"type":0,"owningItem":[55360],"time":"2020-11-16T12:11:33.838Z"},
{"id":103030,"type":0,"owningItem":[55360],"time":"2020-11-16T12:11:33.927Z"},
{"id":104830,"type":0,"owningItem":[55966],"time":"2020-11-16T12:11:34.124Z"},
{"id":10532,"type":0,"owningItem":[4178],"time":"2020-11-16T12:11:34.211Z"},
{"id":108970,"type":0,"owningItem":[57869],"time":"2020-11-16T12:11:34.13Z"},
{"id":1095,"type":0,"owningItem":[1186],"time":"2020-11-16T12:11:34.216Z"},
lmatas commented 3 years ago

Ariel, Muchas gracias por este reporte, vamos a resolverlo esta semana y te avisamos. Tengo pendiente responderte y contarte otras cosas. Lautaro Matas

El mar, 5 oct 2021 a las 15:44, Ariel Lira @.***>) escribió:

Al parecer cuando se recuperan eventos de solr sin el campo ip, la aplicación falla y se cierra. Como retoma desde el mismo punto en el que dejó, queda trabada y no procesa más ítems.

En los logs:

INFO: Finished 'http://my_solr_url:8080/solr/statistics/select/?q=%2A+%2Btime%3A%7B%222020-11-16T11%3A57%3A16.387Z%22+TO+%2A%5D&sort=time+asc&start=0&wt=json&fq=%2Bstatistics_type%3A%22view%22+%2Btype%3A%280+OR+2%29&fl=id%2Cip%2CowningItem%2Creferrer%2Ctime%2Ctype%2CuserAgent&rows=1000' (get) with body '' in 53.403 seconds, with status 200

ERROR: A fatal exception ocurred processing events !!!! 'ip'

termina la ejecución (mar oct 5 08:46:27 -03 2021)

INFO: Finished 'http://my_solr_url:8080/solr/statistics/select/?q=%2A+%2Btime%3A%7B%222020-11-16T11%3A57%3A16.387Z%22+TO+%2A%5D&sort=time+asc&start=0&wt=json&fq=%2Bstatistics_type%3A%22view%22+%2Btype%3A%280+OR+2%29&fl=id%2Cip%2CowningItem%2Creferrer%2Ctime%2Ctype%2CuserAgent&rows=1000' (get) with body '' in 54.192 seconds, with status 200

ERROR: A fatal exception ocurred processing events !!!! 'ip'

termina la ejecución (mar oct 5 09:01:22 -03 2021)

INFO: Finished 'http://my_solr_url:8080/solr/statistics/select/?q=%2A+%2Btime%3A%7B%222020-11-16T11%3A57%3A16.387Z%22+TO+%2A%5D&sort=time+asc&start=0&wt=json&fq=%2Bstatistics_type%3A%22view%22+%2Btype%3A%280+OR+2%29&fl=id%2Cip%2CowningItem%2Creferrer%2Ctime%2Ctype%2CuserAgent&rows=1000' (get) with body '' in 54.719 seconds, with status 200

ERROR: A fatal exception ocurred processing events !!!! 'ip'

termina la ejecución (mar oct 5 09:46:32 -03 2021)

INFO: Finished 'http://my_solr_url:8080/solr/statistics/select/?q=%2A+%2Btime%3A%7B%222020-11-16T11%3A57%3A16.387Z%22+TO+%2A%5D&sort=time+asc&start=0&wt=json&fq=%2Bstatistics_type%3A%22view%22+%2Btype%3A%280+OR+2%29&fl=id%2Cip%2CowningItem%2Creferrer%2Ctime%2Ctype%2CuserAgent&rows=1000' (get) with body '' in 60.873 seconds, with status 200

ERROR: A fatal exception ocurred processing events !!!! 'ip'

termina la ejecución (mar oct 5 10:01:13 -03 2021)

En los datos recuperados de solr hay casos normales, como por ejemplo: {"ip":"78.234.228.111","referrer":"https://scholar.google.com/","userAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.198 Safari/537.36","id":69532,"type":2,"time":"2020-11-16T12:28:13.003Z"}, y algunos (150) sin campo "ip":

{"id":103030,"type":0,"owningItem":[55360],"time":"2020-11-16T12:11:33.838Z"},

{"id":103030,"type":0,"owningItem":[55360],"time":"2020-11-16T12:11:33.927Z"},

{"id":104830,"type":0,"owningItem":[55966],"time":"2020-11-16T12:11:34.124Z"},

{"id":10532,"type":0,"owningItem":[4178],"time":"2020-11-16T12:11:34.211Z"},

{"id":108970,"type":0,"owningItem":[57869],"time":"2020-11-16T12:11:34.13Z"},

{"id":1095,"type":0,"owningItem":[1186],"time":"2020-11-16T12:11:34.216Z"},

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/lareferencia/dspace-stats-collector/issues/13, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAH7G6QL6LYTZZLT35UREKDUFL6L7ANCNFSM5FL2K2UA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

lareferencia commented 3 years ago

@arieljlira esto está arreglado estimo, sin embargo no tengo tu caso de test, te agradezco si podes hacer un update con el pip para que traiga la última versión, que también tiene otros fix de memo, sería la 0.5.5. con eso debiera ser más eficiente, gracias a tu sugerencia del solr y no quedarse ciclando en este tema de la ip. Muchas gracias de antemano por todo

arieljlira commented 3 years ago

Buen día, confirmo que en 0.5.5 ya no falla por ip faltante. Ahora siguie esta procesando sin problemas. Muchas gracias!

lmatas commented 3 years ago

Muchas gracias Ariel, luego contanos en el issue de tu propuesta sobre el tope de fecha en solr si tus tiempos mejoraron. Un abrazo Lautaro Matas

El lun, 18 oct 2021 a las 14:08, Ariel Lira @.***>) escribió:

Closed #13 https://github.com/lareferencia/dspace-stats-collector/issues/13.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/lareferencia/dspace-stats-collector/issues/13#event-5477900724, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAH7G6VATKX4EXQ43HEIS6TUHQE35ANCNFSM5FL2K2UA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.