search

EIDA / eida-statistics

Aggregated statistics of EIDA nodes
GNU General Public License v3.0
0 stars 0 forks source link

Network identifier #6

Closed jschaeff closed 2 years ago

jschaeff commented 2 years ago

The aggregator identifies networks only with 2 letters. This is wrong.

Commit 86a36cf7 fixes this.

jschaeff commented 2 years ago

Can someone test this new aggregator ?

  1. install the new version with the pipenv method (https://github.com/EIDA/eida-statistics/tree/main/aggregator#install-and-execute)
  2. parse an existing logfile and look at the result to see if it looks fine
  3. send the statistics

@kbouk thank you for volunteering :) Let me know how it goes in this issue.

kbouk commented 2 years ago

Hello @jschaeff ,,

I installed the new version (pip3 install eida-statistics-aggregator), however when running it I get the following error (using python 3.6.9):

_eida_stats_aggregator --output-directory aggregates fdsnws_request.log.2022-06-11.bz2 Parsing fdsnws_request.log.2022-06-11.bz2 [------------------------------------] 0% Traceback (most recent call last): File "/usr/local/lib/python3.6/dist-packages/fdsnnetextender/fdsnnetextender.py", line 31, in extend dateparam = date(year=int(date_string), month=1, day=1) ValueError: invalid literal for int() with base 10: '2013-02-15'

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/usr/local/bin/eida_stats_aggregator", line 11, in load_entry_point('eida-statistics-aggregator==0.6.0', 'console_scripts', 'eida_stats_aggregator')() File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 1128, in call return self.main(args, kwargs) File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 1053, in main rv = self.invoke(ctx) File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 1395, in invoke return ctx.invoke(self.callback, ctx.params) File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 754, in invoke return __callback(args, **kwargs) File "/usr/local/lib/python3.6/dist-packages/aggregator/aggregator.py", line 235, in cli statistics.parse_file(f) File "/usr/local/lib/python3.6/dist-packages/aggregator/aggregator.py", line 193, in parse_file extended_network = net_extender.extend(trace['net'], trace['start'][0:10]) File "/usr/local/lib/python3.6/dist-packages/fdsnnetextender/fdsnnetextender.py", line 35, in extend dateparam = date.fromisoformat(datestring) AttributeError: type object 'datetime.date' has no attribute 'fromisoformat'

jschaeff commented 2 years ago

The function datetime.date.fromisoformat() has been introduced in Python 3.7

Do you think the program should be python3.6 compatible ?

javiquinte commented 2 years ago

If it's only this error, it would be nice to make it 3.6 compatible. Probably a simple replacement with strptime could be enough.

jschaeff commented 2 years ago

It would be enough. It's just soo less fun :)

jschaeff commented 2 years ago

@kbouk you have to upgrade a dependency of this project : fdsnnetextender == 3.2.0 Can you give it another try ?

jschaeff commented 2 years ago

https://pypi.org/project/fdsnnetextender/

kbouk commented 2 years ago

Hello @jschaeff,

Yes it works fine now with python 3.6. I am attaching some outputs. 2022-06-12_2022-06-12.json.gz 2022-06-12_2022-06-13.json.gz

jschaeff commented 2 years ago

Thank you @kbouk I was able to ingest the result on my development environment. There was no temporary network in your example, and I would like to test the aggregator with some usage stats from temporary network. I validated that there is no regression bug though.

@all can someone test the new aggregator whith temporary networks involved ?

jschaeff commented 2 years ago

I validated also with temporary networks. The fix is ready to hit the production.