Open 1000i100 opened 4 years ago
After reading this : https://github.com/matomo-org/matomo-log-analytics/issues/282#issuecomment-716676600
i've updated import_logs.py to the latest from this repository.
I've change in first line : #!/usr/bin/python
to #!/usr/bin/python3
and it now have the same behavor with this 2 way for calling it :
cat /var/tmp/nginx.tmp | ./misc/log-analytics/import_logs.py [...] -
./misc/log-analytics/import_logs.py [...] /var/tmp/nginx.tmp
But it does'nt import anything :
Traceback (most recent call last):
File "/usr/lib/python3.5/threading.py", line 914, in _bootstrap_inner
self.run()
File "/usr/lib/python3.5/threading.py", line 862, in run
self._target(*self._args, **self._kwargs)
File "******/misc/log-analytics/import_logs.py", line 1849, in _run_bulk
self._record_hits(hits)
File ""******/misc/log-analytics/import_logs.py", line 1995, in _record_hits
'requests': [self._get_hit_args(hit) for hit in hits]
File ""******/misc/log-analytics/import_logs.py", line 1995, in <listcomp>
'requests': [self._get_hit_args(hit) for hit in hits]
File ""******/misc/log-analytics/import_logs.py", line 1891, in _get_hit_args
site_id, main_url = resolver.resolve(hit)
File ""******/misc/log-analytics/import_logs.py", line 1775, in resolve
return self._resolve_by_host(hit)
File ""******/misc/log-analytics/import_logs.py", line 1758, in _resolve_by_host
site_id = self._resolve(hit)
File ""******/misc/log-analytics/import_logs.py", line 1726, in _resolve
res = self._get_site_id_from_hit_host(hit)
File ""******/misc/log-analytics/import_logs.py", line 1681, in _get_site_id_from_hit_host
url=hit.host,
File ""******/misc/log-analytics/import_logs.py", line 1633, in call_api
return self._call_wrapper(self._call_api, None, None, method, **kwargs)
File ""******/misc/log-analytics/import_logs.py", line 1585, in _call_wrapper
response = func(*args, **kwargs)
File ""******/misc/log-analytics/import_logs.py", line 1574, in _call_api
return json.loads(res)
File "/usr/lib/python3.5/json/__init__.py", line 312, in loads
s.__class__.__name__))
TypeError: the JSON object must be str, not 'bytes'
1365 lines parsed, 0 lines recorded, 0 records/sec (avg), 0 records/sec (current)
[... Ctrl+C to stop it ...]
1365 lines parsed, 0 lines recorded, 0 records/sec (avg), 0 records/sec (current)
Logs import summary
-------------------
0 requests imported successfully
17 requests were downloads
0 requests ignored:
0 HTTP errors
0 HTTP redirects
0 invalid log lines
0 filtered log lines
0 requests did not match any known site
0 requests did not match any --hostname
0 requests done by bots, search engines...
0 requests to static resources (css, js, images, ico, ttf...)
0 requests to file downloads did not match any --download-extensions
Website import summary
----------------------
0 requests imported to 0 sites
0 sites already existed
0 sites were created:
0 distinct hostnames did not match any existing site:
Performance summary
-------------------
Total time: 149 seconds
Requests imported per second: 0.0 requests per second
Processing your log data
------------------------
In order for your logs to be processed by Matomo, you may need to run the following command:
./console core:archive --force-all-websites --force-all-periods=315576000 --force-date-last-n=1000 --url='****'
It probably have some comon point with #258
I've followed the rsyslog part of this : https://github.com/matomo-org/matomo-log-analytics#how-do-i-setup-nginx-to-directly-import-to-matomo-via-syslog
and after some folder manual creation and the change of
/var/cache/nginx/access.socket
by/dev/log
innginx.conf
and10-matomo.conf
if i uncomment the debug linelocal0.* /var/tmp/nginx.tmp;matomo
, log apear in/var/tmp/nginx.tmp
If i try to import them with
matomo.sh
it show me the script run, but import nothing.If i try to run
import_logs.py
itself, it work.So i suspect there is something not working in
/usr/local/matomo/matomo.sh
found nothing to import but :
work
Is there some documentation to fix in README.md about rsyslog
/usr/local/matomo/matomo.sh
syntax :or is it something to fix in
misc/log-analytics/import_logs.py
?or anything else i've not spotted ?