Wrong datetime format in out logs

Michalski-Piotr commented 12 months ago

Hi , there is following error with v0.3.1 https://github.com/ailog-analytics/wls-analytics/releases/tag/v0.3.1


$ wlsanalytics --debug --traceback soa range aabc -c config/config.yaml
ERROR: '<' not supported between instances of 'NoneType' and 'datetime.datetime'
---
Traceback (most recent call last):
  File "/u01/oracle/.local/lib/python3.11/site-packages/wls_analytics/commands/click_ext.py", line 51, in invoke
    click.core.Group.invoke(self, ctx)
  File "/u01/oracle/.local/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/u01/oracle/.local/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/u01/oracle/.local/lib/python3.11/site-packages/wls_analytics/commands/click_ext.py", line 103, in invoke
    return super().invoke(ctx)
           ^^^^^^^^^^^^^^^^^^^
  File "/u01/oracle/.local/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/u01/oracle/.local/lib/python3.11/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/u01/oracle/.local/lib/python3.11/site-packages/wls_analytics/commands/log.py", line 79, in range
    if range_item["min"] is None or first < range_item["min"]:
                                    ^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: '<' not supported between instances of 'NoneType' and 'datetime.datetime'
---

It was working in the past in 0.2 version:


 $ wlsanalytics soa range aabc
SERVER       FILES  SIZE [GB]  MIN                  MAX
soa_server1  51     3.1        2023-10-02 11:23:34  2023-10-05 12:39:33
soa_server2  51     3.04       2023-10-02 12:49:29  2023-10-05 12:40:16
soa_server3  50     3.2        2023-10-02 08:56:39  2023-10-05 12:41:02
soa_server4  51     2.91       2023-10-02 15:04:40  2023-10-05 12:41:47
soa_server5  51     3.19       2023-10-02 13:02:51  2023-10-05 12:42:33
soa_server6  51     3.22       2023-10-02 11:32:59  2023-10-05 12:43:18
soa_server7  50     3.25       2023-10-02 07:47:03  2023-10-05 12:44:06
soa_server8  50     3.21       2023-10-02 10:36:39  2023-10-05 12:44:55

Regards, Piotr Michalski

Michalski-Piotr commented 12 months ago

Hello, I noticed this issue depends on log set for analysis:

Set of logs that is working fine:

$ wlsanalytics --debug --traceback soa range o2c
SERVER       FILES  SIZE [GB]  MIN                  MAX
soa_server1  102    6.62       2023-05-09 11:48:01  2023-10-11 12:07:22
soa_server2  51     2.93       2023-10-09 06:59:50  2023-10-11 12:08:53
soa_server3  51     3.39       2023-10-08 16:19:52  2023-10-11 12:09:34
soa_server4  51     2.98       2023-10-09 03:36:38  2023-10-11 12:10:20
soa_server5  51     3.51       2023-10-08 14:44:29  2023-10-11 12:11:06

Problematic log set:


wlsanalytics --debug --traceback soa range aabc
ERROR: '<' not supported between instances of 'NoneType' and 'datetime.datetime'
---
Traceback (most recent call last):
  File "/u01/oracle/.local/lib/python3.11/site-packages/wls_analytics/commands/click_ext.py", line 51, in invoke
    click.core.Group.invoke(self, ctx)
  File "/u01/oracle/.local/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/u01/oracle/.local/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/u01/oracle/.local/lib/python3.11/site-packages/wls_analytics/commands/click_ext.py", line 103, in invoke
    return super().invoke(ctx)
           ^^^^^^^^^^^^^^^^^^^
  File "/u01/oracle/.local/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/u01/oracle/.local/lib/python3.11/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/u01/oracle/.local/lib/python3.11/site-packages/wls_analytics/commands/log.py", line 79, in range
    if range_item["min"] is None or first < range_item["min"]:
                                    ^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: '<' not supported between instances of 'NoneType' and 'datetime.datetime'
---

Regards, Piotr

Michalski-Piotr commented 12 months ago

Hello, log set to reproduce has been uploaded to the test machine:


[oracle@k8s-registry data]$ pwd
/home/oracle/wlsanalytics/data
[oracle@k8s-registry data]$ ls -l aabc_20231011_logs.tar.gz
-rw-r--r-- 1 oracle oinstall 2005493783 Oct 11 13:05 aabc_20231011_logs.tar.gz

Regards, Piotr

tomvit commented 11 months ago

This problem is caused by the date time format that the out log uses which is different to the one used by other out logs. Not sure why this one is different but in any case the app should check for this error and ignore it. So there should not be any such failure but only a warning. Will fix this.

tomvit commented 11 months ago

This has been fixed in f14081aa8b5ace0f4a25420262fbb04c7723713d.

tomvit commented 11 months ago

There was one more issue related to this one when parsing log entries. This was also fixed in https://github.com/ailog-analytics/wls-analytics/commit/b9764741db4bc96edd5cf104d6b490569cf06c67.

tomvit commented 11 months ago

Fixed in release v0.3.2: https://pypi.org/project/wls-analytics/

ailog-analytics / wls-analytics

Wrong datetime format in out logs #3