Some valid ISO 8601 timestamps are not parsed ("Invalid Date" is shown)

Samyak2 commented 1 year ago

Bug description

Valid ISO 8601 timestamps in the timestamp attribute are sometimes not recognized correctly. The UI shows "Invalid Date" for timestamp.

Example timestamp: 2023-01-27T11:20:17.619927Z.

These timestamps are generated by structlog's timestamper when these parameters are used:

structlog.processors.TimeStamper(fmt="iso", utc=True, key="timestamp")

The timestamp appears to be a valid ISO 8601 timestamp. For example, this jsfiddle in this answer says that the timestamp is valid. The ISO 8601 spec also seems to recognize this format.

Although, python's standard library does not recognize it (which ironic considering it originated from a python library):

>>> from datetime import datetime
>>> datetime.fromisoformat("2023-01-27T11:20:17.619927Z")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: Invalid isoformat string: '2023-01-27T11:20:17.619927Z'

I believe that it is structlog which is wrong. It uses datetime.utcnow() and explicitly adds a Z to add the timestamp. Instead, it should use datetime.now(timezone.utc). I will open a bug in structlog for this, but I think SigNoz should support timestamps given by older version of structlog too.

Expected behavior

The timestamp should be parsed and shown correctly in the UI.

How to reproduce

Instrument a python app that uses structlog to ship logs to Signoz with the Timestamper processor added. Note: steps for this are unclear and need to be added.
Notice that logs in Signoz show "Invalid Date"

Version information

Signoz version: v0.14.0
Browser version: Firefox 108
Your OS and version: Fedora 34 client. Ubuntu 21.04 server.
Your CPU Architecture(ARM/Intel): Intel x86 on both

Additional context

Slack discussion: https://signoz-community.slack.com/archives/C01HWQ1R0BC/p1674827471752029?thread_ts=1674825561.347139&cid=C01HWQ1R0BC

welcome[bot] commented 1 year ago

Thanks for opening this issue. A team member should give feedback soon. In the meantime, feel free to check out the contributing guidelines.

srikanthccv commented 1 year ago

The issue is not clear to me. Are you creating a record with an ISO timestamp? The timestamp on the log record needs to be epoch nanos.

SigNoz / signoz