google / timesketch

Collaborative forensic timeline analysis
Apache License 2.0
2.52k stars 577 forks source link

Timelines import successfully but no entries are showing up #3084

Open Kamforka opened 1 month ago

Kamforka commented 1 month ago

Describe the bug I'm deploying a fresh instance of timesketch for local testing. According to the service logs everything is running as expected. Then I create a sketch and import a timeline csv. The upload is successful, the import is successful, but no entries show up in the timeline UI and the timeline's badge shows 0 entries. However when I check the data sources information of the timeline it correctly displays the number of entries originally contained in the csv: image

It's happening everytime with all the sketches and with all the timeline files, let them be small or large.

Can you please help me figure out what I'm doing wrong? We have an older deployment (from 2021) which is working perfectly fine, and now I'm migrating to the latest version but for some reason I cannot make a fresh deployment work even with the simplest setup.

Expected behavior After the successful upload and import the timeline entries should show up.

Desktop (please complete the following information):

Deployment information:

jkppr commented 1 month ago

Hi @Kamforka , thanks for bringing this to our attention. In general, the number of events (counter) next to the timeline displays how many events are found for your current search. Given you are using a deployment from 2021, this is probably still with the old UI and the behaviour of the counter could have been different back then.

Let's check the low hanging fruits first:

Kamforka commented 1 month ago

Hi @jkppr and thanks for the quick reply!

No database migration yet as this is a brand new instance without any legacy data.

Oh well, searching for * did the trick:

image

I have a couple of weird observations though, first being the datetime values, look like broken. Then I cannot click on any data_types in the left panel as it's showing 0 for some reason. Is that a possible misconfiguration or normal?

jkppr commented 1 month ago

You mentioned above that you are uploading CSV files. Does your CSV entries have a field called data_type to begin with? Is this information also missing, when you upload a Plaso file?

Kamforka commented 1 month ago

Hi @jkppr yeah indeed we don't have the data_type field in our input. Is that also the reason for the cryptic datetime display?

I've successfully migrated our deployment to the latest timesketch version with the opensearch backend, and I've imported a timeline today but I still don't know what's causing this datetime bug:

image

As you can see the row's datetime column displays the weird value of +056316-01-27T04:00:00.000Z while the actual timeline entry has a proper datetime value of 2024-05-06T11:51:36+00:00, also the timestamp value of 1714996296 is looking okay to me.

Any guesses what can cause this bug?

Kamforka commented 1 month ago

I did some research and looks like the past version of timesketch accepted timestamps up to the second precision but now only microsecond precision works.

Is this something that can be configured or should I change our timeline generator tool to produce microsecond precision timestamps?