google / timesketch

Collaborative forensic timeline analysis
Apache License 2.0
2.61k stars 588 forks source link

Impossible to create many timelines from 1 OS index #3166

Open mcichorzA opened 2 months ago

mcichorzA commented 2 months ago

Describe the bug Timesketch API doesn't allow to create many timelines from 1 ES index and also has bug in TimelineListResource. Currently I am not able to create 2 timelines using 1 OpenSearch index (not counting web upload of csv/json/plaso which supports that). I want to have 1 index per sketch. I upload data directly to OpenSearch with psort (data is already with timeline id set properly). Currently I have found impossible to create 2 Timelines from such 1 index.

Things I have tried:

  1. generate_timeline_from_es_index https://github.com/google/timesketch/blob/68c5c9bfda2ab0a8b37cc64c1ad852ddb7f3a24e/api_client/python/timesketch_api_client/sketch.py#L1965-L1970 This doesnt allow passing es_index_name of index which already is assigned to other SearchIndex (index_obj.index_name is the actual name of opensearch index, not the same thing as index_name passed to this funciton.
  2. Using API directly https://github.com/google/timesketch/blob/b9e32b1e3ac981335185f9bdb16c1f51dd634941/timesketch/api/v1/resources/timeline.py#L69-L143 This won't work because
    timeline_id = [
            t.searchindex.id
            for t in sketch.timelines
            if t.searchindex.id == searchindex_id
        ]

    this will save to variable timeline_id ID of already existing searchindex (cool), but then it wont enter IF when Timeline is created https://github.com/google/timesketch/blob/b9e32b1e3ac981335185f9bdb16c1f51dd634941/timesketch/api/v1/resources/timeline.py#L140-L143 Whats strange it will return TIMELINE object based on the SEARCHINDEX ID which is a BUG, because if you look at tables of Searchindex and TImelines theirs IDs dont match up.

To Reproduce Steps to reproduce the behavior:

  1. Create dummy index in OS.
  2. Create first timeline from this index (eg with generate_timeline_from_es_index)
  3. Try to create another timeline with the same index <- Impossible with API

Expected behavior Similar to the situation when 2 csv/json files are uploaded to the same index, it should be also possible from API

Screenshots N/A

Desktop (please complete the following information): N/A

Additional context I can probide if asked

mcichorzA commented 2 months ago

Perhaps this could workout, but I dont see it in routes.

https://github.com/google/timesketch/blob/b9e32b1e3ac981335185f9bdb16c1f51dd634941/timesketch/api/v1/resources/timeline.py#L459-L533