search

log2timeline / plaso

Super timeline all the things
https://plaso.readthedocs.io
Apache License 2.0
1.73k stars 351 forks source link

Plaso import into Elastic - NumberFormatException #1156

Closed n3x77 closed 6 years ago

n3x77 commented 7 years ago

Plaso version: Plaso Version 1.5.1

Operating system Plaso is running on: Ubuntu 16.04 LTS

Installation method: Installed from [GiFT]

Description of problem: After parsing an Disk-Image of a Windows server, we tried to import the Plaso-Storage Container into Elasticsearch Version 2.4 with psort. After importing a few thousand entries the elasticsearch instance crashes. In the elasticsearch log we get the following java exception: MapperParsingException[failed to parse [file_reference]]; nested: NumberFormatException[For input string: "62357-9"]; It seems that the field file_reference that is written by the mft parse has a wrong type. Kibana shows that it has the format number, bot should be string.

Debug output/tracebacks:

[2017-01-31 15:13:46,865][DEBUG][action.bulk              ] [elk-template] [plaso_test_0][1] failed to execute bulk item (index) index {[plaso_test_0][plaso_event][AVn039THEZ9hrjJ-SnQx], source[{"origin": "exampledata replacement manual.lnk", "parser": "lnk/shell_items", "datetime": "2010-02-03T12:49:52+00:00", "long_name": "exampledata.pdf", "tag": [], "message": "Name: exampledataR~1.PDF Long name: exampledata.pdf NTFS file reference: 62357-9 Shell item path: exampledata.pdf Origin: exampledata.lnk", "sha256_hash": "bce6528ca6f8299dbd512c3531007ff435a40a16e554d391978d21e223ced809", "display_name": "TSK:/Users/localadmin/AppData/Roaming/Microsoft/Windows/Recent/exampledata.lnk", "uuid": "f35c4b3d3bdc427ca42021146b917b08", "filename": "/Users/localadmin/AppData/Roaming/Microsoft/Windows/Recent/exampledata.lnk", "source_short": "FILE", "pathspec": "{\"inode\": 110438, \"type_indicator\": \"TSK\", \"__type__\": \"PathSpec\", \"location\": \"/Users/localadmin/AppData/Roaming/Microsoft/Windows/Recent/exampledata.lnk\", \"parent\": {\"parent\": {\"type_indicator\": \"RAW\", \"__type__\": \"PathSpec\", \"parent\": {\"type_indicator\": \"OS\", \"__type__\": \"PathSpec\", \"location\": \"/mnt/share/images/2017-01-23/disk2.img\"}}, \"__type__\": \"PathSpec\", \"start_offset\": 105906176, \"type_indicator\": \"TSK_PARTITION\", \"location\": \"/p2\", \"part_index\": 3}}", "username": "-", "store_number": 1, "shell_item_path": "\\exampledata.pdf", "data_type": "windows:shell_item:file_entry", "timestamp": 1265201392000000, "store_index": 720, "source_long": "File entry shell item", "name": "exampledataR~1.PDF", "timestamp_desc": "Content Modification Time", "file_reference": "62357-9", "sha1_hash": "e6065ce8ce442532114722deb6e9534aa1093772", "md5_hash": "ced8817578be0f7b37ccb23710b712c5"}]}
MapperParsingException[failed to parse [file_reference]]; nested: NumberFormatException[For input string: "62357-9"];
        at org.elasticsearch.index.mapper.FieldMapper.parse(FieldMapper.java:329)
        at org.elasticsearch.index.mapper.DocumentParser.parseObjectOrField(DocumentParser.java:311)
        at org.elasticsearch.index.mapper.DocumentParser.parseValue(DocumentParser.java:438)
        at org.elasticsearch.index.mapper.DocumentParser.parseObject(DocumentParser.java:264)
        at org.elasticsearch.index.mapper.DocumentParser.parseDocument(DocumentParser.java:124)
        at org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:309)
        at org.elasticsearch.index.shard.IndexShard.prepareCreate(IndexShard.java:533)
        at org.elasticsearch.index.shard.IndexShard.prepareCreateOnPrimary(IndexShard.java:510)
        at org.elasticsearch.action.index.TransportIndexAction.prepareIndexOperationOnPrimary(TransportIndexAction.java:214)
        at org.elasticsearch.action.index.TransportIndexAction.executeIndexRequestOnPrimary(TransportIndexAction.java:223)
        at org.elasticsearch.action.bulk.TransportShardBulkAction.shardIndexOperation(TransportShardBulkAction.java:327)
        at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:120)
        at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:68)
        at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryPhase.doRun(TransportReplicationAction.java:657)
        at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
        at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:287)
        at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:279)
        at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:77)
        at org.elasticsearch.transport.TransportService$4.doRun(TransportService.java:378)
        at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NumberFormatException: For input string: "62357-9"
        at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
        at java.lang.Long.parseLong(Long.java:589)
        at java.lang.Long.parseLong(Long.java:631)
        at org.elasticsearch.common.xcontent.support.AbstractXContentParser.longValue(AbstractXContentParser.java:146)
        at org.elasticsearch.index.mapper.core.LongFieldMapper.innerParseCreateField(LongFieldMapper.java:281)
        at org.elasticsearch.index.mapper.core.NumberFieldMapper.parseCreateField(NumberFieldMapper.java:241)
        at org.elasticsearch.index.mapper.FieldMapper.parse(FieldMapper.java:321)

Source data: Image of a Windows server system.

Onager commented 7 years ago

@berggren Can you take a look at this?

n3x77 commented 7 years ago

Hi Guys,

after the manual creation of a specific elasticsearch index template it is possible to import the data of the mft parser.

Maybe we should create an elasticsearch index template, where the data types are correctly defined.

Cheers

berggren commented 7 years ago

This is most probably because the field is sometimes numeric (e.g. 62357) and sometimes string (e.g. 62357-9). Elastic will use the first appearance to create the dynamic mapping. We should probably add an explicit mapping for it in the ElasticOutput

berggren commented 7 years ago

@n3x77 Hi, I'm trying to reproduce this but have been unsuccessful so far. I noticed in your report that you used a elastic template calles elk-template. Can you share that template so I can have a look?

Onager commented 7 years ago

Sounds like this might be due to a template we don't use, so removing from milestone

n3x77 commented 7 years ago

Hi, sorry for the late reply.

@berggren Thats exactly what happened so the filed was numeric at the beginning and then changed to string. I just parsed a few more images into elasticsearch and exported the field mappings. After that i changed the type of the field "file_reference" manually.

@Onager I didn't create an specific template that caused the issue. At the beginning I Started with the Dynamic mapping, the specific template was created to solve the issue.

I think either we could change from dynamic mapping to a static template that is pushed by plaso, or you could implement a feature that scans the pstorage file for all fields where different types exist.

I will create a test template and share it within the post until friday.

n3x77 commented 6 years ago

Sorry for the delay, after getting some errors with the parsing of file_reference, we're currently using the following elasticstearch template mapping, which sets the file_reference to type text. This has been tested wit the current plaso release 20180127 and elasticsearch 5.


      "plaso_event": {
        "dynamic_templates": [
          {
            "strings": {
              "match_mapping_type": "string",
              "mapping": {
                "fields": {
                  "raw": {
                    "ignore_above": 10922,
                    "index": "not_analyzed",
                    "type": "string"
                  }
                }
              }
            }
          }
        ],
        "properties": {
          "access_count": {
            "type": "long"
          },
          "account_rid": {
            "type": "long"
          },
          "application_focus_count": {
            "type": "long"
          },
          "application_focus_duration": {
            "type": "long"
          },
          "attribute_type": {
            "type": "long"
          },
          "birth_droid_file_identifier": {
            "type": "text",
            "fields": {
              "raw": {
                "type": "keyword",
                "ignore_above": 10922
              }
            }
          },
          "birth_droid_volume_identifier": {
            "type": "text",
            "fields": {
              "raw": {
                "type": "keyword",
                "ignore_above": 10922
              }
            }
          },
          "cache_directory_index": {
            "type": "long"
          },
          "cache_identifier": {
            "type": "long"
          },
          "cached_file_size": {
            "type": "long"
          },
          "cached_filename": {
            "type": "text",
            "fields": {
              "raw": {
                "type": "keyword",
                "ignore_above": 10922
              }
            }
          },
          "command_line_arguments": {
            "type": "text",
            "fields": {
              "raw": {
                "type": "keyword",
                "ignore_above": 10922
              }
            }
          },
          "comments": {
            "type": "text",
            "fields": {
              "raw": {
                "type": "keyword",
                "ignore_above": 10922
              }
            }
          },
          "companyname": {
            "type": "text",
            "fields": {
              "raw": {
                "type": "keyword",
                "ignore_above": 10922
              }
            }
          },
          "computer_name": {
            "type": "text",
            "fields": {
              "raw": {
                "type": "keyword",
                "ignore_above": 10922
              }
            }
          },
          "container_identifier": {
            "type": "long"
          },
          "createdts": {
            "type": "long"
          },
          "data_type": {
            "type": "text",
            "fields": {
              "raw": {
                "type": "keyword",
                "ignore_above": 10922
              }
            }
          },
          "datetime": {
            "type": "date"
          },
          "description": {
            "type": "text",
            "fields": {
              "raw": {
                "type": "keyword",
                "ignore_above": 10922
              }
            }
          },
          "display_name": {
            "type": "text",
            "fields": {
              "raw": {
                "type": "keyword",
                "ignore_above": 10922
              }
            }
          },
          "dll_name": {
            "type": "text",
            "fields": {
              "raw": {
                "type": "keyword",
                "ignore_above": 10922
              }
            }
          },
          "drive_serial_number": {
            "type": "long"
          },
          "drive_type": {
            "type": "long"
          },
          "droid_file_identifier": {
            "type": "text",
            "fields": {
              "raw": {
                "type": "keyword",
                "ignore_above": 10922
              }
            }
          },
          "droid_volume_identifier": {
            "type": "text",
            "fields": {
              "raw": {
                "type": "keyword",
                "ignore_above": 10922
              }
            }
          },
          "entry_identifier": {
            "type": "long"
          },
          "entry_index": {
            "type": "long"
          },
          "entrytype": {
            "type": "text",
            "fields": {
              "raw": {
                "type": "keyword",
                "ignore_above": 10922
              }
            }
          },
          "env_var_location": {
            "type": "text",
            "fields": {
              "raw": {
                "type": "keyword",
                "ignore_above": 10922
              }
            }
          },
          "event_identifier": {
            "type": "long"
          },
          "event_level": {
            "type": "long"
          },
          "file_attribute_flags": {
            "type": "long"
          },
          "file_entry_type": {
            "type": "long"
          },
          "file_reference": {
            "type": "text"
          },
          "file_size": {
            "type": "long"
          },
          "file_system_type": {
            "type": "text",
            "fields": {
              "raw": {
                "type": "keyword",
                "ignore_above": 10922
              }
            }
          },
          "filedescription": {
            "type": "text",
            "fields": {
              "raw": {
                "type": "keyword",
                "ignore_above": 10922
              }
            }
          },
          "filename": {
            "type": "text",
            "fields": {
              "raw": {
                "type": "keyword",
                "ignore_above": 10922
              }
            }
          },
          "filepaths": {
            "type": "text",
            "fields": {
              "raw": {
                "type": "keyword",
                "ignore_above": 10922
              }
            }
          },
          "files": {
            "type": "text",
            "fields": {
              "raw": {
                "type": "keyword",
                "ignore_above": 10922
              }
            }
          },
          "filesize": {
            "type": "long"
          },
          "fileversion": {
            "type": "text",
            "fields": {
              "raw": {
                "type": "keyword",
                "ignore_above": 10922
              }
            }
          },
          "full_path": {
            "type": "text",
            "fields": {
              "raw": {
                "type": "keyword",
                "ignore_above": 10922
              }
            }
          },
          "fullname": {
            "type": "text",
            "fields": {
              "raw": {
                "type": "keyword",
                "ignore_above": 10922
              }
            }
          },
          "hostname": {
            "type": "text",
            "fields": {
              "raw": {
                "type": "keyword",
                "ignore_above": 10922
              }
            }
          },
          "http_headers": {
            "type": "text",
            "fields": {
              "raw": {
                "type": "keyword",
                "ignore_above": 10922
              }
            }
          },
          "icon_location": {
            "type": "text",
            "fields": {
              "raw": {
                "type": "keyword",
                "ignore_above": 10922
              }
            }
          },
          "idx_version": {
            "type": "long"
          },
          "imphash": {
            "type": "text",
            "fields": {
              "raw": {
                "type": "keyword",
                "ignore_above": 10922
              }
            }
          },
          "inode": {
            "type": "long"
          },
          "ip_address": {
            "type": "text",
            "fields": {
              "raw": {
                "type": "keyword",
                "ignore_above": 10922
              }
            }
          },
          "is_allocated": {
            "type": "boolean"
          },
          "key_path": {
            "type": "text",
            "fields": {
              "raw": {
                "type": "keyword",
                "ignore_above": 10922
              }
            }
          },
          "languagecode": {
            "type": "long"
          },
          "lastmodifiedts": {
            "type": "long"
          },
          "link_target": {
            "type": "text",
            "fields": {
              "raw": {
                "type": "keyword",
                "ignore_above": 10922
              }
            }
          },
          "linkerts": {
            "type": "long"
          },
          "list_name": {
            "type": "text",
            "fields": {
              "raw": {
                "type": "keyword",
                "ignore_above": 10922
              }
            }
          },
          "list_values": {
            "type": "text",
            "fields": {
              "raw": {
                "type": "keyword",
                "ignore_above": 10922
              }
            }
          },
          "local_path": {
            "type": "text",
            "fields": {
              "raw": {
                "type": "keyword",
                "ignore_above": 10922
              }
            }
          },
          "login_count": {
            "type": "long"
          },
          "long_name": {
            "type": "text",
            "fields": {
              "raw": {
                "type": "keyword",
                "ignore_above": 10922
              }
            }
          },
          "mac_address": {
            "type": "text",
            "fields": {
              "raw": {
                "type": "keyword",
                "ignore_above": 10922
              }
            }
          },
          "md5_hash": {
            "type": "text",
            "fields": {
              "raw": {
                "type": "keyword",
                "ignore_above": 10922
              }
            }
          },
          "message": {
            "type": "text",
            "fields": {
              "raw": {
                "type": "keyword",
                "ignore_above": 10922
              }
            }
          },
          "message_identifier": {
            "type": "long"
          },
          "msipackagecode": {
            "type": "text",
            "fields": {
              "raw": {
                "type": "keyword",
                "ignore_above": 10922
              }
            }
          },
          "msiproductcode": {
            "type": "text",
            "fields": {
              "raw": {
                "type": "keyword",
                "ignore_above": 10922
              }
            }
          },
          "name": {
            "type": "text",
            "fields": {
              "raw": {
                "type": "keyword",
                "ignore_above": 10922
              }
            }
          },
          "network_path": {
            "type": "text",
            "fields": {
              "raw": {
                "type": "keyword",
                "ignore_above": 10922
              }
            }
          },
          "number_of_executions": {
            "type": "long"
          },
          "number_of_hits": {
            "type": "long"
          },
          "offset": {
            "type": "long"
          },
          "origin": {
            "type": "text",
            "fields": {
              "raw": {
                "type": "keyword",
                "ignore_above": 10922
              }
            }
          },
          "original_filename": {
            "type": "text",
            "fields": {
              "raw": {
                "type": "keyword",
                "ignore_above": 10922
              }
            }
          },
          "owner": {
            "type": "text",
            "fields": {
              "raw": {
                "type": "keyword",
                "ignore_above": 10922
              }
            }
          },
          "packagecode": {
            "type": "text",
            "fields": {
              "raw": {
                "type": "keyword",
                "ignore_above": 10922
              }
            }
          },
          "parent_file_reference": {
            "type": "long"
          },
          "parser": {
            "type": "text",
            "fields": {
              "raw": {
                "type": "keyword",
                "ignore_above": 10922
              }
            }
          },
          "parser_chain": {
            "type": "text",
            "fields": {
              "raw": {
                "type": "keyword",
                "ignore_above": 10922
              }
            }
          },
          "path": {
            "type": "text",
            "fields": {
              "raw": {
                "type": "keyword",
                "ignore_above": 10922
              }
            }
          },
          "pathspec": {
            "type": "text",
            "fields": {
              "raw": {
                "type": "keyword",
                "ignore_above": 10922
              }
            }
          },
          "pe_type": {
            "type": "text",
            "fields": {
              "raw": {
                "type": "keyword",
                "ignore_above": 10922
              }
            }
          },
          "product_name": {
            "type": "text",
            "fields": {
              "raw": {
                "type": "keyword",
                "ignore_above": 10922
              }
            }
          },
          "productcode": {
            "type": "text",
            "fields": {
              "raw": {
                "type": "keyword",
                "ignore_above": 10922
              }
            }
          },
          "productname": {
            "type": "text",
            "fields": {
              "raw": {
                "type": "keyword",
                "ignore_above": 10922
              }
            }
          },
          "programid": {
            "type": "text",
            "fields": {
              "raw": {
                "type": "keyword",
                "ignore_above": 10922
              }
            }
          },
          "publisher": {
            "type": "text",
            "fields": {
              "raw": {
                "type": "keyword",
                "ignore_above": 10922
              }
            }
          },
          "record_number": {
            "type": "long"
          },
          "recovered": {
            "type": "boolean"
          },
          "relative_path": {
            "type": "text",
            "fields": {
              "raw": {
                "type": "keyword",
                "ignore_above": 10922
              }
            }
          },
          "section_names": {
            "type": "text",
            "fields": {
              "raw": {
                "type": "keyword",
                "ignore_above": 10922
              }
            }
          },
          "service_pack": {
            "type": "text",
            "fields": {
              "raw": {
                "type": "keyword",
                "ignore_above": 10922
              }
            }
          },
          "sha1": {
            "type": "text",
            "fields": {
              "raw": {
                "type": "keyword",
                "ignore_above": 10922
              }
            }
          },
          "sha1_hash": {
            "type": "text",
            "fields": {
              "raw": {
                "type": "keyword",
                "ignore_above": 10922
              }
            }
          },
          "sha256_hash": {
            "type": "text",
            "fields": {
              "raw": {
                "type": "keyword",
                "ignore_above": 10922
              }
            }
          },
          "shell_item_path": {
            "type": "text",
            "fields": {
              "raw": {
                "type": "keyword",
                "ignore_above": 10922
              }
            }
          },
          "source_append": {
            "type": "text",
            "fields": {
              "raw": {
                "type": "keyword",
                "ignore_above": 10922
              }
            }
          },
          "source_long": {
            "type": "text",
            "fields": {
              "raw": {
                "type": "keyword",
                "ignore_above": 10922
              }
            }
          },
          "source_name": {
            "type": "text",
            "fields": {
              "raw": {
                "type": "keyword",
                "ignore_above": 10922
              }
            }
          },
          "source_short": {
            "type": "text",
            "fields": {
              "raw": {
                "type": "keyword",
                "ignore_above": 10922
              }
            }
          },
          "strings": {
            "type": "text",
            "fields": {
              "raw": {
                "type": "keyword",
                "ignore_above": 10922
              }
            }
          },
          "strings_parsed": {
            "properties": {
              "source_user_id": {
                "type": "text",
                "fields": {
                  "raw": {
                    "type": "keyword",
                    "ignore_above": 10922
                  }
                }
              },
              "source_user_name": {
                "type": "text",
                "fields": {
                  "raw": {
                    "type": "keyword",
                    "ignore_above": 10922
                  }
                }
              },
              "target_machine_ip": {
                "type": "text",
                "fields": {
                  "raw": {
                    "type": "keyword",
                    "ignore_above": 10922
                  }
                }
              },
              "target_machine_name": {
                "type": "text",
                "fields": {
                  "raw": {
                    "type": "keyword",
                    "ignore_above": 10922
                  }
                }
              },
              "target_user_id": {
                "type": "text",
                "fields": {
                  "raw": {
                    "type": "keyword",
                    "ignore_above": 10922
                  }
                }
              },
              "target_user_name": {
                "type": "text",
                "fields": {
                  "raw": {
                    "type": "keyword",
                    "ignore_above": 10922
                  }
                }
              }
            }
          },
          "sync_count": {
            "type": "long"
          },
          "task_identifier": {
            "type": "text",
            "fields": {
              "raw": {
                "type": "keyword",
                "ignore_above": 10922
              }
            }
          },
          "task_name": {
            "type": "text",
            "fields": {
              "raw": {
                "type": "keyword",
                "ignore_above": 10922
              }
            }
          },
          "timestamp": {
            "type": "long"
          },
          "timestamp_desc": {
            "type": "text",
            "fields": {
              "raw": {
                "type": "keyword",
                "ignore_above": 10922
              }
            }
          },
          "uninstallkey": {
            "type": "text",
            "fields": {
              "raw": {
                "type": "keyword",
                "ignore_above": 10922
              }
            }
          },
          "url": {
            "type": "text",
            "fields": {
              "raw": {
                "type": "keyword",
                "ignore_above": 10922
              }
            }
          },
          "urls": {
            "type": "text",
            "fields": {
              "raw": {
                "type": "keyword",
                "ignore_above": 10922
              }
            }
          },
          "user_sid": {
            "type": "text",
            "fields": {
              "raw": {
                "type": "keyword",
                "ignore_above": 10922
              }
            }
          },
          "username": {
            "type": "text",
            "fields": {
              "raw": {
                "type": "keyword",
                "ignore_above": 10922
              }
            }
          },
          "uuid": {
            "type": "text",
            "fields": {
              "raw": {
                "type": "keyword",
                "ignore_above": 10922
              }
            }
          },
          "value_name": {
            "type": "text",
            "fields": {
              "raw": {
                "type": "keyword",
                "ignore_above": 10922
              }
            }
          },
          "version": {
            "type": "text",
            "fields": {
              "raw": {
                "type": "keyword",
                "ignore_above": 10922
              }
            }
          },
          "volume_label": {
            "type": "text",
            "fields": {
              "raw": {
                "type": "keyword",
                "ignore_above": 10922
              }
            }
          },
          "working_directory": {
            "type": "text",
            "fields": {
              "raw": {
                "type": "keyword",
                "ignore_above": 10922
              }
            }
          },
          "xml_string": {
            "type": "text",
            "fields": {
              "raw": {
                "type": "keyword",
                "ignore_above": 10922
              }
            }
          }
        }
      }
    }```
joachimmetz commented 6 years ago

blocked by #1828

joachimmetz commented 6 years ago

@Onager @berggren does this issue require further follow up?

berggren commented 6 years ago

There is a larger question here: Do we want to maintain an explicit mapping for all data_types in plaso instead of relying on ES dynamic types? The root problem here is that the parser is not consistent with the output type. Sometimes it is a string and sometimes it is an integer.

I opt for fixing the parser and have ES deal with the mapping.

joachimmetz commented 6 years ago

Per conversation with @berggren we could consider adding an option to load a custom mappings file (JSON)

joachimmetz commented 6 years ago

Added https://github.com/log2timeline/plaso/issues/1879, closing this issue

joachimmetz commented 5 years ago

@imifos a couple of things.

Can you add an open source license https://choosealicense.com/, https://choosealicense.com/no-permission/. Otherwise I'll remove your comment considering it SPAM (promotion of your own solution on a closed issue) and flag it with github as abuse.

Also why did you do your own thing and not contributed to the project?

imifos commented 5 years ago
@joachimmetz License: Done Own Thing: I would love to contribute to the project and I have my nose in the code quite often. In this case, I "did my own thing" because I was not able to set-up a build environment for the log2timeline project in a reasonable (constrained) time frame. There are delicate dependencies and I was not able to make it all work (yet). Moreover, you have already decided on a solution to the above problem, which would be the way to go for a serious contribution. Being on a case and this issue blocking me, I decided to do a quick-and-dirty hack, the time your solution will be implemented. My "thing" is certainly nothing to integrate into your project. Please note that this is not a reproach. I'm aware that this is a free-time project for you all and do this as best-effort. I, and all the community, am super thankful for your work and this tool! I'll certainly continue on the build environment and once I figure everything out, I would be happy to be able to share this as project documentation.
joachimmetz commented 5 years ago

Unfortunately public domain is not a FOSS compatible "license", also see: https://opensource.org/node/878 and we are not able to use any of this in plaso.

The problem is one offs are fragile, and people will contact this project once the one off is not working. I also really would like to see the DF field in general build less works-for-me solutions and help building works-for-more solutions.

imifos commented 5 years ago

I was not aware of the PD/FOSS issue. Thanks.

I removed the link to avoid people coming back to you about this. I'll also add a clear comment in the script on my side.

joachimmetz commented 5 years ago

Thx