Fix the limitation of the size of yaml file that exceeds 3MB

[kaisecheng]

Fixed: #96

added setting yaml_dictionary_code_point_limit to config the maximum code point limit of the yaml file in dictionary_path to overcome the 3MB size limit from SnakeYaml 1.33. This setting is only effective for yaml. Yaml file over the size limit throws an exception. JSON and CSV currently do not have such restriction. The default value of yaml_dictionary_code_point_limit is 128MB.

How to test

• checkout the branch
• build the gem
  • bundle install
  • gem build logstash-filter-translate.gemspec
• install the gem in Logstash
  • bin/logstash-plugin install --local ~/path/to/your/gem/logstash-filter-translate-3.4.1.gem
• download a 4MB yaml from here
• run the following config

  input {
  file {
  path => "/path/to/yaml/big_yaml_4mb.yml"
  start_position => "beginning"
  }
  }
  filter {
  dissect {
  mapping => {
    "message" => "%{key}: %{val}"
  }
  }
  translate {
  source => "key"
  target => "translated"
  fallback => "UNKNOWN"
  dictionary_path => "/path/to/yaml/big_yaml_4mb.yml"
  yaml_dictionary_code_point_limit => 5000000
  }
  }
  output {
  stdout { codec => dots }
  }

• Logstash should start without error

[mashhurs]

Changes overall LGTM by agreeing @jsvd 's comments. One thing I would like to clarify here, Psych moved from snakeyaml to snakeyaml-engine this year. The reason behind as I understood to mitigate the CVE risks and performances (snakeyaml-engine is more suitable in JVM env). This change also indicates that under the hood we are moving to snakeyaml-engine AND for the long term we should/may move our snakeyaml dependencies to snakeyaml-engine as well.

Reference thread: link