index_set - rotation_strategy - max_size property does not exist

milosz commented 4 years ago

Environment

Terraform version: Terraform v0.13.0
Graylog version: Graylog 3.3.4+279ef2c on 99c4d56e8a80 (Oracle Corporation 1.8.0_265 on Linux 5.4.0-42-generic)
terraform-provider-graylog or go-graylog version: terraform-provider-graylog_1.0.4_linux_amd64.zip (latest)
OS (client side): Ubuntu 20.04

Overview

Example https://github.com/suzuki-shunsuke/go-graylog/blob/master/examples/v0.11/index_set.tf uses max_size property in rotation_strategy, but as of now it is not defined in web interface or api browser.

How to reproduce

For terraform code (same as above example, only different name):

resource "graylog_index_set" "docker" {                                                                                                                                                                                                                                             
  title                               = "docker"                                                                                                                                                                                                                                    
  index_prefix                        = "docker"                                                                                                                                                                                                                                    
  rotation_strategy_class             = "org.graylog2.indexer.rotation.strategies.MessageCountRotationStrategy"                                                                                                                                                                     
  retention_strategy_class            = "org.graylog2.indexer.retention.strategies.DeletionRetentionStrategy"                                                                                                                                                                       
  description                         = "test 2"                                                                                                                                                                                                                                    
  index_analyzer                      = "standard"                                                                                                                                                                                                                                  
  index_optimization_disabled         = true                                                                                                                                                                                                                                        
  writable                            = true                                                                                                                                                                                                                                        
  shards                              = 1                                                                                                                                                                                                                                           
  replicas                            = 0                                                                                                                                                                                                                                           
  index_optimization_max_num_segments = 1                                                                                                                                                                                                                                           
  field_type_refresh_interval         = 5000                                                                                                                                                                                                                                        

  retention_strategy = jsonencode({                                                                                                                                                                                                                                                 
    max_number_of_indices = 14                                                                                                                                                                                                                                                      
    type                  = "org.graylog2.indexer.retention.strategies.DeletionRetentionStrategyConfig"                                                                                                                                                                             
  })                                                                                                                                                                                                                                                                                

  rotation_strategy = jsonencode({                                                                                                                                                                                                                                                  
    max_docs_per_index = 20000000                                                                                                                                                                                                                                                   
    max_size           = 0                                                                                                                                                                                                                                                          
    type               = "org.graylog2.indexer.rotation.strategies.MessageCountRotationStrategyConfig"                                                                                                                                                                              
  })                                                                                                                                                                                                                                                                                
}

Terraform returns:

Error: failed to update a index set 5f42d726d7510e67ed9b184b: status code: 400, {"type":"ApiError","message":"Unable to map property max_size.\nKnown properties include: type, max_docs_per_index"}: status code >= 300

  on graylog.tf line 24, in resource "graylog_index_set" "docker":
  24: resource "graylog_index_set" "docker" {

Removing "max_size = 0" (rotation_strategy) property solves problem.

JSON from api does not contain this field.

    {
      "id": "5f3d8ca9128eb63543e8a6e8",
      "title": "Default index set",
      "description": "The Graylog default index set",
      "index_prefix": "graylog",
      "shards": 1,
      "replicas": 0,
      "rotation_strategy_class": "org.graylog2.indexer.rotation.strategies.MessageCountRotationStrategy",
      "rotation_strategy": {
        "type": "org.graylog2.indexer.rotation.strategies.MessageCountRotationStrategyConfig",
        "max_docs_per_index": 20000000
      },
      "retention_strategy_class": "org.graylog2.indexer.retention.strategies.DeletionRetentionStrategy",
      "retention_strategy": {
        "type": "org.graylog2.indexer.retention.strategies.DeletionRetentionStrategyConfig",
        "max_number_of_indices": 5
      },
      "creation_date": "2020-08-19T20:33:45.825Z",
      "index_analyzer": "standard",
      "index_optimization_max_num_segments": 1,
      "index_optimization_disabled": false,
      "field_type_refresh_interval": 5000,
      "writable": true,
      "default": true
    },

milosz commented 4 years ago

max_size property is also referenced in the https://github.com/suzuki-shunsuke/go-graylog/blob/master/docs/resources/index_set.md

suzuki-shunsuke commented 4 years ago

Thank you for your information. I guess this is a change of the Graylog API. I don't confirm yet but I think the older Graylog supports the max_size. I want to know which version the change is added.

milosz commented 4 years ago

There are likely more changes in Graylog API. I do not know about earlier Graylog API versions, but it looks like there are some similarities between dashboards and views which now makes the resource:

resource "graylog_dashboard" "log_sources" {                                                                                             
  title       = "sources"                                                                                                                
  description = "sources"                                                                                                                
}

plan:

Terraform will perform the following actions:

  # graylog_dashboard.log_sources will be created
  + resource "graylog_dashboard" "log_sources" {
      + created_at  = (known after apply)
      + description = "sources"
      + id          = (known after apply)
      + title       = "sources"
    }

Plan: 1 to add, 0 to change, 0 to destroy.

fail in execution:

Error: failed to create a dashboard: status code: 405, {"type":"ApiError","message":"HTTP 405 Method Not Allowed"}: status code >= 300

  on graylog.tf line 137, in resource "graylog_dashboard" "log_sources":
 137: resource "graylog_dashboard" "log_sources" {

Useful reading about Dashboard API changes:

suzuki-shunsuke / go-graylog