Azure / azure-diagnostics-tools

Plugins and tools for collecting, processing, managing, and visualizing diagnostics data and configuration
98 stars 93 forks source link

Cannot start example #216

Open maczikasz opened 4 years ago

maczikasz commented 4 years ago

Trying to run a simple example to connect to AZ blob storage:

FROM docker.elastic.co/logstash/logstash:7.6.1                                                                                         

RUN logstash-plugin install logstash-input-azureblob                                                                                   

ADD azblob.yml /usr/share/logstash/pipeline/
input
{
    azureblob
    {
        storage_account_name => "mystorageaccount"
        storage_access_key => "VGhpcyBpcyBhIGZha2Uga2V5Lg=="
        container => "mycontainer"
    }
}
[2020-03-24T14:00:02,965][INFO ][logstash.setting.writabledirectory] Creating directory {:setting=>"path.queue", :path=>"/usr/share/logstash/data/queue"}
[2020-03-24T14:00:02,987][INFO ][logstash.setting.writabledirectory] Creating directory {:setting=>"path.dead_letter_queue", :path=>"/usr/share/logstash/data/dead_letter_queue"}
[2020-03-24T14:00:03,391][INFO ][logstash.runner          ] Starting Logstash {"logstash.version"=>"7.6.1"}
[2020-03-24T14:00:03,415][INFO ][logstash.agent           ] No persistent UUID file found. Generating new UUID {:uuid=>"b8a1dd93-0cd9-4cc1-bf31-d63cd21af99e", :path=>"/usr/share/logstash/data/uuid"}
[2020-03-24T14:00:03,901][WARN ][logstash.monitoringextension.pipelineregisterhook] xpack.monitoring.enabled has not been defined, but found elasticsearch configuration. Please explicitly set `xpack.monitoring.enabled: true` in logstash.yml
[2020-03-24T14:00:04,756][INFO ][logstash.licensechecker.licensereader] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[http://elasticsearch:9200/]}}
[2020-03-24T14:00:05,061][WARN ][logstash.licensechecker.licensereader] Attempted to resurrect connection to dead ES instance, but got an error. {:url=>"http://elasticsearch:9200/", :error_type=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError, :error=>"Elasticsearch Unreachable: [http://elasticsearch:9200/][Manticore::ResolutionFailure] elasticsearch: Name or service not known"}
[2020-03-24T14:00:05,106][WARN ][logstash.licensechecker.licensereader] Marking url as dead. Last error: [LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError] Elasticsearch Unreachable: [http://elasticsearch:9200/][Manticore::ResolutionFailure] elasticsearch {:url=>http://elasticsearch:9200/, :error_message=>"Elasticsearch Unreachable: [http://elasticsearch:9200/][Manticore::ResolutionFailure] elasticsearch", :error_class=>"LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError"}
[2020-03-24T14:00:05,117][ERROR][logstash.licensechecker.licensereader] Unable to retrieve license information from license server {:message=>"Elasticsearch Unreachable: [http://elasticsearch:9200/][Manticore::ResolutionFailure] elasticsearch"}
[2020-03-24T14:00:05,171][ERROR][logstash.monitoring.internalpipelinesource] Failed to fetch X-Pack information from Elasticsearch. This is likely due to failure to reach a live Elasticsearch cluster.
[2020-03-24T14:00:06,529][INFO ][org.reflections.Reflections] Reflections took 57 ms to scan 1 urls, producing 20 keys and 40 values 
[2020-03-24T14:00:07,185][INFO ][logstash.inputs.logstashinputazureblob] Using version 0.9.x input plugin 'azureblob'. This plugin should work but would benefit from use by folks like you. Please let us know if you find bugs or have suggestions on how to improve this plugin.
[2020-03-24T14:00:07,596][WARN ][org.logstash.instrument.metrics.gauge.LazyDelegatingGauge][main] A gauge metric of an unknown type (org.jruby.RubyArray) has been create for key: cluster_uuids. This may result in invalid serialization.  It is recommended to log an issue to the responsible developer/development team.
[2020-03-24T14:00:07,605][INFO ][logstash.javapipeline    ][main] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>8, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>1000, "pipeline.sources"=>["/usr/share/logstash/pipeline/azblog.yml", "/usr/share/logstash/pipeline/logstash.conf"], :thread=>"#<Thread:0x5ed50d39 run>"}
[2020-03-24T14:00:08,585][INFO ][logstash.inputs.beats    ][main] Beats inputs: Starting input listener {:address=>"0.0.0.0:5044"}
[2020-03-24T14:00:09,719][ERROR][logstash.javapipeline    ][main] Pipeline aborted due to error {:pipeline_id=>"main", :exception=>#<NoMethodError: undefined method `version' for nil:NilClass>, :backtrace=>["/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-azureblob-0.9.13-java/lib/logstash/inputs/azureblob.rb:135:in `register'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:200:in `block in register_plugins'", "org/jruby/RubyArray.java:1814:in `each'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:199:in `register_plugins'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:310:in `start_inputs'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:270:in `start_workers'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:154:in `run'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:109:in `block in start'"], "pipeline.sources"=>["/usr/share/logstash/pipeline/azblog.yml", "/usr/share/logstash/pipeline/logstash.conf"], :thread=>"#<Thread:0x5ed50d39 run>"}
[2020-03-24T14:00:09,741][ERROR][logstash.agent           ] Failed to execute action {:id=>:main, :action_type=>LogStash::ConvergeResult::FailedAction, :message=>"Could not execute action: PipelineAction::Create<main>, action_result: false", :backtrace=>nil}
[2020-03-24T14:00:10,014][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600}
[2020-03-24T14:00:14,897][INFO ][logstash.runner          ] Logstash shut down.
pinochioze commented 4 years ago

Hi Maczikasz, I am glad to support you, can you let me know what do you try to do this example? I am not sure what are you doing and your steps