search

elastic / crawler

Other
125 stars 10 forks source link

Output sink type `file` is broken #138

Closed navarone-feekery closed 2 months ago

navarone-feekery commented 2 months ago

Bug Description

Output sink type file is broken in 0.2 release.

Unexpected error while executing a crawl task <CrawlTask: url=https://www.example.com/, type=content, depth=1, redirect_count=0, auth=none>: /Users/nav/Workspace/elastic-crawler/lib/crawler/output_sink/file.rb:28:in `write': undefined method `write' for Crawler::OutputSink::File:Class (NoMethodError)
        from /Users/nav/Workspace/elastic-crawler/lib/crawler/coordinator.rb:522:in `output_crawl_result'
        from /Users/nav/Workspace/elastic-crawler/lib/crawler/coordinator.rb:392:in `process_crawl_result'
        from /Users/nav/Workspace/elastic-crawler/lib/crawler/coordinator.rb:340:in `execute_crawl_task'
        from /Users/nav/Workspace/elastic-crawler/lib/crawler/coordinator.rb:325:in `block in prepare_crawl_task'
        from /Users/nav/.rbenv/versions/jruby-9.4.7.0/lib/ruby/gems/shared/gems/concurrent-ruby-1.1.10/lib/concurrent-ruby/concurrent/executor/java_executor_service.rb:79:in `run'
navarone-feekery commented 2 months ago

Fix is out, will be included in 0.2.1

In the meantime you can get this fix by building from main branch, or pulling the 0.2.1-SNAPSHOT docker image:

$ docker pull docker.elastic.co/integrations/crawler:0.2.1-SNAPSHOT