treasure-data / chef-td-agent

Chef Cookbook for td-agent (Treasure Agent or Fluentd)
https://supermarket.chef.io/cookbooks/td-agent
Apache License 2.0
127 stars 120 forks source link

packages.teasuredata.com s3 bucket 404 #156

Open couture-ql opened 2 years ago

couture-ql commented 2 years ago

Hi there, while not specifically a chef-td-agent problem, anybody using this chef recipe will run into issues with the update or install action, because the http://packages.treasuredata.com/2/ubuntu/bionic 404s (or rather has issues -- see chef output below). I understand the s3 bucket won't let people browse the directory.

It appears the hosting for these packages has updated to somewhere else:

For example, Ubuntu users can find repos hosted here instead: https://td-agent-package-browser.herokuapp.com/2/ubuntu/ https://td-agent-package-browser.herokuapp.com/4/ubuntu/

While not the responsibility of this package, it would be great if the domain names resolved to the correct hosting location.

Otherwise, I suspect a change would be needed to update recipes/install.rb URI paths.

Here's my chef output of the relevant lines failing during update action:

[2022-02-17T17:36:07+00:00] INFO: Processing execute[apt-cache gencaches] action nothing (/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.18.31/lib/chef/provider/apt_repository.rb line 54)
[2022-02-17T17:36:07+00:00] INFO: Processing apt_update[treasure-data] action nothing (/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.18.31/lib/chef/provider/apt_repository.rb line 59)
[2022-02-17T17:36:07+00:00] INFO: Processing file[/etc/apt/sources.list.d/treasure-data.list] action create (/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.18.31/lib/chef/provider/apt_repository.rb line 79)
[2022-02-17T17:36:07+00:00] INFO: Processing directory[/etc/td-agent/conf.d] action create (td-agent::install line 121)
[2022-02-17T17:36:07+00:00] INFO: Processing apt_package[td-agent] action upgrade (td-agent::install line 128)
[2022-02-17T17:36:08+00:00] INFO: Retrying execution of apt_package[td-agent], 2 attempt(s) left
[2022-02-17T17:36:18+00:00] INFO: Retrying execution of apt_package[td-agent], 1 attempt(s) left
[2022-02-17T17:36:28+00:00] INFO: Retrying execution of apt_package[td-agent], 0 attempt(s) left

================================================================================
Error executing action `upgrade` on resource 'apt_package[td-agent]'
================================================================================

Chef::Exceptions::Package
-------------------------
No candidate version available for td-agent

While I can set my attributes to use a newer td-agent, my immediate need is that we can still install td-agent 2

default['td_agent']['version'] = '2'

I run into different problems to solve if jumping to td-agent 4.


I'm technically identifying two problems:

  1. td-agent 2 has install / update issues
  2. trying to browse the packages publicly isn't possible, but is under a new domain.
ynuyasha commented 2 years ago

if you browse repository https://td-agent-package-browser.herokuapp.com/2/ubuntu/ you can find that bionic is not present, you need to use the 2.5 repo https://td-agent-package-browser.herokuapp.com/2.5/ubuntu/ that have bionic