Closed radu-gheorghe closed 8 years ago
@radu-gheorghe RPM and deb of 5.0~alpha1 should be available now.
Sorry for the hiccups.
suyog@ubuntu:~$ sudo apt-cache policy logstash
logstash:
Installed: 1:2.2.2-1
Candidate: 1:5.0.0~alpha1-1
Version table:
1:5.0.0~alpha1-1 0
500 http://packages.elastic.co/logstash/5.0/debian/ stable/main i386 Packages
1:2.2.4-1 0
500 http://packages.elastic.co/logstash/2.2/debian/ stable/main i386 Packages
1:2.2.3-1 0
500 http://packages.elastic.co/logstash/2.2/debian/ stable/main i386 Packages
1:2.2.2-1 0
500 http://packages.elastic.co/logstash/2.2/debian/ stable/main i386 Packages
Thanks @ph! apt-get update
works now but installation fails, saying:
E: Failed to fetch http://packages.elastic.co/logstash/5.0/debian/pool/main/l/logstash/logstash_5.0.0~alpha1-1_all.deb 403 Forbidden [IP: 107.22.242.161 80]
Which is strange because wget from that URL works. On the other hand, installing another package works, too (specifically, I tried with htop and topbeat), so it might be something specific to Logstash's repo.
I am able to reproduce the issue this is really weird.
After this operation, 122 MB of additional disk space will be used.
Err http://packages.elastic.co/logstash/5.0/debian/ stable/main logstash all 1:5.0.0~alpha1-1
403 Forbidden [IP: 107.22.242.161 80]
E: Failed to fetch http://packages.elastic.co/logstash/5.0/debian/pool/main/l/logstash/logstash_5.0.0~alpha1-1_all.deb 403 Forbidden [IP: 107.22.242.161 80]
E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?
running with strace gives us a bit more details. (on ubuntu 14.04)
(24, "601 Configuration\nConfig-Item: APT::Architecture=i386\nConfig-Item: APT::Build-Essential::=build-essential\nConfig-Item: APT::Install-Recommends=1\nConfig-Item: APT::Install-Suggests=0\nConfig-Item: APT::"..., 6787) = 6787
gettimeofday({1460646586, 547520}, NULL) = 0
rt_sigprocmask(SIG_BLOCK, [WINCH], [], 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
write(1, "\r\r0% [Working]", 14) = 14
select(22, [21], [], NULL, {0, 500000}) = 1 (in [21], left {0, 499622})
read(21, "102 Status\nURI: http://packages.elastic.co/logstash/5.0/debian/pool/main/l/logstash/logstash_5.0.0~alpha1-1_all.deb\nMessage: Connecting to packages.elastic.co\n\n", 64000) = 160
select(22, [21], [], NULL, {0, 499622}) = 1 (in [21], left {0, 432033})
read(21, "102 Status\nURI: http://packages.elastic.co/logstash/5.0/debian/pool/main/l/logstash/logstash_5.0.0~alpha1-1_all.deb\nMessage: Connecting to packages.elastic.co (107.22.238.179)\n\n", 64000) = 177
select(22, [21], [], NULL, {0, 432033}) = 1 (in [21], left {0, 404477})
read(21, "102 Status\nURI: http://packages.elastic.co/logstash/5.0/debian/pool/main/l/logstash/logstash_5.0.0~alpha1-1_all.deb\nMessage: Waiting for headers\n\n", 64000) = 146
select(22, [21], [], NULL, {0, 404477}) = 1 (in [21], left {0, 355343})
read(21, "400 URI Failure\nURI: http://packages.elastic.co/logstash/5.0/debian/pool/main/l/logstash/logstash_5.0.0~alpha1-1_all.deb\nMessage: 403 Forbidden [IP: 107.22.238.179 80]\nFailReason: HttpError403\n\n", 64000) = 195
write(1, "\r \rErr http://packages.elastic.co/logstash/5.0/debian/ stable/main logstash all 1:5.0.0~alpha1-1\n", 108) = 108
write(1, " 403 Forbidden [IP: 107.22.238.179 80]\n", 41) = 41
gettimeofday({1460646586, 693663}, NULL) = 0
full strace https://gist.github.com/ph/f848fb47dcb838ce5a6b89c5036668cc
@jordansissel I am not sure what is going here, any idea or clue about what is wrong?
apt-get is fetching this file: /logstash/5.0/debian/pool/main/l/logstash/logstash_5.0.0%7ealpha1-1_all.deb
It's turning the tilde ~
into %7e
and the webserver is returning 403.
This is a bug with S3's webservers, iirc. One hack is that we can symlink (maybe) any files with ~ in the name to the path with %7e replacing it. Not sure though.
Let note that the tilde isnt in the S3 safe list of characters for keys. Safe Characters
The following character sets are generally safe for use in key names:
Alphanumeric characters [0-9a-zA-Z]
Special characters !, -, _, ., *, ', (, and )
source http://docs.aws.amazon.com/AmazonS3/latest/dev/UsingMetadata.html
uugh. I so baddly want to stop using S3 for package storage.
Renaming the file changing the tilde ~
to a dash -
(but not changing the package metadata) seems to work.
% apt-get update
% apt-get install logstas
...
Get:1 http://packages.elastic.co/logstash/5.0/debian/ stable/main logstash all 1:5.0.0~alpha1-1 [65.1 MB]
Fetched 65.1 MB in 18s (3,457 kB/s)
...
I don't think we'll use S3 forever, so I'm not sure we need to change the package build step to rename the file. However, for publishing, we'll probably want to automate the file rename.
Notably, this will only impact non-GA releases (alpha, beta, etc) because GA releases do not have ~
in the version scheme.
I can confirm that Red Hat systems are unlikely to be affected by this (I tested Fedora 23) because their fetching http client doesn't turn ~
into %7e
. The bug is an integration bug between S3 and apt-get, and has been reported many times over the years to upstream debian or ubuntu but I think they only fixed the +
escaping problem.
@jordansissel Concerning the renaming I have already created #5100, to standardize the name with our different package. So I doubt we will have this issue again.
Thanks @jordansissel
@radu-gheorghe confirmed it works now
Granted, this is 16.04 beta2, but I don't think it makes a difference. After I do the recommended
echo "deb http://packages.elastic.co/logstash/5.0/debian stable main" | sudo tee -a /etc/apt/sources.list
from the docs, I'm getting this back onapt-get update
:It's not entirely clear to me where the problem is, because the logstash package isn't architecture specific, and it downloaded the
all
version just fine. Yet because thei386
andamd64
packages are missing,apt-get update
exists with an error andapt-get install
can't find logstash. So I guess there are references to the architecture-specific stuff in the repository that shouldn't be there?Downloading the .deb package and installing it seems to work, though.