Open ghost opened 6 years ago
It seems your chunk size is much larger than per line. Try using v.each_line { }
to get the per line JSON.
oh! thanks
also, how do I change the chunk size?
so, does the readme need to be updated? cause I started with a copy/paste and it resulted in this error.
Actually, looking at one of our internal tools, it looks like we do exactly what you're doing and it works fine. I'll pull out the bits of code:
...
::Docker::Image.build_from_tar(archive, options, @connection) do |line|
log_build_line(line)
end
...
then:
def log_build_line(line)
stream = JSON.parse(line)['stream']
stream.each_line { |msg| debug "[build] #{msg.chomp}" } if stream
rescue
debug "[build] #{line}"
end
You can reduce the chunk size by creating a new docker connection object with additional parameters:
::Docker::Connection.new(
::Docker.url,
::Docker.env_options.merge(chunk_size: 1)
)
thank you very much for the quick response!
I was still running into issues getting a normal looking stream from docker after trying the above, using the yajl-ruby
gem got me what I was looking for:
parser = Yajl::Parser.new
parser.on_parse_complete = ->(obj) { print obj['stream'] }
# In context of a rails app, change to your dockerfile dir
image = Docker::Image.build_from_dir(Rails.root.to_s) { |line| parser.parse(line) }
You should see stdout identical to running docker build
from your shell.
It looks like log streaming example shown here in the README does still result in the second issue OP reported (the JSON::ParserError
) when lines of output contain a newline character (e.g. {"stream":"\n"}
).
Here's a small example, build.rb
:
#!/usr/local/bin/ruby -w
require 'docker'
Docker::Image.build_from_dir('.') do |v|
if (log = JSON.parse(v)) && log.has_key?("stream")
$stdout.puts log["stream"]
end
end
$ ./build.rb
(Excon::Error::Socket)s/2.4.0/gems/json-2.1.0/lib/json/common.rb:156:in `parse': 765: unexpected token at '{"stream":"\n"}
' (JSON::ParserError)
from /usr/local/lib/ruby/gems/2.4.0/gems/json-2.1.0/lib/json/common.rb:156:in `parse'
from ./build.rb:6:in `block in <main>'
from /usr/local/lib/ruby/gems/2.4.0/gems/docker-api-1.34.2/lib/docker/image.rb:345:in `block in response_block'
from /usr/local/lib/ruby/gems/2.4.0/gems/excon-0.62.0/lib/excon/response.rb:119:in `parse'
from /usr/local/lib/ruby/gems/2.4.0/gems/excon-0.62.0/lib/excon/middlewares/response_parser.rb:7:in `response_call'
from /usr/local/lib/ruby/gems/2.4.0/gems/docker-api-1.34.2/lib/excon/middlewares/hijack.rb:45:in `response_call'
from /usr/local/lib/ruby/gems/2.4.0/gems/excon-0.62.0/lib/excon/connection.rb:414:in `response'
from /usr/local/lib/ruby/gems/2.4.0/gems/excon-0.62.0/lib/excon/connection.rb:263:in `request'
from /usr/local/lib/ruby/gems/2.4.0/gems/docker-api-1.34.2/lib/docker/connection.rb:40:in `request'
from /usr/local/lib/ruby/gems/2.4.0/gems/docker-api-1.34.2/lib/docker/connection.rb:65:in `block (2 levels) in <class:Connection>'
from /usr/local/lib/ruby/gems/2.4.0/gems/docker-api-1.34.2/lib/docker/image.rb:274:in `build_from_tar'
from /usr/local/lib/ruby/gems/2.4.0/gems/docker-api-1.34.2/lib/docker/image.rb:293:in `build_from_dir'
from ./build.rb:5:in `<main>'
Explicitly changing my chunk_size
to 1 gets me further, but I do fail later on with other output from Docker:
(Excon::Error::Socket)s/2.4.0/gems/json-2.1.0/lib/json/common.rb:156:in `parse': 765: unexpected token at '{"stream":" ---\u003e 8e5bf52abd1a\n"}
{"stream":"Step 7/7 : RUN locale-gen"}
{"stream":"\n"}
The yajl-ruby
solution posted above by @hayfever does work beautifully, however.
Passing a block to
build_from_dir
does not stream the output, also results in a JSON parse error.docker-api 1.28.0
Repro
Output:
First, I expect to see
*** LOG STATEMENT ***
once per line or on a regular basis.Second, this JSON is invalid. and I get
JSON::ParserError: