Watson1978
commented
3 weeks ago The data received by the cool.io gem is delimited to 16384 bytes or less and passed to the application.
However, Fluentd might not handle it properly, I think.
require 'bundler/inline'
gemfile do
source 'https://rubygems.org'
gem 'cool.io'
end
require "socket"
require "json"
require 'fileutils'
UNIX_SOCKET_PATH = "/tmp/fluentd-unix.sock"
DATA_LENGTH = 1024 * 256
FileUtils.rm_f(UNIX_SOCKET_PATH)
class Client
def data_generater
d = ['test', Time.now.to_i, {"length_#{DATA_LENGTH}": "a" * DATA_LENGTH}]
d.to_json
end
def run
Thread.new do
s = UNIXSocket.new(UNIX_SOCKET_PATH)
sleep 0.5
data = data_generater
s.send(data, 0)
puts "[client] sent size = #{data.size}"
s.close
puts "[client] Finished!!"
end
end
end
class ServerConnection < Cool.io::UNIXSocket
def on_connect
puts "[server] connected"
end
def on_close
puts "[server] disconnected"
exit
end
def on_read(data)
puts "[server] received size = #{data.size}"
end
end
server = Cool.io::UNIXServer.new(UNIX_SOCKET_PATH, ServerConnection)
server.attach(Cool.io::Loop.default)
Client.new.run
Cool.io::Loop.default.run$ ruby coolio.rb
/home/watson/.rbenv/versions/3.3.5/lib/ruby/3.3.0/json/common.rb:3: warning: ostruct was loaded from the standard library, but will no longer be part of the default gems starting from Ruby 3.5.0.
You can add ostruct to your Gemfile or gemspec to silence this warning.
[server] connected
[client] sent size = 262184
[server] received size = 16384
[client] Finished!!
[server] received size = 16384
[server] received size = 16384
[server] received size = 16384
[server] received size = 16384
[server] received size = 16384
[server] received size = 16384
[server] received size = 16384
[server] received size = 16384
[server] received size = 16384
[server] received size = 16384
[server] received size = 16384
[server] received size = 16384
[server] received size = 6272
[server] disconnected
Describe the bug
When it send the very long data at once from the client, it causes error with JSON data or warning with MessagePack format. In my environment, this always happens when I send more than 256 KB at once.
To Reproduce
It just sends data using following code
client side
Fluentd side
Expected behavior
No errors
Your Environment
Your Configuration
Your Error Log
JSON
Send JSON data from client
MessagePack
Send MessagePack data from client
Additional context
No response