json 日志无法提交到阿里云日志服务

[tttlkkkl]

无法提交json格式的数据到阿里云日志服务，我确定这个不是log-pilot 的bug就是阿里云日志服务的问题，因为日志格式改成none后可以正常提交。劳烦大佬们有空帮解答一下。 控制台输出错误信息为：

2019-06-06 16:10:49 +0800 [error]: #0 Could not puts logs to aliyun sls: LogException: 
{
ErrorCode: PostBodyInvalid
 ErrorMessage: fail to parse protobuf
RequestId: ["5CF8CA891DD1282C5BA272C4"]
}

目标日志收集服务docker-compose代码：

  admin:
    image: '/web/admin:latest'
    container_name: 'admin-fpm'
    expose:
      - 9000
    environment:
      PHP_ENV_SPACE: local
    networks: 
      - lnmp-network
    command: php-fpm
    # 日志目录申明为匿名卷
    volumes:
      - /www/logs
    labels:
      aliyun.logs.php-admin: /www/logs/admin/*.log
      aliyun.logs.php-admin.format: json
      aliyun.logs.php-admin.tags: 'dc=dev'

log-pilot编排代码：

  log:
    environment:
      PILOT_TYPE: fluentd
      LOGGING_OUTPUT: aliyun_sls
      ALIYUNSLS_PROJECT: dn-dt-log
      ALIYUNSLS_REGION_ENDPOINT: 'cn-shenzhen.log.aliyuncs.com'
      ALIYUNSLS_ACCESS_KEY_ID: $id
      ALIYUNSLS_ACCESS_KEY_SECRET: $secret
      SSL_VERIFY: 'true'
      ALIYUNSLS_NEED_CREATE_LOGSTORE: 'true'
      ALIYUNSLS_CREATE_LOGSTORE_TTL: 100
      ALIYUNSLS_CREATE_LOGSTORE_SHARD_COUNT: 1
    image: 'registry.cn-hangzhou.aliyuncs.com/acs/log-pilot:0.9.7-fluentd'
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - /etc/localtime:/etc/localtime
      # 不管容器日志文件目录是挂载到目录还是卷，始终会从本容器 /host 目录下面去查找，并且映射到文件在宿主机上的实际文职。所以这个挂载配置是写死的，而且至关重要
      - /:/host:ro
    privileged: 'true'
    hostname: pilot

日志来自PHP，是按照标准的json格式输出的。日志样例如下：

{"time":"2019-06-06 16:10:03","channel":"local","level_name":"ERROR","message":"日志异常抛出测试#0 [internal function]: App\\Http\\Controllers\\Web\\Index\\IndexController->index()\n#1 /www/web/admin/vendor/illuminate/container/BoundMethod.php(29): call_user_func_array(Array, Array)\n#2 /www/web/admin/vendor/illuminate/container/BoundMethod.php(87): Illuminate\\Container\\BoundMethod::Illuminate\\Container\\{closure}()\n#3 /www/web/admin/vendor/illuminate/container/BoundMethod.php(31): Illuminate\\Container\\BoundMethod::callBoundMethod(Object(Laravel\\Lumen\\Application), Array, Object(Closure))\n#4 /www/web/admin/vendor/illuminate/container/Container.php(564): Illuminate\\Container\\BoundMethod::call(Object(Laravel\\Lumen\\Application), Array, Array, NULL)\n#5 /www/web/admin/vendor/laravel/lumen-framework/src/Concerns/RoutesRequests.php(373): Illuminate\\Container\\Container->call(Array, Array)\n#6 /www/web/admin/vendor/laravel/lumen-framework/src/Concerns/RoutesRequests.php(339): Laravel\\Lumen\\Application->callControllerCallable(Array, Array)\n#7 /www/web/admin/vendor/laravel/lumen-framework/src/Concerns/RoutesRequests.php(313): Laravel\\Lumen\\Application->callLumenController(Object(App\\Http\\Controllers\\Web\\Index\\IndexController), 'index', Array)\n#8 /www/web/admin/vendor/laravel/lumen-framework/src/Concerns/RoutesRequests.php(275): Laravel\\Lumen\\Application->callControllerAction(Array)\n#9 /www/web/admin/vendor/laravel/lumen-framework/src/Concerns/RoutesRequests.php(260): Laravel\\Lumen\\Application->callActionOnArrayBasedRoute(Array)\n#10 /www/web/admin/vendor/laravel/lumen-framework/src/Concerns/RoutesRequests.php(160): Laravel\\Lumen\\Application->handleFoundRoute(Array)\n#11 [internal function]: Laravel\\Lumen\\Application->Laravel\\Lumen\\Concerns\\{closure}(Object(Illuminate\\Http\\Request))\n#12 /www/web/admin/vendor/laravel/lumen-framework/src/Routing/Pipeline.php(52): call_user_func(Object(Closure), Object(Illuminate\\Http\\Request))\n#13 /www/web/admin/vendor/illuminate/session/Middleware/StartSession.php(63): Laravel\\Lumen\\Routing\\Pipeline->Laravel\\Lumen\\Routing\\{closure}(Object(Illuminate\\Http\\Request))\n#14 /www/web/admin/vendor/illuminate/pipeline/Pipeline.php(151): Illuminate\\Session\\Middleware\\StartSession->handle(Object(Illuminate\\Http\\Request), Object(Closure))\n#15 [internal function]: Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}(Object(Illuminate\\Http\\Request))\n#16 /www/web/admin/vendor/laravel/lumen-framework/src/Routing/Pipeline.php(32): call_user_func(Object(Closure), Object(Illuminate\\Http\\Request))\n#17 /www/web/admin/vendor/illuminate/pipeline/Pipeline.php(104): Laravel\\Lumen\\Routing\\Pipeline->Laravel\\Lumen\\Routing\\{closure}(Object(Illuminate\\Http\\Request))\n#18 /www/web/admin/vendor/laravel/lumen-framework/src/Concerns/RoutesRequests.php(410): Illuminate\\Pipeline\\Pipeline->then(Object(Closure))\n#19 /www/web/admin/vendor/laravel/lumen-framework/src/Concerns/RoutesRequests.php(166): Laravel\\Lumen\\Application->sendThroughPipeline(Array, Object(Closure))\n#20 /www/web/admin/vendor/laravel/lumen-framework/src/Concerns/RoutesRequests.php(107): Laravel\\Lumen\\Application->dispatch(NULL)\n#21 /www/web/admin/public/index.php(28): Laravel\\Lumen\\Application->run()\n#22 {main}","extra":[],"context":[]}
{"time":"2019-06-06 16:10:03","channel":"local","level_name":"INFO","message":"index","extra":[],"context":["abc","bbccdd"]}

[tttlkkkl]

补充一点,怀疑跟我日志文件的编码有关系，正在进一步排查： 其他可以收集的日志文件信息为:

access.log: ASCII text, with very long lines

不能被收集的PHP日志文件信息为:

web-2019-06-06.log: UTF-8 Unicode text

[tttlkkkl]

https://docs.fluentd.org/quickstart/faq 中貌似有提到这个问题，但是我不确定。

[chenquanzhao]

@tttlkkkl 可以通过none方式采集，SLS支持日志的Split

[cccvip]

您好，参考您的这个我同样配置，无法成功。能帮我解决一下吗