search

SeldonIO / seldon-server

Machine Learning Platform and Recommendation Engine built on Kubernetes
https://www.seldon.io/
Apache License 2.0
1.47k stars 300 forks source link

td-agent configuration file #12

Closed boogardgodig closed 8 years ago

boogardgodig commented 8 years ago

I recently deployed seldon-server 1.2.3. I can add actions using the api but no new action logs are created in ${TOMCAT_HOME}/logs/fluentd/ and GroupActionsJob is not working. td-agent.conf is set to parse the previous (not json) action logs. How can I fix it?

ukclivecox commented 8 years ago

We have updated the td-agent.conf file in the master latest until we do a release. Update this file from master or make the one line diff to change to "format json" Thanks for spotting this.

boogardgodig commented 8 years ago

Thank you for your reply. Would you kindly help me out a little more? I still cannot make it work with docker seldonio/td-agent 0.1. I don't know what else is wrong. I can get actions log in fluentd if i change actions.log to 2016,03,09,08,56,08,client,default,23,291,3,1.0,"user123","291" with the old regex format. but nothing happens with new td-agent.config file and json format actions. what should I do?

On Wed, Mar 16, 2016 at 12:27 AM, cliveseldon notifications@github.com wrote:

Closed #12 https://github.com/SeldonIO/seldon-server/issues/12.

— You are receiving this because you authored the thread. Reply to this email directly or view it on GitHub https://github.com/SeldonIO/seldon-server/issues/12#event-591474299

ukclivecox commented 8 years ago

That is strange. And you are using the latest code on master? You say you can now get it working with the old td-agent.conf? That would seem to imply you are running an older version of the server.

boogardgodig commented 8 years ago

I used git clone https://github.com/SeldonIO/seldon-server.git to get seldon. maven generates seldon-server-1.2.3.war. I was manually changing the ${TOMCAT_HOME}/logs/seldon-server/actions/actions.log to see if there is a problem with td-agent. when I use the old td-agent.conf file with csv actions.log everything works. but with new td-agent.conf file with actions that are actually logged by seldon 1.2.3 I don't get any fluentd action logs. Is there a problem with td-agent version?

On 3/18/16, cliveseldon notifications@github.com wrote:

That is strange. And you are using the latest code on master? You say you can now get it working with the old td-agent.conf? That would seem to imply you are running an older version of the server.

You are receiving this because you authored the thread. Reply to this email directly or view it on GitHub: https://github.com/SeldonIO/seldon-server/issues/12#issuecomment-198565693

boogardgodig commented 8 years ago

I finally noticed docker image seldonio_tdagent is using /etc/td-agent/td-agent.conf with old format. I changed the script that starts td-agent to mount that folder and fixed it :) Thank you very much for your time

On 3/19/16, boogard godig boovidig@gmail.com wrote:

I used git clone https://github.com/SeldonIO/seldon-server.git to get seldon. maven generates seldon-server-1.2.3.war. I was manually changing the ${TOMCAT_HOME}/logs/seldon-server/actions/actions.log to see if there is a problem with td-agent. when I use the old td-agent.conf file with csv actions.log everything works. but with new td-agent.conf file with actions that are actually logged by seldon 1.2.3 I don't get any fluentd action logs. Is there a problem with td-agent version?

On 3/18/16, cliveseldon notifications@github.com wrote:

That is strange. And you are using the latest code on master? You say you can now get it working with the old td-agent.conf? That would seem to imply you are running an older version of the server.

You are receiving this because you authored the thread. Reply to this email directly or view it on GitHub: https://github.com/SeldonIO/seldon-server/issues/12#issuecomment-198565693

ukclivecox commented 8 years ago

Thanks for giving us the info and glad its working. We'll update the docker image.