search

Hydrospheredata / hydro-serving

MLOps Platform
http://docs.hydrosphere.io
Apache License 2.0
271 stars 42 forks source link

Does it support cassandra? #147

Closed AbhayAg closed 5 years ago

AbhayAg commented 6 years ago

Does it support cassandra?

@AbhayAg I've removed template text from your message

Zajs commented 6 years ago

Hi @AbhayAg, For what purposes do you want to use Cassandra?

AbhayAg commented 6 years ago

For data storing and making data storage fault tolerant.

On Mon 7 May, 2018, 2:21 PM Eduard, notifications@github.com wrote:

Hi @AbhayAg https://github.com/AbhayAg, For what purposes do you want to use Cassandra?

  • as model storage?
  • as metadata storage instead of Potgresql?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Hydrospheredata/hydro-serving/issues/147#issuecomment-387001489, or mute the thread https://github.com/notifications/unsubscribe-auth/Ad8joyok0BTj1OrPvURaektcrkQ1OsbSks5twAsWgaJpZM4T0j25 .

Zajs commented 6 years ago

Now we store next things:

Can you clarify which kind of data you want to store in Cassandra?

AbhayAg commented 6 years ago

I will be using Cassandra for storing transactions data which I can't afford to loose at any cost. and one more thing I was trying to install hs for serving cli but it is not there. How can I deploy my model?

On Mon, May 7, 2018 at 2:58 PM, Eduard notifications@github.com wrote:

Now we store next things:

  • serialized models - we use Amazon S3 or local file system (HDFS in future releases)
  • metadata for models (contracts, build history, versions links) - we use PostgreSQL
  • system and model performance metrics - we use InfluxDF/Elasticsearch/ Prometheus

Can you clarify which kind of data you want to store in Cassandra?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Hydrospheredata/hydro-serving/issues/147#issuecomment-387010787, or mute the thread https://github.com/notifications/unsubscribe-auth/Ad8jo3u0VelxUNbge9q5x7KKdf9S0hGQks5twBO_gaJpZM4T0j25 .

KineticCookie commented 6 years ago

Hello @AbhayAg

and one more thing I was trying to install hs for serving cli but it is not there

What version of python do you use? Did you use pip install hs?

How can I deploy my model?

Basically it's just cd model_folder && hs upload --host $HOST -- port $PORT But you need to fill some metadata first. Here you can find detailed information. Here you can find 3 example models with filled metadata.

AbhayAg commented 6 years ago

Does it work with python 3.0. Because I am using it with python 2.0

On Mon, May 7, 2018 at 3:55 PM, Bulat notifications@github.com wrote:

Hello @AbhayAg https://github.com/AbhayAg

and one more thing I was trying to install hs for serving cli but it is not there

What version of python do you use? Did you use pip install hs?

How can I deploy my model?

Basically it's just cd model_folder && hs upload --host $HOST -- port $PORT But you need to fill some metadata first. Here https://github.com/Hydrospheredata/hydro-serving-cli you can find detailed information. Here https://github.com/Hydrospheredata/hydro-serving-example/tree/master/stateful_lstm_example/pipeline you can find 3 example models with filled metadata.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Hydrospheredata/hydro-serving/issues/147#issuecomment-387024111, or mute the thread https://github.com/notifications/unsubscribe-auth/Ad8jo7fPke5T-1At3jDZaEOK3HfWP9bZks5twCE2gaJpZM4T0j25 .

KineticCookie commented 6 years ago

@AbhayAg Yes, python3 is the one you need.

AbhayAg commented 6 years ago

Ok. I will try that. Thanks man.

On Mon 7 May, 2018, 5:25 PM Bulat, notifications@github.com wrote:

@AbhayAg https://github.com/AbhayAg Yes, python3 is the one you need.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Hydrospheredata/hydro-serving/issues/147#issuecomment-387041873, or mute the thread https://github.com/notifications/unsubscribe-auth/Ad8jo5qIV1v7ec2z2pZba9Jrg-5sggPTks5twDYlgaJpZM4T0j25 .

Zajs commented 6 years ago

@AbhayAg Are you going to store each request and response to your models?

AbhayAg commented 6 years ago

Yeah in some situations I may store that too. But my main reason behind using that is I want my data to be safe and always available. So I cannt move on from Cassandra. Do you have connector for that?

On Mon 7 May, 2018, 5:35 PM Eduard, notifications@github.com wrote:

@AbhayAg https://github.com/AbhayAg Are you going to store each request and response to your models?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Hydrospheredata/hydro-serving/issues/147#issuecomment-387044091, or mute the thread https://github.com/notifications/unsubscribe-auth/Ad8jo6Hfv8cPeutZIuUlIr1q3vcvk06Zks5twDiUgaJpZM4T0j25 .

Zajs commented 6 years ago

Right now we don't have a connector to Cassandra.

AbhayAg commented 6 years ago

Are you guys planning to build one for Cassandra? Because I think cassandra is getting more and more traction. May be you can guide me how to write connector for cassandra?

On Mon, May 7, 2018 at 5:43 PM, Eduard notifications@github.com wrote:

Right now we don't have a connector to Cassandra.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Hydrospheredata/hydro-serving/issues/147#issuecomment-387045839, or mute the thread https://github.com/notifications/unsubscribe-auth/Ad8jo1vn7e5G7oXeDTYXXY_m5BC3yUJgks5twDp2gaJpZM4T0j25 .

Zajs commented 6 years ago

Yes, we are planning to build a connector for Cassandra, also we are going to create seed project for integration with SMACK stack.

If you want to create the connector for Cassandra you will have to implement *Repository interfaces.

AbhayAg commented 6 years ago

I have only 1.5 years of experience . So, I think I will be needed more help. Can you start the initial structure and then I can contribute in that. Then you can test it. Is it possible?

On Tue, May 8, 2018 at 1:44 PM, Eduard notifications@github.com wrote:

Yes, we are planning to build a connector for Cassandra, also we are going to create seed project for integration with SMACK stack.

If you want to create the connector for Cassandra you will have to implement *Repository https://github.com/Hydrospheredata/hydro-serving/tree/master/manager/src/main/scala/io/hydrosphere/serving/manager/repository interfaces.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Hydrospheredata/hydro-serving/issues/147#issuecomment-387322245, or mute the thread https://github.com/notifications/unsubscribe-auth/Ad8jowr1ty6XBnuYoQMdnk-8x8zKBZMpks5twVPQgaJpZM4T0j25 .

AbhayAg commented 6 years ago

Hey By the way I am also getting some issue while running docker-compose. Can you please check once. error is :-

prometheus | time="2018-05-08T08:21:05Z" level=error msg="Error reading file "/var/targets/allServices.json": invalid character 'o' in literal null (expecting 'u')" source="file.go:199"

Revert me asap.

On Tue, May 8, 2018 at 1:47 PM, Abhay Agarwal < abhay.agarwal@squareoneinsights.com> wrote:

I have only 1.5 years of experience . So, I think I will be needed more help. Can you start the initial structure and then I can contribute in that. Then you can test it. Is it possible?

On Tue, May 8, 2018 at 1:44 PM, Eduard notifications@github.com wrote:

Yes, we are planning to build a connector for Cassandra, also we are going to create seed project for integration with SMACK stack.

If you want to create the connector for Cassandra you will have to implement *Repository https://github.com/Hydrospheredata/hydro-serving/tree/master/manager/src/main/scala/io/hydrosphere/serving/manager/repository interfaces.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Hydrospheredata/hydro-serving/issues/147#issuecomment-387322245, or mute the thread https://github.com/notifications/unsubscribe-auth/Ad8jowr1ty6XBnuYoQMdnk-8x8zKBZMpks5twVPQgaJpZM4T0j25 .

KineticCookie commented 6 years ago

@AbhayAg could you please create a new issue and fill more information about docker-compose?

  1. what docker compose file do you use?
  2. what environment variables did you set prior to launching compose
  3. did you try to docker-compose pull before launch?
AbhayAg commented 6 years ago

Yeah It worked. Thanks man.

AbhayAg commented 6 years ago

Thanks Bulat. It worked. But I am finding it little hassled as every time I change my docker it stops working and It created rigid coupling between docker and docker-compose. So, can you suggest something for that?

On Tue, May 8, 2018 at 2:57 PM, Bulat notifications@github.com wrote:

@AbhayAg https://github.com/AbhayAg could you please create a new issue and fill more information about docker-compose?

  1. what docker compose file do you use?
  2. what environment variables did you set prior to launching compose
  3. did you try to docker-compose pull before launch?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Hydrospheredata/hydro-serving/issues/147#issuecomment-387342335, or mute the thread https://github.com/notifications/unsubscribe-auth/Ad8joxSXmpVKriUg-8OchUZlwnlGaQZLks5twWUNgaJpZM4T0j25 .