search

WallarooLabs / wally

Distributed Stream Processing
https://www.wallaroolabs.com
Apache License 2.0
1.48k stars 68 forks source link

using giles sender distracts from learning about applications #1552

Open aturley opened 7 years ago

aturley commented 7 years ago

"A python or C++ based script to generate the data that the Giles Sender does would lessen the number of things needed to run the initial app and help to focus on what the application is about."

aturley commented 7 years ago

I think this is related to the conversations that we've had about adding file sources and things like that to reduce the number of things that need to be done to see an example application process data.

nisanharamati commented 7 years ago

There were python examples for a sender and receiver in the first iteration of the python api docs, IIRC

nisanharamati commented 7 years ago

https://github.com/WallarooLabs/wallaroo/blob/8307a7ac370a65374b7150a19369ed3c1061d086/book/appendix/writing-your-own-feed.md

SeanTAllen commented 7 years ago

I'm not in favor of having c++ apps and python apps and apps in every other language that are used to feed data into example applications.

SeanTAllen commented 7 years ago

I think it should also be noted as part of this issue that we also got user feedback that the inclusion of giles sender/receiver was a good thing.

aturley commented 7 years ago

My takeaway from this was that it was more about reducing the number of things the user needs to deal with in order to run one of our example applications. I think it also may not be exactly clear to the user what Giles Sender is doing, so to them it seems like a weird thing that they have to build with a weird compiler, and then run with a bunch of strange arguments.

I can reach out to the user and see if they can provide any more information on what the specific issue was that they were trying to describe, and how specifically they think that including a python or C++ program would solve this issue.

I believe that they are describing some sort of issue that could exist for other folks, but

  1. the description may not capture the root of the problem
  2. there may be a better way to address the problem than the solution they've described

Also, I agree with @SeanTAllen I'm not a fan of having a bunch of programs out there to feed data into example applications. But maybe there's a better way to send data than what we're doing right now, where "better way" could mean any number of things that we can fight about later.

jtfmumm commented 5 years ago

@aturley We now have GenSource and a related Python generator source. We could use this in examples, in which case the inputs will be generated internally and there's no need for an external sender. Would this address your concern here?