search

logbee / keyscore

Apache License 2.0
3 stars 0 forks source link

Integration of Live-Editing in Pipely #66

Closed EndallBatan93 closed 5 years ago

EndallBatan93 commented 5 years ago

After the evaluation it is clear, that the current workflow of switching between live-editing and pipely is not intuitiv. Therefore it is inevitable that we integrate the live-editing data table as a preview to the pipely editor.

For this to happen we have to develop a concept that is really working. Some question that are not clearly answered:

  1. Currently the live-editing is only working on a running pipeline. If we integrate the live editing into pipely this would mean that we really pause the instance of the pipeline. Maybe this is not an expected behaviour for the user?

  2. Maybe we should rework the live-editing completely to enable testing a configuration without the necessity of a running pipeline instance?

  3. Does the integration of the Live-Editing into Pipely automatically means that the Live-Editing as an own site is gone or do we provide it additionally?

If you have any ideas or more questions please add them.

kKdH commented 5 years ago

Hey @EndallBatan93 some thoughts:

Currently the live-editing is only working on a running pipeline. If we integrate the live editing into piply this would mean that we really pause the instance of the pipeline. Maybe this is not an expected behavior for the user?

Why do you think it is a problem to pause the 'real' pipeline? You imply that we should deploy a new instance of the pipeline for testing? If so, think of the trouble we run into when we deploy multiple instances of a pipeline, even if it is just for testing and the computed results would be discarded. Most people forget the problems of parallelism and concurrency.

Maybe we should rework the live-editing completely to enable testing a configuration without the necessity of a running pipeline instance?

Running a pipeline is cheap. And that is one of the keystones of KS: What you see is what you get. There is no special configuration testing or something else. For sure we could run something similar to a pipeline, grab date from somewhere and put it into a filter instance, but the behavior could be completely different and you lose the WYSIWYG feature.

Does the integration of the Live-Editing into Pipely automatically means that the Live-Editing as an own site is gone or do we provide it additionally?

In its current state yes we could completely remove the page and integrate it into the piply page. But when we add more features to explore the data and want to give the user more space with a clearer layout why not using a complete page?

EndallBatan93 commented 5 years ago

In my head it always was a problem that we stop the pipeline when we are editing it but as you explained the other way would bring to much difficulties with it.

Running a pipeline is cheap. And that is one of the keystones of KS: What you see is what you get. There is no special configuration testing or something else. For sure we could run something similar to a pipeline, grab date from somewhere and put it into a filter instance, but the behavior could be completely different and you lose the WYSIWYG feature.

I agree with you. But then we can show the datatable only after the pipeline is build completely and is instantiated.

kKdH commented 5 years ago

Let us integrate LE into the Piply page and then we develop features which ensure that a user can run pipelines easily. e.g.:

EndallBatan93 commented 5 years ago

https://github.com/logbee/keyscore/commit/dcab1117ab22550ed9df2aeccb562021947691bc

EndallBatan93 commented 5 years ago

Closing this issue because it's largely integrated. Creating a new issue with a list of things which need to be done for completion of the feature.