gaia-pipeline / gaia

Build powerful pipelines in any programming language.
Apache License 2.0
5.2k stars 245 forks source link

Allow pipeline update #248

Closed X-Ryl669 closed 4 years ago

X-Ryl669 commented 4 years ago

First, I'd like to thank you for your much appreciated work. I've successfully started by first real pipeline on Gaia. I'm developping in C++, and using latest docker image.

Currently, while developping my pipeline, I first entered the repository for the pipeline in the Create Pipeline page, and it failed because of dumb C++ mistake. Since I can't test it easily, I had to commit a large number of "fix" commit until it worked. Since my own machine does not have gRPC (and I didn't want to pollute it with all the dependencies), it was very painful.

So I finally docker exec and clone the repository in the docker's image so at least, I was able to build it without error. So far so good, it only took one more commit to get to this building correctly point.

Then... well it was a PITA. Once I had a way to build a successful pipeline, Gaia accepted it and would not pull from the repository anymore. So I had to delete the pipeline, and recreate it for every change. In my pipeline I used the vault, so I could not stub the pipeline with define, it would not work.

Can you either provide a button to force Gaia to reload the pipeline from the initial repository ? Or better, could you document or provide a way to trigger a pipeline from the docker image/manually, so we don't have to commit tens of dumb changes to get the complete pipeline to work ?

Skarlso commented 4 years ago

Hi!

First of all, thank you for using Gaia! Much appreciated. :)

Second, you can turn on polling in the settings menu. This will periodically make pulls and rebuild pipelines automatically.

Now, there is no reason to not add a force pull or something like that to a pipeline, so I think that's still a good idea, I'm just saying what you can do right now.

Another option. albeit I realise that that is not really a nice dev flow you can rebuild your pipeline and copy over the binary. But if I understand correctly, that's not what you want.

So your best bet for now is turning on the poller which polls ever minute.

Meanwhile I'll look into the force rebuild button ;)

Skarlso commented 4 years ago

Oh I forgot to add that you can also setup a webhook for git, which will immediately rebuild your pipeline when there is a commit.

X-Ryl669 commented 4 years ago

Oh I forgot to add that you can also setup a webhook for git, which will immediately rebuild your pipeline when there is a commit.

How do you do that ? I'm not using github here, but Gitea.

X-Ryl669 commented 4 years ago

Another option. albeit I realise that that is not really a nice dev flow you can rebuild your pipeline and copy over the binary.

That's a very good idea indeed. I want to prevent all the useless "fix one line" commit until I bump on the next one. This would help a lot. Thanks!

Skarlso commented 4 years ago

Oh I forgot to add that you can also setup a webhook for git, which will immediately rebuild your pipeline when there is a commit.

How do you do that ? I'm not using github here, but Gitea.

Oh yeah, we don't support Gitea hooks yet. Sorry mate. :/ But it shouldn't be too hard to add one. :)

Skarlso commented 4 years ago

@X-Ryl669 hi.

This update in the next release will contain a pull button which updates the pipeline's code.

https://github.com/gaia-pipeline/gaia/pull/265

Skarlso commented 4 years ago

Done, please reopen if still an issue.