search

hampgoodwin / iata-finder

Spec, Proto, and Scripts to etl, and access IATA airline information.
0 stars 0 forks source link

Build out CI/CD so we can practice deployments? #20

Open hampgoodwin opened 4 years ago

hampgoodwin commented 4 years ago

I would like to get some more experience with CI/CD stuff but I/m not 100% sure on which service I want to use.

I'm mostly keen on this: https://concourse-ci.org/ because it's a chance to taken an open-source ci/cd (which I think is going to be popular) and deploy that, which will then deploy the app.

Here is a list of CI/CD tools we could use. https://github.com/ligurio/awesome-ci

hampgoodwin commented 4 years ago

Semaphore also seems like a reasonable option https://semaphoreci.com/pricing

JacobSNGoodwin commented 4 years ago

I'm also interested in this as I am not terribly familiar and need to learn. I'll take a look at these links

hampgoodwin commented 4 years ago

I think Concourse might be better to read about in their docs, it seemed good quality when I started reading it. If not, you could look into TravisCI, it's a very popular well supported one 🤷‍♀

JacobSNGoodwin commented 4 years ago

I am not educated enough in CI/CD to compare them. But concourse looks like it will do what we need.

The tutorials from https://concoursetutorial.com/basics/ are quite useful.

The real challenge may actually be configuring what sort of tests we want run with each commit or PR. So we need to figure you what parts of gRPC might be unit tested and what might require some sort of integration testing.

I guess if we take a "snapshot" of the airline data, or assume it doesn't change, maybe we can use gRPC client code to run some tests. Or maybe evans CLI can be used for this.

hampgoodwin commented 4 years ago

I think we could write a basic rpc client, and use that client in our tests?

For example, if we obtain a csv of just IATA, ICAO and NAME/DESCRIPTION or something like this, the IATA should always equal the same-named airport. We could use the client to run through the list and make sure no name/iata-icao pairings have changed. I think that's a reasonable test.

Then some basic stuff, like calling get GetAllAirports and GetAllAirlines should return an array, the array should not be empty.

Stuf like this might be useful unit tests we could get started on. This hinges on making a basic client though...

I would be comfortable making an issue for making a client and eventually getting started on that; I've gone a little dark for a moment since I'm work working again, but I'm feeling the itch to work on this some more.

...

Should our concourse build be it's own repo? I imagine we have config files in this repo that concourse uses, but I don't think we want to jam all the concourse stuff in here...

Let me know your thoughts :)

hampgoodwin commented 4 years ago

Actually, if we have the .proto and the .pb.go; we can use those methods there to run tests, I'm not sure we need a "Client" per say.

JacobSNGoodwin commented 4 years ago

@abelgoodwin1988, you're right about not needing a "client" since the generated code provides all of the methods we need. We'd just need a go test file that can be run from concourse.

I think Concourse can probably be configured inside of the current project as the config file is not too complex, and it runs in a docker container.

It may be possible (and likely), that concourse will handle the current docker builds within one of it's tasks/jobs (I forget the naming conventions). These tasks can do some basic work with docker containers, but can also call bash (*.sh) scripts (and maybe Makefiles, too, but it wouldn't be hard to switch over if needed).

If you want to work on Concourse, I could work on writing some tests that you can call. But I'm probably gonna hack around with Concourse anyway for learning pleasure.

hampgoodwin commented 4 years ago

Sounds good to me! I think I mentioned somewhere that I'm a little busy with work/life and haven't had a chance to work on this since I was home. I'm down to review anything though. If you feel you start to get a handle on concourse I'd super appreciate a crash course at some point.

@Maxbrain0 As far as "deployment" with concourse, I'm more thinking we'll need to have a server somewhere running concourse, right?

JacobSNGoodwin commented 4 years ago

@abelgoodwin1988,

Ideally we would need a server somewhere running concourse. I haven't seen much in the tutorials about deployment, but I think it would be straightforward.

I'm spending this morning looking into some projects to see how folks are using it.