Continuous integration prototype

[patrickceg]

Add a subsystem to automatically check code when it is pushed to Github. Hopefully the system will have these attributes as well:

• Run completely using Amazon AWS Lambda or other function so we don't need to have a dev/test server all the time
• A dev can work on a development branch and call the integration tests at will on their dev branch (so they don't need to mess up Master to be able to run tests)
• Mark any issues on the Github commit as a passing or failed build, with logs (similar to a Jenkins multi-branch pipeline)

[patrickceg]

Note my schedule this week won't be great for working on this project (appointment messing up my work hours + I was crazy enough to sign up for https://global.azurebootcamp.net/ the weekend after the hackathon). I'm going to read up on solutions and post them here before committing to a platform and making a prototype.

Some suggestions posted on Slack to start looking:

https://medium.com/vandium-software/unit-testing-aws-lambda-functions-in-node-js-7ad6c8f5000

https://medium.com/@bezdelev/how-to-test-a-python-aws-lambda-function-locally-with-pycharm-run-configurations-6de8efc4b206

[jacobcode91]

Lambda functions can be updated via cli. As for testing, any platform that can run python code should be able to run the lambda functions with data from the AWS services by using the boto3 library, and using the correct region and IAM credentials.

We shouldn't need anything more than a platform with Python3, and the boto3 library installed for testing, the pycharm link @patrickceg posted gives some really good info on setting up a local test for the lambda functions, but I don't think we need to limit ourselves to the pycharm IDE for them.

GitLab offers CI/CD pipelines in their free acounts, and the pipelines play nice with Github repos as well, so I propose we try using that. If we end up requiring the paid features, I have a gitlab server I host and we can migrate to that if we end up choosing this solution and require features free accounts don't offer.

[patrickceg]

I set up a dummy "CI" (a script that does nothing, but it's still called by Gitlab on commits)

https://gitlab.com/patrickceg/rhok-bad-date-serverless/-/jobs/64350987

Now that I found that test runners are no problem, the next issues are:

1. Setting up a testing environment so we're not running tests directly on the live system
2. Designing the code to be unit testable (i.e. not mixing a direct database call in the same code as parsing the HTTP request object)

[patrickceg]

As an update, I'm pretty much doing a lot of reading into Amazon AWS.

I came across these blog posts that talk about being able to specify a serverless application via script (similar to or part of cloud formation - not sure how to describe it as yet) and then be able to use that spec to spin up a local test instance that behaves like an Amazon API gateway:

• https://aws.amazon.com/blogs/aws/new-aws-sam-local-beta-build-and-test-serverless-applications-locally/
• https://aws.amazon.com/blogs/compute/introducing-simplified-serverless-application-deplyoment-and-management/

These are just the pieces I need for a test environment, so looks like even more reading is in order for me to figure out how to leverage this stuff. The ideal case is we keep 'serverless' by having the Gitlab continuous integration build spin up all of this stuff to run the test. If that won't work too well, I'll play around with my local machine doing this.