search

LamaAni / postgres-xl-helm

A postgres-xl chart for helm
MIT License
9 stars 2 forks source link

Add testing for the chart #38

Open LamaAni opened 4 years ago

LamaAni commented 4 years ago

FUTURE FEATURE

What

Add testing for the helm chart on:

  1. Deployment
  2. Persistant deployment.
  3. Query the database.
  4. Create database
  5. Create table
  6. Create schema.

Acceptance criteria

  1. All tests running.
  2. All tests pass.
  3. There is a script for running all tests.
sstubbs commented 4 years ago

I will do this and do some benchmarking scripts. I should have an 2 initial prs done by the end of tomorrow. Is there any testing library you want me to use in a different container? Or should I just echo results? I've done testing in many languages but not bash.

LamaAni commented 4 years ago

Well in general there is bats testing library for bash, but its not very good.

I have not tested helm charts before, and not sure what tools to use. This ticket is also about creating the test structure for this repo, including all the tools that will be used to integrate the various test scenarios and the scope and definition of the test suite. Moreover, there is a question about github integration that needs to be taken into account, though I think for this specific ticket a bash command script to test is good enough, as long as we can integrate that later.

I know this tool exists: https://github.com/helm/chart-testing, but have not found the time to go about and check it out.

@sstubbs You think you want to take all that on :) ? I know its a bit of work.

LamaAni commented 4 years ago

Maybe I should expand the ticket info.

sstubbs commented 4 years ago

I'm happy to do it. I honestly wouldn't have been able to get something working nearly as well as this project and will be using it a lot in the future both for dev and in production at some point.

Been running postgres xl in production for years and it's a headache.

I will get it started and we can see what you want. I can do bash tests for what is proposed in this ticket and have opened a new one for benchmarking https://github.com/LamaAni/postgres-xl-helm/issues/41

In terms of the chart testing I won't be able to look into that for some time to be honest as I'm still quite new to helm and need to understand what is going on this chart first before I understand how to test it.

LamaAni commented 4 years ago

Hi, ok. So I will take the side of chart testing. I wanna learn about that anyways, it will probably be just template generation testing since I cannot connect the chart result to a kubernetes instance in github.

It might be a bit before I get there since I think we still need to get the backup and restore features of WAL in, which are actually much more important then testing automatically.

sstubbs commented 4 years ago

OK great thanks.

sstubbs commented 4 years ago

You right all these bash testing libraries are pretty old and badly maintained. I will just use standard bash and this https://github.com/torokmark/assert.sh

LamaAni commented 4 years ago

Hi,

I think this would require a testing platform more similar to bats. See list of options and some help about this here: https://www.leadingagile.com/2018/10/unit-testing-shell-scriptspart-one/

Best Zav

sstubbs commented 4 years ago

Just read this now. I did that pull request with that assertion library I posted but can happily change to anything you prefer. Looked at these options but don't see any real benefits to using libraries that havent been updated in a while. Perhaps I'm missing something though.

sstubbs commented 4 years ago

I guess https://zunit.xyz/ does look good but requires people to be using zsh. Personally I do use zsh but do you think adding this dependancy is a good idea? Anyway let me know what you think.

LamaAni commented 4 years ago

The reason is not the project itself, rather the reason is to have a specific format for curating tests.

Considering the images used, and the fact that we are using bash across the board, I think we should be using https://github.com/kward/shunit2/

Its a bit old but very solid.

sstubbs commented 4 years ago

ok makes sense let me update my pr to rather use this then instead of that assertion library. Will let you know once it's done.

sstubbs commented 4 years ago

I've changed the pull request to use shunit2. Please have a look at it when you have some time. It seems a bit quicker than the assertion library at running the script so I've increased the sleep 5 to sleep 15 as it was failing with 5. Just so you know what to look for when you want to try and remove it.