Maintenance

[mambax]

Hi @dee-kryvenko

Sorry I wonder, 8months no update. Either its rock-solid (in this case I'd start to use it) or it has been abandoned for some reason 🤣

Which is the case?

Br

[dee-kryvenko]

Hi @mambax, thanks for your interest in this project!

This project is not abandoned, in a sense that I am not aware of any better solutions out there (to date) to the problem statement I outlined in the https://github.com/plumber-cd/terraform-backend-git#why-storing-state-in-git and I personally have a lot of interest and motivation in solving it.

However I wouldn't recommend using this in production unless you voluntarily accept everything that comes with being a pioneer in anything (at least - wrapper functionality in pair with GitHub should work as I was using that for a bit in early 2020). Around the time I released the last v0.0.14 9 months ago - I switched jobs, and there is no use case for this project at my new place. Yet. There doesn't seem to be much interest in the OSS community in using this either - I can see v0.0.12 linux binary was downloaded 27 times (which was probably my own CI builds) and that is a top downloaded file at this repository (others is only 1-3 downloads). On the contrary, I can see small but substantial amount of web traffic coming to this repo from Google and other search engines:

I read this as either there's small amount of people interested in this project or a problem statement it outlines - but understandably don't want to be pioneers in using something not very well tested and not mature enough.

I just want to be clear that it's a chicken-egg problem. No one, including me, currently using this - so there is no feedback to act on - so the level of maturity remains on its current. Realistically, the chances anyone besides me will jump into uncharted waters with this are pretty slim, so I doubt anything will change until I have a use case for this backend again.

The good news is that there is a use case for this backend emerging at my new job, so there is almost 100% chance I will end up using this project at work again which will inevitably drive further development and maturity up. I just can't make any commitments as to when that'll happen.

I'll leave this issue open so everyone can see my statement until I get back to working on this again.

[mambax]

Hi @dee-kryvenko

First of all, thank you for the extensive response! Second, that is a bit sad to hear 😔

I gave it a shot today and I only made it work with the "most basic" option yet.

terraform-backend-git git \   
  --repository xxx.git \
  --ref state \
  --state local.tfstate \
    terraform apply

The other ways did somehow not work out for me (probably because I did it wrong).

Anyhow, I planned actually to use it in production (kind of) and I can not really see too much reason not to because it works as it should. I also do not see too much of a reason why an S3 should be better. Our git has existing roles and permission etc. etc. so it would be way more perfect than an S3 and than storing some crappy key-pair in Passpack where everyone needs to get access again.

I can not now state that I would hop on your train, but I would most certainly give it a go. Pun unintended a bit a bummer is that I am totally un-read in GO.

I would appreciate it if you could somehow check the readme.md a bit. The one table is messed up a bit so I had problems reading it.

Also, I did not make the hcl example work. I added the hcl file, added my git info, and started it, all good. Then I added the backend information to my TF, but when I do terraform plan or whatever, he does not find any state, also I do not see any logs from the terraform-backend-git. Maybe I miss a point, what I did:

• Add HCL, add 3 git config flags (state, ref and repo)
• Added HTTP backend as described by you

  terraform {
  backend "http" {
  address = "http://localhost:6061/?type=git&repository=git@github.com:my-org/tf-state.git&ref=master&state=my/state.json"
  lock_address = "http://localhost:6061/?type=git&repository=git@github.com:my-org/tf-state.git&ref=master&state=my/state.json"
  unlock_address = "http://localhost:6061/?type=git&repository=git@github.com:my-org/tf-state.git&ref=master&state=my/state.json"
  }
  }

• Ran tf init, works
• Ran tf plan --> It wants to create 147 resources, which is wrong, they already exist.

When I use

terraform-backend-git git \   
  --repository xxx.git \
  --ref state \
  --state local.tfstate \
    terraform apply

it works perfectly so something I do wrong or maybe the wiring is not finished.

Anyway, let me know what you think 😊

[dee-kryvenko]

If it works for you - please, go ahead. If there will be bug reports - I will absolutely put an effort to allocate some time fixing it.

Thanks for noticing it - I fixed README.md.

Just out of curiosity - what makes you want to define HCL yourself and not run this backend in a wrapper mode? I'm asking is because I would love to make the wrapper mode as useful as possible and if you're not using it that tells me there's something you're missing with it.

As to your issue - you don't need a terraform-backend-git.hcl in that scenario, all configuration pieces it needs in this case will be coming from Terraform. I wonder if presence of terraform-backend-git.hcl did messed up things somehow. Does backend spits out any logs? You can add --access-logs=true, that will show us if it receives any requests at all and if yes - what exactly was in the requests.

[mambax]

what makes you want to define HCL yourself and not run this backend in a wrapper mode?

I have no clue actually what is what. I just had the requirement to run native terraform commands (for brevity and simplicity) - not prefixing 7 commands. I prefer the wrapper to wrap somewhere else than my CLI 🤣

Does backend spits out any logs?

It did not spit out a single peep. Nothing happened at all. I think TF did not even connect to it somehow. Or like you say, it got confused by the HCL.

FYI: I am gonna make a PR towards you in some minutes to add a Github Action to your backend so I can pull it into my CI 💪🏼

[dee-kryvenko]

Interesting - yeah by not using wrapper it gives you more control, but the price for this is you have to manage the gluing on your own (i.e. remote backend definition and etc).

If it didn't spit out any logs - it must be that tf didn't use your backend definition for some reason. Try not to run the backend and make sure tf is trying to use it - it should be failing to read the state while server is down.

[mambax]

but the price for this is you have to manage the gluing on your own

A worthy price 🤑 I would love that option more tbh. But I can try it out myself no worries here.

Regarding the backend: I will definitely try it again! But given the CI will need it with wrappers anyway, I might for now just stay with the wrapper (I don't want to set up docker-compose or such within Github Actions).

Appreciate your help @dee-kryvenko 🎖️

[mambax]

FYI @dee-kryvenko I use your project in "production" now. Works like a charm! 🥳

[dee-kryvenko]

@mambax I'm a little embarrassed that someone started using this in prod earlier than myself, bit hooray for you! Let me know if you encounter any issues.

[RickyGrassmuck]

@dee-kryvenko I too have been using this in production for internal projects that we deploy to Nomad using Gitlab CI/CD and it's worked wonderfully!

I actually made some changes to my fork a while back to, according to my extremely descriptive commit message(admittedly it's been a hot minute since I made the changes and once they worked I kinda just set and forgot it lol) , facilitate using the server backend while also using the wrapper.

Basically, the end workflow ended up with:

• Not needing to define the backend in the terraform block
• Using the wrapper (ie terraform-backend-git git terraform apply -input=false awesome.plan)
• The wrapper would then proceed to use the server backend.

Pretty sure I intended to open an issue/PR to see if there was interest in adopting the changes but I think I ended up getting caught up in other things and ended up forgetting (I mean, it just works so I never really have to interact with it often lol).

I could clean up my changes(Mainly imports that I changed due to my ignorance of Go Modules replace directive at the time) and open a PR if you would be interested in looking at it?

[dee-kryvenko]

Yeah that would be nice to have. Thanks @rigrassm - if you don't get to it, I'll try to make some time in April to incorporate your changes.

[ghost]

@dee-kryvenko In 7 months I became a GitHub Actions Knight from the Padawan I started as 🤣 I would now know better ways to solve what I contributed in the Action back then (besides that it uses an absolute path which I need to make a contribution anyway for).

Question: I would like to have terraform-backend-git as a docker container so I can just spin it up in a GHA Workflow as a docker container in the background. The config could be imposed either like @rigrassm proposes or we can add ENV vars.

What you think? Would you be around to approve/review and foremost, would you be able to push this either to dockerhub or GH packages? 😄

[dee-kryvenko]

@dominikmeyersap yes absolutely! It's been a long time but I'll try to take a look into that config files PR tomorrow or Monday.

[dee-kryvenko]

@rigrassm and everyone - my apologies, I know I promised to take a look into that PR in April and it's September now - I was a little bit busy with some other matters (some of it ended up being open sourced as well). Just wanted to let everybody know - this project is not abandoned, and if there's any feedback/issues - please let me know. I still intend to start using this backend myself and the current timeline for it - end of 2021 / early 2022.

@dominikmeyersap - maybe you could join the conversation in https://github.com/plumber-cd/terraform-backend-git/pull/18#issuecomment-913295943 - sounds like you were interested in this PR too and I am trying to understand what needs to be done and what's missing currently.

[dee-kryvenko]

https://github.com/plumber-cd/terraform-backend-git/issues/24