how to code get commited to cert gitlab repo using this?

pawantlor commented 3 years ago

Can you please explain how the certs are getting committed inside "https://gitlab.com/certhub-gitlab-demo"

using which image.

For me certs are getting generated but how do commit happening automatically to other certificate repo?

znerol commented 3 years ago

The following happens when running one of the certhub images in gitlab ci / docker:

The container entrypoint is run: https://github.com/certhub/certhub-certbot-docker/blob/ee8beffed77f5012a5301e3f42c7a5a8c8682690/Dockerfile#L108-L114

ENTRYPOINT [ \
    "/sbin/tini", \
    "--", \
    "/usr/bin/ssh-agent", \
    "/usr/lib/git-gau/docker-entry", \
    "/usr/lib/git-gau/docker-entry.d" \
]

Looking at the docker-entry script you find https://github.com/znerol/git-gau/blob/5bf689158c3589688595c1c9aba41c6848e82bb6/lib/docker-entry#L19-L27

Thus docker-entry first runs everything in /usr/lib/git-gau/docker-entry.d using /bin/run-parts utility. After that it checks whether GAU_REPO environment variable is present. If it is, then the certificate repository gets checked out using git gau-exec and docker CMD is executed (in case of gitlab, this is the ci script). After the cmd terminates, gau-exec will push all changes back to the repository.

You find an example of the gitlab ci script here, but I guess you are aware of that already: https://gitlab.com/certhub-gitlab-demo/certbot/-/blob/master/.gitlab-ci.yml#L48-54

  script:
    - >
      git gau-ac
      git gau-xargs -I{WORKDIR}
      certhub-message-format "${CERTHUB_CERT_PATH}" x509
      certhub-certbot-run "${CERTHUB_CERT_PATH}" "${CERTHUB_CSR_PATH}"
      certbot ${CERTHUB_CERTBOT_ARGS} --config "${CERTHUB_CERTBOT_CONFIG}"

Also it helps to examine the output of pipline runs. See the following for reference (one success, one failure): Example of a successful job: https://gitlab.com/certhub-gitlab-demo/certbot/-/jobs/1535766713 Example of a failed job: https://gitlab.com/certhub-gitlab-demo/certbot/-/jobs/1451033231

znerol commented 3 years ago

Btw: All of this happens automatically if you run one of the official docker images in your pipeline (e.g. image: certhub/certhub:certbot). https://gitlab.com/certhub-gitlab-demo/certbot/-/blob/master/.gitlab-ci.yml#L1

If you run your own images, then please make sure that you run git gau-exec in your entrypoint.

pawantlor commented 3 years ago

For me,

Certificates are not getting pushed, even though proper variables are set. I cant find the git repository getting pulled. I am running your default image , but my CI runner is creating a kube pod with your image and executing all stuff, could that be a problem? Few more question:

I want to retain certificates and pvt key, I see they get generated inside tmp folder and get deleted. How do i retain and push to gcloud or gitlab as a whole?
Can we connect somehow through email/zoom/mobile to troubleshoot :)

znerol commented 3 years ago

Do you see the following ascii art output in your gitlab job log?

Official Certhub/Certbot Docker Image
=====================================
Build Log: [Travis CI Build #348](https://app.travis-ci.com/certhub/certhub-certbot-docker/builds/236059334)
                            .oO'Oo.       .oO'Oo.
                          .oO'   `Oo.   .oO'   `Oo.
                     .oO'Oo:       `° .oO'       :oO'Oo.
                   .oO'   `Oo.       oO°       .oO'   `Oo.
                 .oO'       `Oo.   .oO  o.   .oO'       `Oo.
                 oO           `Oo.oO'   `Oo.oO'           Oo
                 `Oo.           `Oo:     :oO'           .oO'
                   `Oo.           Oo     oO           .oO'
                     `Oo.       .oO'     `Oo.       .oO'
                       `Oo.   .oO'         `Oo.   .oO'
                         `Oo.oO'             `Oo.oO'
                             _    _             _         _
            ___   ___  _ __ | |_ | |__   _   _ | |__     (_)  ___
           / __| / _ \| '__|| __|| '_ \ | | | || '_ \    | | / _ \
          | (__ |  __/| |   | |_ | | | || |_| || |_) | _ | || (_) |
           \___| \___||_|    \__||_| |_| \__,_||_.__/ (_)|_| \___/
Manifest
--------
* alpine: 3.13.5
* certbot: v1.18.0
* certhub: v1.0.0
* dns-lexicon: v3.7.0
* git-gau: v1.1.0

If you do not see the ascii art, then that might be a sign that the entrypoint script is not executed.

pawantlor commented 3 years ago

I can see above output but not "git-exec" thing, This is getting executed.

Certs are getting generated inside /tmp folder but those get deleted or they are retained in any different folder?. I can see a out of result like this:

Hook '--manual-auth-hook' for *****certname_domain* ran with output: RESULT

True Hook '--manual-cleanup-hook' for *****certname_domain* ran with output: RESULT

True Successfully received certificate. Certificate is saved at: /tmp/tmp.KJdcHe/cert.pem Intermediate CA chain is saved at: /tmp/tmp.KJdcHe/chain.pem Full certificate chain is saved at: /tmp/tmp.KJdcHe/fullchain.pem This certificate expires on 2021-11-24.

pawantlor commented 3 years ago

Sorry, very truly said, i cant see above asciii art. Why its not visible?

znerol commented 3 years ago

Sorry, very truly said, i cant see above asciii art. Why its not visible?

My guess is that the entrypoint script is not executed. Please try the following experiment. In your .gitlab-ci.yml file. Modify the script section and insert the entrypoint code at the top (without the call to tini). It should look like this:

  script:
    - >
      /usr/bin/ssh-agent
      /usr/lib/git-gau/docker-entry
      /usr/lib/git-gau/docker-entry.d
      git gau-ac
      git gau-xargs -I{WORKDIR}
      certhub-message-format "${CERTHUB_CERT_PATH}" x509
      certhub-certbot-run "${CERTHUB_CERT_PATH}" "${CERTHUB_CSR_PATH}"
      certbot ${CERTHUB_CERTBOT_ARGS} --config "${CERTHUB_CERTBOT_CONFIG}"

pawantlor commented 3 years ago

Yes will let you know the output.

But one bigger qs, you just commit fullchain.pem right, what about pvt key if i want to commit? or rather Certs are getting generated inside /tmp folder but those get deleted or they are retained in any different folder? what if i want to retain them?

Also cant we generate this certificate without CSR? I dont want to pass CSR as a parameter. Certbot generally doesnt need CSR parameter while generating new cert, then why this utility needs?

pawantlor commented 3 years ago

Sorry, very truly said, i cant see above asciii art. Why its not visible?

My guess is that the entrypoint script is not executed. Please try the following experiment. In your .gitlab-ci.yml file. Modify the script section and insert the entrypoint code at the top (without the call to tini). It should look like this:
  script:
    - >
      /usr/bin/ssh-agent
      /usr/lib/git-gau/docker-entry
      /usr/lib/git-gau/docker-entry.d
      git gau-ac
      git gau-xargs -I{WORKDIR}
      certhub-message-format "${CERTHUB_CERT_PATH}" x509
      certhub-certbot-run "${CERTHUB_CERT_PATH}" "${CERTHUB_CSR_PATH}"
      certbot ${CERTHUB_CERTBOT_ARGS} --config "${CERTHUB_CERTBOT_CONFIG}"

WIth this experiment, ASCI output is diplayed on JOB output now. Cert gets created but after that job fails (i guess at uploading stage.) Can we apply sleep some how to this script, to check why its failing?

znerol commented 3 years ago

job fails (i guess at uploading stage.)

Could you please post the generated error message? If it contains private data (repository names, domain names, etc.), then please replace it with XXXX or example.com. If you copy paste log lines, then please format them as 'code' (use the < > button from the editor toolbar).

pawantlor commented 3 years ago

Yes will let you know the output.

But one bigger qs, you just commit fullchain.pem right, what about pvt key if i want to commit? or rather Certs are getting generated inside /tmp folder but those get deleted or they are retained in any different folder? what if i want to retain them?

Also cant we generate this certificate without CSR? I dont want to pass CSR as a parameter. Certbot generally doesnt need CSR parameter while generating new cert, then why this utility needs?

Please answer over this, i will run my pipeline as docker contianer rather than POD, But i want to upload pvt key as well :( Where they get stored.

znerol commented 3 years ago

Please open an issue over in the certhub repository for questions and suggestions regarding certhub. Limiting discussions to exactly one subject makes it easier for me and other people to follow up on them.

pawantlor commented 3 years ago

job fails (i guess at uploading stage.)

Could you please post the generated error message? If it contains private data (repository names, domain names, etc.), then please replace it with XXXX or example.com. If you copy paste log lines, then please format them as 'code' (use the < > button from the editor toolbar).

Logs:

Hook '--manual-cleanup-hook' for example.com ran with output: RESULT

True Successfully received certificate. Certificate is saved at: /tmp/tmp.gKMJgI/cert.pem Intermediate CA chain is saved at: /tmp/tmp.gKMJgI/chain.pem Full certificate chain is saved at: /tmp/tmp.gKMJgI/fullchain.pem This certificate expires on 2021-11-24. NEXT STEPS:

Certificates created using --csr will not be renewed automatically by Certbot. You will need to renew the certificate before it expires, by running the same Certbot command again. Saving debug log to /var/log/letsencrypt/letsencrypt.log
If you like Certbot, please consider supporting our work by:
- Donating to ISRG / Let's Encrypt: https://letsencrypt.org/donate
- Donating to EFF: https://eff.org/donate-le
Certificate: Data: Data got generate SHA256 Fingerprint=fingerprint got generated ERROR: Job failed: command terminated with exit code 123

If logs are needed from /var/log/letsencrypt/letsencrypt.log, how to add sleep to above script.

znerol commented 3 years ago

There should be more output if the final git push failed. I.e., after the certificate fingerprint you should find log output indicating what went wrong with git push.

SHA256 Fingerprint=XXXX
--->>> here should be more lines indicating what went wrong <<<---
ERROR: Job failed: command terminated with exit code 123

See the example failure here: https://gitlab.com/certhub-gitlab-demo/certbot/-/jobs/1451033231

If logs are needed from /var/log/letsencrypt/letsencrypt.log.

Those logs do not contain any relevant data for this particular case.

If you post logs, then pretty please try to format them properly using the code button (<>) from the editor toolbar. Otherwise your logs are very hard to read. See the github help for more info on how to properly format code blocks (for your log lines): https://docs.github.com/en/github/writing-on-github/getting-started-with-writing-and-formatting-on-github/basic-writing-and-formatting-syntax#quoting-code

znerol / git-gau