Closed jasperf closed 3 years ago
Perhaps I can sync every now and then with - "-wait=14400"
but that would not be a deployment when I have pushed to the repository either manually or automatically... unless it can wait and sync only if there has been an update?
P.S. Was reading https://tech.travelaudience.com/publishing-code-documentation-in-kubernetes-4afa7f7b6d58
I can deploy
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: git-data
labels:
tier: backend
namespace: smt-local
spec:
# autoscale using `kubectl autoscale deployment x --cpu-percent=50 --min=1 --max=10` instead of setting replicas
# https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale-walkthrough/
# replicas: 2
selector:
matchLabels:
app: git-data
tier: backend
strategy:
rollingUpdate:
maxSurge: 1
maxUnavailable: 1
minReadySeconds: 5
template:
metadata:
labels:
app: git-data
tier: backend
spec:
containers:
- name: git-data
image: k8s.gcr.io/git-sync:v3.0.1
# command: [ "sh", "-c", "ls /code/project" ]
volumeMounts:
- name: code-storage
mountPath: /code
env:
- name: GIT_SYNC_REPO
value: https://github.com/smart48/smt-demo.git
- name: GIT_SYNC_ROOT
value: /code
- name: GIT_SYNC_BRANCH
value: master
- name: GIT_SYNC_DEPTH
value: "1"
## symlink directory
- name: GIT_SYNC_DEST
value: current
# not one off syncing as we will deploy often
- name: GIT_SYNC_ONE_TIME
value: "0"
# - name: GIT_SYNC_USERNAME
# valueFrom:
# secretKeyRef:
# name: git-creds
# key: username
# - name: GIT_SYNC_PASSWORD
# valueFrom:
# secretKeyRef:
# name: git-creds
# key: password
volumes:
- name: code-storage
persistentVolumeClaim:
claimName: code-pv-claim
and it works and finally stores in the correct volume and under the correct directory. But how can I deploy again without destroying the pod and doing it all again? The latter removes all revisions and that is not what I want of course.
Hmm, my bad. As soon as I updated the repository the sync happened. So I guess I do need to add wait if I want the update to be delayed. Still have questions:
Also had to do
chown -R laradock:laradock code
to get the code directory to be owned by laradock which is equal to docker. It was nobody at the start
root@workspace-566b747498-7dwd2:/code# ll
total 16
drwxrwxrwx 4 root root 4096 Dec 22 05:43 ./
drwxr-xr-x 1 root root 4096 Dec 22 05:38 ../
lrwxrwxrwx 1 nobody nogroup 44 Dec 22 05:43 current -> rev-959ac23559f27850a6af708f29a1b97da724aa8a/
drwxr-xr-x 9 nobody nogroup 4096 Dec 22 05:43 .git/
drwxr-xr-x 13 nobody nogroup 4096 Dec 22 05:43 rev-959ac23559f27850a6af708f29a1b97da724aa8a/
Need this at least until I can make it a separate user and group called www-data
.. And then I still have the issue with Git Sync as used now.
Besides that I had to do:
sudo find . -type f -exec chmod 664 {} \;
sudo find . -type d -exec chmod 775 {} \;
Other outstanding questions:
Due to chown changes I could also no longer update to latest version using Git Sync. So I do need to be able to set user and group if possible here.
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
runAsUser: 1000
runAsGroup: 1000
works for setting the user and group now. So only need to work out:
php artisan key:generate
On sync update adjusted .env
gets overwritten / removed. This should not happen. The same goes for the composer packages. We want to keep the PHP composer packages as well as env file and other . files.
I do now see two revisions:
root@workspace-566b747498-c6xs5:/code# ll
total 20
drwxrwxrwx 5 root root 4096 Dec 22 09:50 ./
drwxr-xr-x 1 root root 4096 Dec 22 06:43 ../
lrwxrwxrwx 1 laradock laradock 44 Dec 22 09:50 current -> rev-349f933222e29953cc244e3c3553ecf04c2b8ee8/
drwxr-xr-x 9 laradock laradock 4096 Dec 22 06:43 .git/
drwxr-xr-x 14 laradock laradock 4096 Dec 22 09:50 rev-349f933222e29953cc244e3c3553ecf04c2b8ee8/
drwxr-xr-x 3 laradock laradock 4096 Dec 22 09:50 rev-587a3dd9326187c0df394871fe2bf36030ec0113/
So revisions are made after all... well, one rev is almost empty:
root@workspace-566b747498-c6xs5:/code# cd rev-587a3dd9326187c0df394871fe2bf36030ec0113/
root@workspace-566b747498-c6xs5:/code/rev-587a3dd9326187c0df394871fe2bf36030ec0113# ll
total 12
drwxr-xr-x 3 laradock laradock 4096 Dec 22 09:50 ./
drwxrwxrwx 5 root root 4096 Dec 22 09:50 ../
drwxr-xr-x 51 root root 4096 Dec 22 07:21 vendor/
and for some reason this revision directory is owned by root:root
I also do miss vendor directory and added .env
. So the issue
On sync update adjusted .env gets overwritten / removed. This should not happen. The same goes for the composer packages. We want to keep the PHP composer packages as well as env file and other . files.
still remains.
I am thinking symlinks to ../shared/.env
and ../shared/storage/..
. Not sure yet how. Buut phph composer vendor
I normally do keep there in the project root. Could do a build locally and add that build and npm build (excluding node_modules
directory) to the repository I guess.
I do need to add wait if I want the update to be delayed
The "wait" is really just the loop's retry period. It will sleep(wait) and then look for changes upstream.
can I do a manual sync?
Not really. I'm not sure why you need this - you can set the wait period very low - like every few seconds - and it will catch up quickly. That said, how would you want to do a manual sync? From outside the pod? Wouldn't you need some form of auth to do that?
can I exclude sub modules?
You can turn submodules off, if you like. -submodules=off
can I keep more than one revision?
You can keep history, but git-sync doesn't have a way to check out multiple revs at the same time. I'm not AGAINST the idea, but it would be a bit complicated and has not really been asked before.
how to set number of revisions
Can you explain, hypothetically, how you would want it to work?
option to revert to previous revision or deployment
If you change the tag in upstream, git-sync will sync again. There isn't a "local" revert. The whole purpose of git-sync is to represent a given symbol in a remote repository in your pod. To change it locally, change it remotely.
So revisions are made after all. I however do miss vendor directory and added .env. So the issue
Revisions are made, but they are garbage collected. Git-sync is locked to upstream on a single branch/tag/sha. I'm not clear exactly what behavior you are hoping for - can you explain?
The "wait" is really just the loop's retry period. It will sleep(wait) and then look for changes upstream.
I see. Understand that now. Like a bash sleep x seconds
thing. Okay.
You can turn submodules off, if you like. -submodules=off
Yes, found out a bit later yesterday. Thanks
You can keep history, but git-sync doesn't have a way to check out multiple revs at the same time. I'm not AGAINST the idea, but it would be a bit complicated and has not really been asked before.
That is something we were looking for .. so we can rollback to older revisions or releases with ease.
If you change the tag in upstream, git-sync will sync again. There isn't a "local" revert. The whole purpose of git-sync is to represent a given symbol in a remote repository in your pod. To change it locally, change it remotely.
Okay, understood.
Revisions are made, but they are garbage collected. Git-sync is locked to upstream on a single branch/tag/sha. I'm not clear exactly what behavior you are hoping for - can you explain?
Well normally, using PHP Deployer, we update to the latest Github git repository version and update the symlink to link to that deployment or release.
forge@app-prod-w-2:~/site.com$ ll
total 20
drwxrwxr-x 5 forge forge 4096 Dec 7 03:41 ./
drwxr-xr-x 9 forge forge 4096 Nov 6 23:30 ../
drwxrwxr-x 2 forge forge 4096 Dec 7 03:41 .dep/
lrwxrwxrwx 1 forge forge 11 Dec 7 03:40 current -> releases/15/
drwxrwxr-x 12 forge forge 4096 Dec 7 03:41 releases/
drwxrwxr-x 4 forge forge 4096 Dec 7 03:35 shared/
releases directory:
forge@app-prod-w-2:~/app.com/releases$ ll
total 48
drwxrwxr-x 12 forge forge 4096 Dec 7 03:41 ./
drwxrwxr-x 5 forge forge 4096 Dec 7 03:41 ../
drwxrwxr-x 14 forge forge 4096 Nov 18 01:00 10/
drwxrwxr-x 14 forge forge 4096 Nov 18 12:16 11/
drwxrwxr-x 14 forge forge 4096 Nov 25 06:51 12/
drwxrwxr-x 14 forge forge 4096 Nov 25 09:18 13/
drwxrwxr-x 14 forge forge 4096 Dec 4 02:45 14/
drwxrwxr-x 14 forge forge 4096 Dec 7 03:41 15/
drwxrwxr-x 14 forge forge 4096 Oct 21 03:52 6/
drwxrwxr-x 14 forge forge 4096 Oct 21 06:57 7/
drwxrwxr-x 14 forge forge 4096 Nov 5 02:25 8/
drwxrwxr-x 14 forge forge 4096 Nov 14 06:02 9/
The former release or revision is kept and we can always rollback to that release or revision if need be.
Inside the symlinked revision or release we symlink to a ../shared
directory
forge@app-prod-w-2:~/domain.com/current$ ll
total 3624
drwxrwxr-x 14 forge forge 4096 Dec 7 03:41 ./
drwxrwxr-x 12 forge forge 4096 Dec 7 03:41 ../
-rw-rw-r-- 1 forge forge 96 Dec 7 03:41 .babelrc
lrwxrwxrwx 1 forge forge 17 Dec 7 03:41 .env -> ../../shared/.env
.....
where we load the .env
file from:
forge@app-prod-w-2:~/app.com/shared$ ll
total 24
drwxrwxr-x 4 forge forge 4096 Dec 7 03:35 ./
drwxrwxr-x 5 forge forge 4096 Dec 7 03:41 ../
-rw-rw-r-- 1 forge forge 2633 Dec 7 03:41 .env
-rw-rw-r-- 1 forge forge 1704 Dec 7 03:41 .some_key
drwxrwxr-x 4 forge forge 4096 Oct 16 07:52 public/
drwxrwxr-x 5 forge forge 4096 Oct 16 07:53 storage/
which we never want to lose. We also load the uploads directory, logs and some other shared data from there.
So I wonder if we could keep revisions, do possible rollbacks and or add symlinks to data we never want to lose. I somehow feel like we may not be able to do all this and I am still looking to have PHP Deployer kubectl
into our workshop containter to do its thing as well. Just seems hard as well so ever since I have been trying a Kubernetes way to have these type of deployments and rollbacks.
Are those releases subdirs in the git repo or actually git tags ?
On Tue, Dec 22, 2020, 5:55 PM Jasper Frumau notifications@github.com wrote:
The "wait" is really just the loop's retry period. It will sleep(wait) and then look for changes upstream.
I see. Understand that now. Like a bash sleep x seconds thing. Okay.
You can turn submodules off, if you like. -submodules=off
Yes, found out a bit later yesterday. Thanks
You can keep history, but git-sync doesn't have a way to check out multiple revs at the same time. I'm not AGAINST the idea, but it would be a bit complicated and has not really been asked before.
If you change the tag in upstream, git-sync will sync again. There isn't a "local" revert. The whole purpose of git-sync is to represent a given symbol in a remote repository in your pod. To change it locally, change it remotely.
Revisions are made, but they are garbage collected. Git-sync is locked to upstream on a single branch/tag/sha. I'm not clear exactly what behavior you are hoping for - can you explain?
Well normally, using PHP Deployer we update to the latest Github git repository version and update the symlink to link to that deployment or release.
forge@app-prod-w-2:~/site.com$ ll total 20 drwxrwxr-x 5 forge forge 4096 Dec 7 03:41 ./ drwxr-xr-x 9 forge forge 4096 Nov 6 23:30 ../ drwxrwxr-x 2 forge forge 4096 Dec 7 03:41 .dep/ lrwxrwxrwx 1 forge forge 11 Dec 7 03:40 current -> releases/15/ drwxrwxr-x 12 forge forge 4096 Dec 7 03:41 releases/ drwxrwxr-x 4 forge forge 4096 Dec 7 03:35 shared/
releases directory:
forge@app-prod-w-2:~/app.com/releases$ ll total 48 drwxrwxr-x 12 forge forge 4096 Dec 7 03:41 ./ drwxrwxr-x 5 forge forge 4096 Dec 7 03:41 ../ drwxrwxr-x 14 forge forge 4096 Nov 18 01:00 10/ drwxrwxr-x 14 forge forge 4096 Nov 18 12:16 11/ drwxrwxr-x 14 forge forge 4096 Nov 25 06:51 12/ drwxrwxr-x 14 forge forge 4096 Nov 25 09:18 13/ drwxrwxr-x 14 forge forge 4096 Dec 4 02:45 14/ drwxrwxr-x 14 forge forge 4096 Dec 7 03:41 15/ drwxrwxr-x 14 forge forge 4096 Oct 21 03:52 6/ drwxrwxr-x 14 forge forge 4096 Oct 21 06:57 7/ drwxrwxr-x 14 forge forge 4096 Nov 5 02:25 8/ drwxrwxr-x 14 forge forge 4096 Nov 14 06:02 9/
The former release or revision is kept and we can always rollback to that release or revision if need be. Inside the symlinked revision or release we symlink to a ../shared directory
forge@app-prod-w-2:~/domain.com/current$ ll total 3624 drwxrwxr-x 14 forge forge 4096 Dec 7 03:41 ./ drwxrwxr-x 12 forge forge 4096 Dec 7 03:41 ../ -rw-rw-r-- 1 forge forge 96 Dec 7 03:41 .babelrc lrwxrwxrwx 1 forge forge 17 Dec 7 03:41 .env -> ../../shared/.env .....
where we load the .env file from:
forge@app-prod-w-2:~/app.com/shared$ ll total 24 drwxrwxr-x 4 forge forge 4096 Dec 7 03:35 ./ drwxrwxr-x 5 forge forge 4096 Dec 7 03:41 ../ -rw-rw-r-- 1 forge forge 2633 Dec 7 03:41 .env -rw-rw-r-- 1 forge forge 1704 Dec 7 03:41 .some_key drwxrwxr-x 4 forge forge 4096 Oct 16 07:52 public/ drwxrwxr-x 5 forge forge 4096 Oct 16 07:53 storage/
which we never want to lose. We also load the uploads directory, logs and some other shared data from there.
So I wonder if we could keep revisions, do possible rollbacks and or add symlinks to data we never want to lose. I somehow feel like we may not be able to do all this and I am still looking to have PHP Deployer kubectl into our workshop containter to do its thing as well. Just seems hard as well so ever since I have been trying a Kubernetes way to have these type of deployments and rollbacks.
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/kubernetes/git-sync/issues/316#issuecomment-749877244, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABKWAVBTVQMK62XEO3BSHFDSWFE23ANCNFSM4VEAXWXA .
The release sub directories are generated by PHP Deployer based on the last deployment. And these deploymentsare based on the Github repository status at that time. On deployment deployer grabs the latest master , ssh-ed into the server and deploys that as the current release. At the same time it runs some stuff like composer install, adds symlinks and such. I was hoping we can pull in current revision and keep previous revisions to mimic this setup.
Sorry, I am not that familiar with the rest of your setting, so some of it isn't clear to me.
Let's remove some of the specifics. There is a git repo. It holds your to-be-deployed code.
When a deployment happens, do you make a new subdir inside the git repo (so it accumulates directories) or do you edit the state of the "true" repo and make a commit? If so, do you tag the state?
When you check it out, so you check out a specific tag? Or just the main branch?
When you do a rollback, what does that mean?
On Tue, Dec 22, 2020, 8:55 PM Jasper Frumau notifications@github.com wrote:
The release sub directories are generated by PHP Deployer based on the last deployment. And these deploymentsare based on the Github repository status at that time. On deployment deployer grabs the latest master , ssh-ed into the server and deploys that as the current release. At the same time it runs some stuff like composer install, adds symlinks and such. I was hoping we can pull in current revision and keep previous revisions to mimic this setup.
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/kubernetes/git-sync/issues/316#issuecomment-749931814, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABKWAVDPTRBQZ3DLJT2B7ETSWFZ4HANCNFSM4VEAXWXA .
When a deployment happens, do you make a new subdir inside the git repo (so it accumulates directories) or do you edit the state of the "true" repo and make a commit? If so, do you tag the state?
We would/ are going to tag git updates based on states, yes. Currently deployer just creates a state/release based on latest code. We do not create new sub directories with symlinks are the shared directory as the php script does that.
When you check it out, so you check out a specific tag? Or just the main branch?
Currently we just check out the main branch. We are going to tag, but we never needed it using Deployer.
When you do a rollback, what does that mean?
When we rollback Deployer grabs a release or revision place on the server from last git repo checkout. Then one before the current symlinked version.
So if I understand correctly, I'm not sure git-sync can help, at least within the confines of what you are describing.
Git-sync currently only syncs ONE remote rev (branch, tag, sha) and publishes that one.
If you are asserting that the HEAD of the "master" brach is always what you want to deploy, git-sync can manage that by keeping track of the SHA that "master" references in the upstream and making the local repotrack that. If you need multiple of those, git-sync doesn't handle it.
On Tue, Dec 22, 2020 at 9:34 PM Jasper Frumau notifications@github.com wrote:
When a deployment happens, do you make a new subdir inside the git repo (so it accumulates directories) or do you edit the state of the "true" repo and make a commit? If so, do you tag the state?
We would/ are going to tag git updates based on states, yes. Currently deployer just creates a state/release based on latest code. We do not create new sub directories with symlinks are the shared directory as the php script does that.
When you check it out, so you check out a specific tag? Or just the main branch?
Currently we just check out the main branch. We are going to tag, but we never needed it using Deployer.
When you do a rollback, what does that mean?
When we rollback Deployer grabs a release or revision place on the server from last git repo checkout. Then one before the current symlinked version.
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/kubernetes/git-sync/issues/316#issuecomment-749944335, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABKWAVHKSKYK7E5Q6OHS2VDSWF6PJANCNFSM4VEAXWXA .
@thockin Thanks a lot for hanging in there with me and explaining things . Yes, it seems I need to look elsewhere. Going to see if I can ssh into Ingress and then get port forwarded to workspace to do deployment there now https://github.com/kubernetes/minikube/issues/10031 Still quite new to Kubernetes so need to see what Minikube Ingress and general Ingress tricks there are to realize our standard deployments.
Thanks again for your insights. Appreciate it a lot.
I have to say that, from a distance, it sounds like you're doing things The Hard Way - SSH and port forwarding generally are hacky things you do to figure out what's up, but NOT productionable practices.
I'm still not 100% confident I could draw your deployment architecture, but unless other pieces are mutable, I don't think git-sync as-is helps. I'm sorry to hear that.
On Tue, Dec 22, 2020 at 9:50 PM Jasper Frumau notifications@github.com wrote:
@thockin Thanks a lot for hanging in there with me and explaining things . Yes, it seems I need to look elsewhere. Going to see if I can ssh into Ingress and then get port forwarded to workspace to do deployment there now kubernetes/minikube#10031 Still quite new to Kubernetes so need to see what Minikube Ingress and general Ingress tricks there are to realize our standard deployments.
Thanks again for your insights. Appreciate it a lot.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.
Well, I was looking for a non ssh way to upgrade. One without using Deployer that does seem to be able to run composer
, php artisan
and other commands inside the workspace container. One option that allows me to automate this all with one command preferably.
I have not found a non commercial deployment SaaS way to do this with Laravel PHP in Kubernetes yet. Tried Git Sync.. and though it does get the latest from Git repo as I have seen now it does not do uninterrupted deployments with the option to rollback. I will keep on searching .
And if you know an option do please let me know. I do need one where code is separated from Docker image preferably..
FYI. The Deployment architecture in progress is at https://github.com/smart48/smt-deploy with local using Minikube progressed the most
These are questions from someone coming from a PHP Deployer world where I use
dep deploy production
locally with PHP Deployer package installed to grab the latest repository code from Github chosen and push this to the production website. This using a simple config file in which we add a Laravel recipe, caching, repository, ssh native as choice,.. , number or releases and more. Seedeploy.php
example below.What I am wondering about is how I can I
GIT_SYNC_WEBHOOK_URL
, but it is not clear to me how this could be used / triggered from Github for example..Here the deployment draft:
Here the deployment script we use for a standard DO VPS setup:
Update
Did find this inside https://github.com/kubernetes/git-sync/blob/release-3.x/test_e2e.sh so under a test shell file:
Is this a webhook of sorts perhaps? Does not look like it and other example in the same file at least have
-webhook-url="http://127.0.0.1:$NCPORT" \
but again I am not seeing how to use this. We could we just use a shell script and only run this pod/container when we want to sync but that is not automated..