Open morteza1131 opened 1 year ago
srikumar003
commented
1 year ago Hi, I haven't tried this mode of using datashim but the error failed to start daemon: Unable to get the TempDir under /var/lib/docker: mkdir /var/lib/docker/tmp: transport endpoint is not connected seems to indicate a network issue. Please check if you have specified endpoint url correctly
morteza1131
commented
1 year ago I used the PV in simple Nginx pod and its working fine, but when using it as docker root volume , daemon does not start.
srikumar003
commented
1 year ago Could you post here how you configured datashim/pod ? Also, if you could explain why you are doing it, since so far we have not tried using the PVC in this manner.
You could also try overriding the mountPath in the CSI-S3 manifests: https://github.com/datashim-io/datashim/blob/8ec79a3aa25334c287dd2c877a2e3765c441f6ca/src/csi-s3/chart/templates/csi-s3.yaml#L140C30-L140C45 and see if that works (re: #160)
morteza1131
commented
1 year ago I want to use s3 volume as my pod volume to use s3 disk instead of using local disk in my kubernetes environment. I use this:
apiVersion: com.ie.ibm.hpsys/v1alpha1
kind: Dataset
metadata:
name: docker-cache-s3-dataset
spec:
local:
type: "COS"
accessKeyID: "adf"
secretAccessKey: "adfadsfasdfa"
endpoint: "http://somes3.com"
bucket: "docker-cache"
readonly: "false"
---
kind: ConfigMap
apiVersion: v1
metadata:
name: dind-daemon-test
namespace: adminstuff
data:
config.json: |
{
"auths": {
"docker.example.com": {
"auth": "sfdadfa"
},
"https://index.docker.io/v1/": {
"auth": "adfadsfa="
},
"pvreg.example.com": {
"auth": "adfadfa="
}
},
"proxies":
{
"default":
{
"httpProxy": "http://examples.com",
"httpsProxy": "http://examples.com",
}
}
}
---
kind: Deployment
apiVersion: apps/v1
metadata:
name: dockerbuilder-test
namespace: adminstuff
labels:
app: dockerbuilder
spec:
replicas: 1
selector:
matchLabels:
app: dockerbuilder
template:
metadata:
creationTimestamp: null
labels:
app: dockerbuilder
spec:
volumes:
- name: "docker-dir"
persistentVolumeClaim:
claimName: "docker-cache-s3-dataset"
- name: dind-daemon
configMap:
name: dind-daemon
defaultMode: 420
containers:
- name: dind
image: 'docker:23.0.2-dind'
command:
- dockerd-entrypoint.sh
- '--insecure-registry=pvreg.example.com'
- '--registry-mirror=https://pvreg.example.com'
ports:
- containerPort: 2375
protocol: TCP
env:
- name: http_proxy
value: https://pvreg.example.com
- name: https_proxy
value: https://pvreg.example.com
- name: DOCKER_OPTS
value: >-
-H tcp://0.0.0.0:2375
- name: DOCKER_TLS_CERTDIR
volumeMounts:
- name: "docker-dir"
mountPath: /var/lib/docker
- name: dind-daemon
mountPath: /root/.docker/config.json
subPath: config.json
imagePullPolicy: IfNotPresent
securityContext:
privileged: true@morteza1131 i would really not recommend changing your docker-volumes to use the volumes provided by csi-s3. S3 volumes are not POSIX compliant, so there could be serious problems when docker daemon provisions the filesystem for the containers.
morteza1131
commented
1 year ago I don't want to use it as docker container volume, I only need it to use as docker daemon root volume to store container images build cache.
I used created pvc as docker volume in /var/lib/docker I got the following error: