Figure out how to obtain deployment name + namespace

[vlerkin]

Figure out how to obtain scrapyd-k8s' deployment name + current namespace from within scrapyd-k8s. This will probably need the Downward API.

Document this in https://github.com/q-m/scrapyd-k8s/issues/13.

[vlerkin]

During the first look it seems not complicated to achieve, docs provide a clear way, for example, how to implement it using environment variables, also possible using volumes. Working on this issue.

[wvengen]

Also, the namespace can be used as a default namespace (simplifying configuration, so it can be omitted usually).

Note that the pod's (=deployment's) namespace can be found in /var/run/secrets/kubernetes.io/serviceaccount/namespace already (but I don't think the deployment name is there). Also see https://stackoverflow.com/a/57809472

[vlerkin]

namespace is there, I checked /var/run/secrets/kubernetes.io/serviceaccount/namespace but for deployment name we can use ENV.

- name: MY_POD_NAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name

Also, I can see that somebody worked on a related issue, is there a PR waiting for approval? Where in python code do we want to read this info?

[wvengen]

Ah, great. This looks like it is the pod name, whereas we want to know the deployment name. Can we also get the namespace name in an environment variable?

Work was started in #13 but no PR was submitted. When you're a bit familiar with scrapyd-k8s, it's probably less work to do this yourself than to ask them to submit a PR and finish it - so feel free!

[vlerkin]

Yes, sorry, for deployment name we need to check metadata.labels. to be clear is this the namespace (pod namespace) we want to get?

- name: MY_POD_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace

Ok, thank you.

[wvengen]

Yes, pod namespace is fine.

[wvengen]

While at it, there are two cases: in-cluster (what you looked at here) and when running not on the cluster itself.

When not running on the cluster, the kubernetes Python client reads the configuration (like, from ~/.kube), it should be possible to get the default namespace too, so that can be used.

[vlerkin]

when running not on the cluster itself

Could you please give more details about this? Where k8s runs if not on the cluster? Do you use a different yaml file for that or the same?

[wvengen]

Please see the different ways to run scrapyd-k8s in the README. You can run it in Docker, in Kubernetes, and Local. "when running not on the cluster itself" would be for the last (Local).

[vlerkin]

Changes in kubernetes.yaml file:

          env:
            - name: MY_POD_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
            - name: DEPLOYMENT_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.labels['app.kubernetes.io/name']

Extraction of deployment name is done following the example from the last comment of this stack overflow issue you recommended to look into, pod namespace extracted following Kubernetes docs.

Result:

valeriiaklestova@BA92-HC7TN99KW6 scrapyd-k8s % kubectl exec -it scrapyd-k8s-745dc46d6f-894cn sh                       
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.
$ echo $DEPLOYMENT_NAME
scrapyd-k8s
$ echo $MY_POD_NAMESPACE
default

Is this something you had in mind?

[wvengen]

Super, yes!

[wvengen]

maybe call DEPLOYMENT_NAME -> MY_DEPLOYMENT_NAME, in harmony with the other variable.

[wvengen]

Anyway, the figuring out is done, closing.