Story: Server Ready Event Scraper

[AvocadoMoon]

Problem

I need to have the event scraper run without intervention, which can only be done if it's setup on a server.

Solution

Create a docker image that can be deployed on any server, and within this image it calls itself every Monday.

Tasks:

• [x] Create a docker image file for the application
  • [x] Have the image utilize a CRON job that calls the application
  • [x] Handle secrets appropriately
• [x] Find a docker image repository that I can upload this image to, or create my own
• [x] Get authentication to be server to server communication, requiring only secrets and no longer the browser Auth flow

[AvocadoMoon]

Seems as if the best practice for use with secrets with K8 is to employ them as secrets, which can then be read as environment variables within the application. Although this of course leaves the container vulnerable if any outside force gains access to it.

Git Gurdian | Code Camp | K8 | Docker

[AvocadoMoon]

Access tokens seem to be the best bet for not having to develop some web application just so that authentication can work properly. It requires google CLI, and places the access token into some default JSON file location. But a script that has documentation can be made to generate all of that, and place the file in a more convenient location. Then the result is taken into the image as a secret.

Otherwise another avenue is to do the OAuth 2.0 method through some Python script, and to extend the refresh token expiration date somehow. Although it seems that whole process may be finicky Same Problem | Supposed solution

ADC Tokens | ADC Storage

[AvocadoMoon]

What seems to be the easiest method is to have a volume mount for the secrets, and ensure the external file is handled in a secure manner. This is because secrets with K8 or Swarm seem to work best when they are distributed, and don't want to deal with some file management solution. In addition the secrets are encrypted in transit, and ensure proper authorization handling.

Slack | Reddit