Have MING pull from project-maintained dockerfiles

[MatthewCroughan]

Currently we maintain our own dockerfiles which is non-standard and less ideal for a standardized setup. We should see how pulling from dockerhub for these applications goes and try and make them work well together. I think we can offer more value this way anyway, as it would shift the aims of the project towards making this stack work well and alleviate the need for us to maintain the even nittier and grittier details of the environment that needs to be composed for Grafana for example, when they have a working and maintained alpine version on Dockerhub.

If we can source their dockerfiles from git repositories, it would be wise to find a way to include those dockerfiles as submodules so we retain the ability to build the images. As well as this, it'd be great to find a way to implement additional layers as INCLUDES, such that we can append device specific layers/informatio to the dockerfile if need be.

[MatthewCroughan]

I currently don't have push access for whatever reason, so I'm going to work on this in my own fork then submit a PR.

[MatthewCroughan]

An interesting thing to note is that there are documentation benefits to be had, since this docker image has documentation we can link to/clone/symlink/alias here on github.

Not only that, but this docker image is only 3MB. For some reason, the Balenalib alpine images are upwards of 15MB. I'm sure we don't need whatever it's providing just to run the mosquitto executable, so we can take some space savings here.

https://hub.docker.com/_/eclipse-mosquitto

[MatthewCroughan]

You can use env variables to assert configuration in docker-compose and grafana. So instead of copying in the grafana.ini you can have envvars.

 default section
instance_name = ${HOSTNAME}

[security]
admin_user = admin

[auth.google]
client_secret = 0ldS3cretKey

This would become:

export GF_DEFAULT_INSTANCE_NAME=my-instance
export GF_SECURITY_ADMIN_USER=true
export GF_AUTH_GOOGLE_CLIENT_SECRET=newS3cretKey

Or in our case:

  grafana:
    restart: always
    image: grafana/grafana:latest
    environment:
      - GF_PATHS_DATA=/data/grafana
      - GF_AUTH_ANONYMOUS_ENABLED=true
      - GF_AUTH_ANONYMOUS_ORG_NAME=DoES
      - GF_AUTH_ANONYMOUS_ORG_ROLE=Viewer
    ports:
      - "80:3000"
    volumes:
      - 'grafana-data:/data'
    depends_on:
      - influxdb

for example. This follows the format. GF_<SectionName>_<KeyName> https://grafana.com/docs/installation/configuration/

[MatthewCroughan]

After testing a build derived from dockerhub images the total image size is 648.37MB

[MatthewCroughan]

The functionality described earlier relating to INCLUDES can be and often is accomplished by using dockerfiles in the build directories for your applications.

I found this out here https://forums.balena.io/t/magicmirror-issues-with-bind-mounts/5644/4

In our case it would be:

FROM grafana/grafana
STUFF
MORESTUFF

in which case grafana essentially becomes an INCLUDE

https://docs.docker.com/compose/compose-file/compose-file-v2/#dockerfile

[MatthewCroughan]

This is a good reference for provisioning datasources in Grafana. https://github.com/grafana/grafana/blob/master/devenv/datasources.yaml

[MatthewCroughan]

I ran into a massive issue relating to the balena build system not supporting architecture manifests. I've figured it out, my adventure is logged here

https://forums.balena.io/t/balena-cloud-cli-builder-is-still-not-able-to-interpret-image-manifests/43203/3

It is not stated anywhere in the dockerfile documentation that it is possible to use --platform in the FROM directive. But you can. I gleamed this from this open issue on Moby.

https://github.com/moby/moby/issues/36552

[MatthewCroughan]

Even though there is an outstanding usability issue in Balena relating to a bunch of things that can be followed by the references injected here, this issue is resolved since the title has been completed.

[MatthewCroughan]

@richbayliss I was kind of new to Balena during this time, so ignore inadequacies in my terminology. This issue kind of documents how I ran into the issue and links to the forum thread I made on it. That forum thread references other forum threads.

[MatthewCroughan]

@wrboyce