search

Ankermgmt / ankermake-m5-protocol

Ankermake M5 protocol specifications and libraries *NOT AFFILIATED WITH ANKER*
GNU General Public License v3.0
161 stars 33 forks source link

Docker installation documentation is not complete #116

Open kivinen opened 1 year ago

kivinen commented 1 year ago

Describe the bug

The first level of instructions say you select either docker or git installation and then says you need to follow instructions for the install method you selected.

The Docker installation method https://github.com/Ankermgmt/ankermake-m5-protocol/blob/master/documentation/install-from-docker.md only says you get the login.json and then follow Docker Compose instructions, but if you just say in any random directory docker compose pull and docker compose up nothing useful happens. Also the instructions refers to the docker compose, I think it should use docker-compose (with dash).

To Reproduce

Log in to linux system (in this 64bit aarch64 raspberry pi running 64 desktop rasbian) 

raspberrypi:~ $ docker compose pull
docker: 'compose' is not a docker command.
See 'docker --help'
pi@raspberrypi:~ $ docker-compose pull
ERROR: 
        Can't find a suitable configuration file in this directory or any
        parent. Are you in the right directory?

        Supported filenames: docker-compose.yml, docker-compose.yaml

pi@raspberrypi:~ $ git clone https://github.com/Ankermgmt/ankermake-m5-protocol.git
pi@raspberrypi:~ $ cd ankermake-m5-protocol/
pi@raspberrypi:~/ankermake-m5-protocol $ docker-compose pull
ERROR: The Compose file './docker-compose.yaml' is invalid because:
Unsupported config option for services: 'ankerctl'
Unsupported config option for volumes: 'ankerctl_vol'
pi@raspberrypi:~/ankermake-m5-protocol $ docker-compose --version
docker-compose version 1.25.0, build unknown
pi@raspberrypi:~/ankermake-m5-protocol $

But even then the docker-compose fails with error messages. Perhaps some other version of docker-compose is required.

Expected behavior

It should say that even if you ARE using docker install, you do need to get some files from somewhere before you run

docker-compose pull

Also it is also nice to list debian or other package requirements if known, I think in this case this means docker-compose package for debian.

Desktop (please complete the following information):

kvanzuijlen commented 1 year ago

docker-compose is outdated, see https://docs.docker.com/compose/migrate/

kivinen commented 1 year ago

Meaning I need some special newer version of docker, so documenting what version is needed should be added to the page... (and no I will not start installing any funny software coming from some untrusted repository, thus I most likely will not be able to test docker images on raspberry pi then).

On the other hand there is still an issue that you still most likely are required to use git clone first before saying that docker compose command...

billyjbryant commented 1 year ago

This isn't some special new version of Docker... It is the fact that docker-compose (Compose v1) is legacy and is EoL in July 2023. It was last updated in May 2021. Docker Compose v2 was released in April of 2022. This is unrelated to our project or its documentation really.

I do agree that we should instruct you to download the repository or at least the docker-compose.yaml file before running docker compose pull.

Maybe: 

curl -o docker-compose.yaml https://raw.githubusercontent.com/Ankermgmt/ankermake-m5-protocol/master/docker-compose.yaml
docker compose up --no-build --pull always
kivinen commented 1 year ago

If you see my output docker compose pull returned error in my install:

pi@raspberrypi:~/anker $ curl -o docker-compose.yaml https://raw.githubusercontent.com/Ankermgmt/ankermake-m5-protocol/master/docker-compose.yaml
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   553  100   553    0     0    891      0 --:--:-- --:--:-- --:--:--   891
pi@raspberrypi:~/anker $ ls -lag
total 12
drwxr-xr-x  2 pi 4096 May 25 03:45 .
drwxr-xr-x 20 pi 4096 May 25 03:45 ..
-rw-r--r--  1 pi  553 May 25 03:45 docker-compose.yaml
pi@raspberrypi:~/anker $ docker compose up --no-build --pull always
unknown flag: --no-build
See 'docker --help'.

Usage:  docker [OPTIONS] COMMAND

A self-sufficient runtime for containers

Options:
      --config string      Location of client config files (default "/home/pi/.docker")
  -c, --context string     Name of the context to use to connect to the daemon (overrides
                           DOCKER_HOST env var and default context set with "docker context use")
  -D, --debug              Enable debug mode
  -H, --host list          Daemon socket(s) to connect to
  -l, --log-level string   Set the logging level ("debug"|"info"|"warn"|"error"|"fatal")
                           (default "info")
      --tls                Use TLS; implied by --tlsverify
      --tlscacert string   Trust certs signed only by this CA (default "/home/pi/.docker/ca.pem")
      --tlscert string     Path to TLS certificate file (default "/home/pi/.docker/cert.pem")
      --tlskey string      Path to TLS key file (default "/home/pi/.docker/key.pem")
      --tlsverify          Use TLS and verify the remote
  -v, --version            Print version information and quit

Management Commands:
  builder     Manage builds
  config      Manage Docker configs
  container   Manage containers
  context     Manage contexts
  image       Manage images
  manifest    Manage Docker image manifests and manifest lists
  network     Manage networks
  node        Manage Swarm nodes
  plugin      Manage plugins
  secret      Manage Docker secrets
  service     Manage services
  stack       Manage Docker stacks
  swarm       Manage Swarm
  system      Manage Docker
  trust       Manage trust on Docker images
  volume      Manage volumes

Commands:
  attach      Attach local standard input, output, and error streams to a running container
  build       Build an image from a Dockerfile
  commit      Create a new image from a container's changes
  cp          Copy files/folders between a container and the local filesystem
  create      Create a new container
  diff        Inspect changes to files or directories on a container's filesystem
  events      Get real time events from the server
  exec        Run a command in a running container
  export      Export a container's filesystem as a tar archive
  history     Show the history of an image
  images      List images
  import      Import the contents from a tarball to create a filesystem image
  info        Display system-wide information
  inspect     Return low-level information on Docker objects
  kill        Kill one or more running containers
  load        Load an image from a tar archive or STDIN
  login       Log in to a Docker registry
  logout      Log out from a Docker registry
  logs        Fetch the logs of a container
  pause       Pause all processes within one or more containers
  port        List port mappings or a specific mapping for the container
  ps          List containers
  pull        Pull an image or a repository from a registry
  push        Push an image or a repository to a registry
  rename      Rename a container
  restart     Restart one or more containers
  rm          Remove one or more containers
  rmi         Remove one or more images
  run         Run a command in a new container
  save        Save one or more images to a tar archive (streamed to STDOUT by default)
  search      Search the Docker Hub for images
  start       Start one or more stopped containers
  stats       Display a live stream of container(s) resource usage statistics
  stop        Stop one or more running containers
  tag         Create a tag TARGET_IMAGE that refers to SOURCE_IMAGE
  top         Display the running processes of a container
  unpause     Unpause all processes within one or more containers
  update      Update configuration of one or more containers
  version     Show the Docker version information
  wait        Block until one or more containers stop, then print their exit codes

Run 'docker COMMAND --help' for more information on a command.

To get more help with docker, check out our guides at https://docs.docker.com/go/guides/

pi@raspberrypi:~/anker $

Even if I remove the extra options I get error:

pi@raspberrypi:~/anker $ docker compose up
docker: 'compose' is not a docker command.
See 'docker --help'
pi@raspberrypi:~/anker $

So my guess is that I need so me new version of docker than what is provided by the debian. The version that comes with system is:

pi@raspberrypi:~/anker $ docker --version
Docker version 20.10.5+dfsg1, build 55c4c88
pi@raspberrypi:~/anker $
kivinen commented 1 year ago

Or it could be that I need some other package than docker-compose:

pi@raspberrypi:~/anker $ dpkg -l 'docker*'
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name                      Version                 Architecture Description
+++-=========================-=======================-============-================================>
ii  docker                    1.5-2                   all          transitional package
un  docker-ce                 <none>                  <none>       (no description available)
un  docker-ce-cli             <none>                  <none>       (no description available)
un  docker-ce-rootless-extras <none>                  <none>       (no description available)
ii  docker-compose            1.25.0-1                all          Punctual, lightweight developmen>
un  docker-doc                <none>                  <none>       (no description available)
ii  docker.io                 20.10.5+dfsg1-1+deb11u2 armhf        Linux container runtime
pi@raspberrypi:~/anker $
billyjbryant commented 1 year ago

No you need to upgrade the version of Docker that you have installed. But the issue isn't that you don't have a "specific" version of Docker installed its Docker Compose that isn't the right version, but it means that you are using outdated software that is EoL. See if you can upgrade docker, if not, not sure how to restore functionality on the RPI

billyjbryant commented 1 year ago

You need to upgrade docker-compose to 2+

kivinen commented 1 year ago

If I need to upgrade to some newer version of docker-compose, then there is some requirement to use some specific newer version of docker than what is installed by the debian 11 by default. Such requirement needs to be documented so people knows that they can't use docker on default install on debian 11, raspbian OS etc.

To find instructions how to do that, I need first to know which version of docker/docker-compose is needed. Normal instructions simply say use apt install docker-compose, but that is clearly not enough.

So do we know what is the minimum version of docker/docker-compose that what is needed?

On the other hand for those systems you need to find a way to install python3.10 also, as the system only comes with python3.9. It would be actually good idea if we could get ankerctl to run on python 3.9 as that would allow much wider install base to run it. I think raspberry pi is one of those things where I do assume lots of people will want to run ankerctl as web server.

kivinen commented 1 year ago

Hmmm. the docker-compose.yaml claims to be version 1 file, because there is no version number specified in the file, i.e., there is no 

version: '2'

or similar line in the beginning of file, but it still seems to use version 2 format, i.e., has the 

services:

and

volumes:

sections.

So the docker-compose.yaml file is clearly incorrect. The docker-compose installed on debian is 1.25.0 which do support version 2.4 docker compose yaml files.

Ah, I see those idiots in docker removed the version line from the docker compose V2, so now the yaml file of docker compose V2 and docker compose V1 version 1 are identical. What are those people smoking when they are defining these things? Isn't there enough to have at least 3 different version numbers for the same thing (docker compose program version, yaml file format version, docker compose high level versioning) enough to cause confusion, that they needed to remove the ability to know the version number from the yaml file?

kivinen commented 1 year ago

I think it would be better to remove the whole docker compose feature as it seems to be completely unusable because of the mess of docker versions. Debian is the 2nd most used distribution of linux, and if there is no easy way of getting it running there, I think there is no point of providing it. Docker is nice on cloud environment, but I hope nobody will be running ankerctl in cloud services, as there is no authentication in the web ui...

Cisien commented 1 year ago

The version of docker and associated tooling included with debian and derivatives is ancient. If you follow the documentation on the docker site, you are instructed to install the docker apt repository for your distro.

In recent versions of docker, compose is integrated into the docker cli directly as docker compose …

My opinion is that all we need to do is document the minimum versions.

spuder commented 1 year ago

Hmmm. the docker-compose.yaml claims to be version 1 file, because there is no version number specified in the file, i.e., there is no

Looks like that was mistakenly removed here:

https://github.com/Ankermgmt/ankermake-m5-protocol/commit/1b77c288b73d13e23b2e9e4392be7758cbb26cbb

chrivers commented 1 year ago

@spuder It was a deliberate choice, but I realize it is a controversial one.

This issue is fraught with confusion, and bizarre choices from docker... I agree with @kivinen, those people are smoking some strong stuff :joy:

Let's take a look, shall we? :smile:

docker docs

First, let's start here:

https://docs.docker.com/compose/compose-file/compose-versioning/

This is Docker's own documentation site, on the section of Compose Versioning.

It says:

The latest and recommended version of Compose is the Compose Specification (https://docs.docker.com/compose/compose-file/)

Alrighty then. A specification. Good stuff. Let's follow that link...

docker compose spec

...and read up the version field:

https://github.com/compose-spec/compose-spec/blob/master/spec.md#version-top-level-element

Version top-level element

Top-level version property is defined by the specification for backward compatibility but is only informative.

A Compose implementation SHOULD NOT use this version to select an exact schema to validate the Compose file, but prefer the most recent schema at the time it has been designed.

Okay, so the version field is not required. But not only that, it is in fact deprecated now, it seems...

docker docs, again

...and this is confirmed on the current specification of the docker compose format, from docker.com:

https://docs.docker.com/compose/compose-file/03-compose-file/

I'll quote the first paragraph:

The Compose file is a YAML file defining version (DEPRECATED), services (REQUIRED), networks, volumes, configs and secrets.

Once again, "version" is referred to as "deprecated" (capitalized, no less).

Versions of compose

Here are some of the facts, as I understand them. Please, feel free to correct any misunderstandings on my part:

Available software

As mentioned earlier in this thread, Debian (and derivatives) are extremely popular.

At the moment, my machine is running Debian Testing (debian 12, "bookwork"). It's expected to release around June, so it's going to become the next stable version soon.

Debian 12 comes with docker 20.10.2, which is a couple of years old at this point.

However, it also comes with docker-compose. I realize this is also a bit old at this point, but it's not exactly broken.

And the main point is, it's one apt install away, for anyone out there using a debian derivative, without extra steps.

Software compatibility

Here's the kicker.

The docker-compose program (as opposed to the docker compose plugin), refuses to run if there's a version: field in the docker-compose file.

However, it does support v2 features just fine (at least all the features we use).

Inching towards a conclusion.

The whole state of affairs with docker and compose is beyond comedy. Docker really dropped the plot on this one...

So where does that leave us?

Well, let's consider a few key points:

Surprisingly, this leaves us in situation where it is easier for end users to use docker-compose, even though docker compose is the modern alternative.

Conclusion

We can absolutely agree that we need to have clear and consistent documentation.

To get to that point, we need to decide on these points:

This is a complicated issue. I'm looking forward to hearing what you all think about my summary of the facts as we know them, and any thoughts on how we should proceed.

kivinen commented 1 year ago

I think we should document the minimum version number of the docker (i.e., output of docker --version), and say that on some operating systems like debian/raspbian you cannot use the docker provided by the system, but you must follow docker installation instructions given in docker site (and provide link).

Does anybody have ubuntu installed, does that include new enough version of docker in their latest stable release, or do they also need to go and install docker from docker site?

anselor commented 6 months ago

I know this thread is nearly a year old at this point but thought I'd chime in. I'm running Ubuntu 22.04 and the way to install docker is by adding the docker apt repository and installing from there.

https://docs.docker.com/engine/install/ubuntu/