Open rrobinet opened 3 years ago
ithinkido
commented
3 years ago There are some issue with the new IOT stack If you need to use the old menu, run these commands in the meantime
cd ~/IOTstack
git pull origin master
git checkout old-menu
./menu.sh
Slyke
commented
3 years ago Hey @rrobinet it appears there's a few different issues. First of all, it appears that the docker group isn't being created. I'll need to investigate this. I usually test with a fresh instance of Raspberry Pi OS. Can you check if the docker and bluetooth groups exist? You can run sudo cat /etc/group to view all groups on your system. The docker installation script should create this group automatically. You can also try running: sudo bash menu.sh --run-env-setup and exiting once it finishes its thing and see if that fixes it.
Secondly, the weird ASCII characters you see are from not having enough horizontal space to render the menu correctly, or enough vertical space to render the help and issues text.
Also, don't worry about the network warnings. It's due to there being networks existing in the docker-compose file, that are not used by any services you've selected. These warnings will be removed in a future update.
Slyke
commented
3 years ago Adding the user to the docker group is the last thing the installation script does: https://github.com/SensorsIot/IOTstack/blob/master/install.sh#L306 The only 2 reasons I can think of that the user isn't being added is either the script is exiting early, or the group doesn't exist.
rrobinet
commented
3 years ago @Slyke
Yes after point 3. the docker issue solved, however still not solve the installation
Using the "old menu" or starting the installation from (after deleting the IOTstack directory)
git clone https://github.com/gcgarner/IOTstack.git ~/IOTstack
Allow me to access a displayable menu, however I still have 2 new problems
found 7 vulnerabilities (4 low, 3 moderate)
run `npm audit fix` to fix them, or `npm audit` for detailsnpm audit fix didn't the problem
So for;
point 1. I suppose that the some rubbish is still installed from previous installations, so I will start again with a new Raspberry installation and using the code from gcgarner (which I suppose is the same as the so called old one")
point 2. I suppose that there is an issue with the latest update of node-red, maybe a previous version will help
I will let you know the results Thanks for your support Robert
rrobinet
commented
3 years ago @Slyke @ithinkido
After fresh raspberry installation and as expected everything works file with the version
git clone https://github.com/gcgarner/IOTstack.git ~/IOTstack
Endless message regarding to reboot the system while installing docker seems to be normal, when docker is already installed :-$ (no warning that it is already installed)
mariadb, mosquitto and openhab are installed without problems
node-red still generates vulnerability issues that can't be solved with npm audit fix, but no endless loop anymore and the installation is successful, so nothing to do with IOTstack.
So I believe there is a serious bug in the "?new" SenorsIot version.
At least I can continue my installation now.
Regards
Robert
Slyke
commented
3 years ago @rrobinet have a potential fix for the groups issue on the experimental branch. I want to test it out on a fresh RPi before merging to master in case it breaks anything.
Yeah, the nodered vulnerabilities are from inside nodered, or some installed dependency. The script installs the latest version of each selected dependency that's publish to npm. We need the authors to update their dependencies before it'll remove those warnings on our end.
itsslamdunx
commented
3 years ago Somewhat similarly, the new menu writes a docker-compose.yml that throws an error if you choose prometheus. It leaves in a blank "depends-on:" which previously would have been populated by
git checkout new-menu.itsslamdunx
commented
3 years ago Also I see that docker-compose.yml has the following added to most services.
logging:
options:
max-size: "5m"
max-file: "3"This needs to be removed for grafana, mariadb to be installed. For home_assistant the value for ports: must be removed because it conflicts with network_mode: host.
Slyke
commented
3 years ago Looks like prometheus' template file had nothing there for depends_on, so it was removed. Will have this pushed in an hour or so.
Slyke
commented
3 years ago @itsslamdunx I've been installing grafana and mariadb with logging enabled. Do you have journald logging enabled? I recently updated the Getting Started guide on how to fix this.
The ports were intentionally left in, even though network mode is host. The reason is that there is no way for the menu to know which port the service would bind to, and so it can't determine if there'll be a conflict or not.
Slyke
commented
3 years ago Another thing is that prometheus has not been ported to the new menu system, since it was recently added. At the moment, it will just inject the service template file into docker compose. It will not create any volumes or service directories, since it doesn't have a python build.py file. This goes for dashmachine, heimdall and homer too.
AlbertoGnz
commented
3 years ago Same issue here. Im not able to install it on a fresh RPi. Have you get a chance to test the experimental branch and push the changes to the main branch?
rrobinet
commented
3 years ago Well I started all over and over again and here are my findings
Automatic installation: curl -fsSL https://raw.githubusercontent.com/SensorsIot/IOTstack/master/install.sh | bash
a. Menu claims that Python is not at level; however installed version is 3.7.3 which is above 3.6.9
$python3 --version
Python 3.7.3b. After pseudo upgrade and reboot it looks OK beside a funny menu that doesn't fit a terminal window (see my first dump) c. For info: Group looks OK
$ groups
pi adm dialout cdrom sudo audio video plugdev games users input netdev bluetooth lpadmin **docker** gpio i2c spi
$ sudo groups
root lpadminManual installation: git clone https://github.com/SensorsIot/IOTstack.git IOTstack
a. As above menu claims that Python is not at level however installed version is 3.7.3 which is above 3.6.9
$python3 --version
Python 3.7.3b. After pseudo upgrade, Running ./menu.sh; always error message on Docker group and menu says Docker not installed
c. For info Group
$ groups
pi adm dialout cdrom sudo audio video plugdev games users input netdev bluetooth lpadmin gpio i2c spi
$ sudo groups
root lpadmin **docker**
$ docker version -f "{{.Server.Version}}"
Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Get http://%2Fvar%2Frun%2Fdocker.sock/ v1.24/version: dial unix /var/run/docker.sock: connect: permission deniedd. So because docker is in the super user group it need to be ran as `sudo ./menu.sh` this looks now OK beside a funny menu that doesn't fit a terminal window as above
3. My conclusions
IOTstack is a nice start, if the current funny stuff are/will be bypassed, however if other applications (e.g.: samba, minidlna, ...) not included to the stack menu need to be added manually to the docker-compose file, menu doesn't allow to append new features other than the ones selected and the docker-compose file is overwritten it (may be a tip for an newer version )
4. Finally, to keep it nice and clean after the current standard installation is; do: $ git pull origin master
$ git checkout old-menu
and then from the menu do an "Update IOTstak"
Gives a nice menu for which I can't see the difference with the funny one and is a good basis for future docker add-ons, but to be done interactively. 
Paraphraser
commented
3 years ago With all possible respect given to Slyke's work with the new menu, I'm afraid that I completely agree.
There's a lot of chatter on Discord (a bit less in Issues), which mostly winds up disabling or overriding things done by the new menu. Personally, I find the changes to networking and logging particularly troubling. They are the root cause of many problems. I turned all that off in my own experimentation before concluding that the new menu was not where I wanted to be - yet.
If you want a quiet life, old-menu is definitely the way to go.
I've documented how I install IOTstack in a gist. To summarise:
$ sudo apt install -y git curl
$ git clone -b old-menu https://github.com/SensorsIot/IOTstack.git ~/IOTstack
$ curl -fsSL https://get.docker.com | sh
$ sudo usermod -G docker -a $USER
$ sudo usermod -G bluetooth -a $USER
$ sudo pip3 install docker-compose
$ sudo rebootIn words:
usermod commands stick).Once you've done that, you can either restore a backup and bring up your stack or run the menu and build your stack.
I also recommend treating the menu (old or new) as a one-shot. Once you've set up your stack, learn how it works and maintain it from the command line. Using the menu (old or new) to maintain your stack is a fool's errand. Forget the compose-override file. Just edit docker-compose.yml as and when you need to and get on with your life.
The gist explains also how to uninstall docker and docker-compose. It's quicker than starting over with a clean Raspbian image.
Hi, Few days ago I did install IOTstack with no problem. To day I did start the installation from a clean raspberry installation, and tons of messages are generated while starting the menu, no readable menu is displayed and no image installation is successful
Server: Docker Engine - Community Engine: Version: 20.10.1 API version: 1.41 (minimum version 1.12) Go version: go1.13.15 Git commit: f001486 Built: Tue Dec 15 04:33:02 2020 OS/Arch: linux/arm Experimental: false containerd: Version: 1.4.3 GitCommit: 269548fa27e0089a8b8278fc4fc781d7f65a939b runc: Version: 1.0.0-rc92 GitCommit: ff819c7e9184c13b7c2607fe6c30ae19403a7aff docker-init: Version: 0.19.0 GitCommit: de40ad0 If you would like to use Docker as a non-root user, you should now consider adding your user to the "docker" group with something like:
sudo usermod -aG docker your-user
Remember that you will have to log out and back in for this to take effect!
WARNING: Adding a user to the "docker" group will grant the ability to run containers which can be used to obtain root privileges on the docker host. Refer to https://docs.docker.com/engine/security/security/#docker-daemon-attack-surface for more information. Install docker-compose Reading package lists... Done Building dependency tree
Reading state information... Done The following additional packages will be installed: cgroupfs-mount golang-docker-credential-helpers python3-cached-property python3-docker python3-dockerpty python3-dockerpycreds python3-docopt python3-jsonschema python3-texttable python3-websocket python3-yaml Suggested packages: python-jsonschema-doc Recommended packages: docker.io The following NEW packages will be installed: cgroupfs-mount docker-compose golang-docker-credential-helpers python3-cached-property python3-docker python3-dockerpty python3-dockerpycreds python3-docopt python3-jsonschema python3-texttable python3-websocket python3-yaml 0 upgraded, 12 newly installed, 0 to remove and 0 not upgraded. Need to get 915 kB of archives. After this operation, 3,774 kB of additional disk space will be used. Get:2 http://ftp.acc.umu.se/mirror/raspbian/raspbian buster/main armhf python3-cached-property all 1.5.1-3 [12.1 kB] Get:3 http://ftp.acc.umu.se/mirror/raspbian/raspbian buster/main armhf golang-docker-credential-helpers armhf 0.6.1-2 [482 kB] Get:1 http://mirror.truenetwork.ru/raspbian/raspbian buster/main armhf cgroupfs-mount all 1.4 [6,276 B]
Get:4 http://mirror.serverius.net/raspbian/raspbian buster/main armhf python3-dockerpycreds all 0.3.0-1 [5,204 B]
Get:5 http://ftp.halifax.rwth-aachen.de/raspbian/raspbian buster/main armhf python3-websocket all 0.53.0-1 [36.6 kB] Get:6 http://ftp.halifax.rwth-aachen.de/raspbian/raspbian buster/main armhf python3-docker all 3.4.1-4 [77.8 kB] Get:7 http://ftp.halifax.rwth-aachen.de/raspbian/raspbian buster/main armhf python3-dockerpty all 0.4.1-1 [10.7 kB] Get:10 http://ftp.halifax.rwth-aachen.de/raspbian/raspbian buster/main armhf python3-texttable all 1.6.0-1 [10.9 kB] Get:9 http://mirror.nl.leaseweb.net/raspbian/raspbian buster/main armhf python3-jsonschema all 2.6.0-4 [33.5 kB] Get:11 http://mirror.netzwerge.de/raspbian/raspbian buster/main armhf python3-yaml armhf 3.13-2 [111 kB] Get:12 http://mirror.truenetwork.ru/raspbian/raspbian buster/main armhf docker-compose all 1.21.0-3 [108 kB] Get:8 http://ftp.heanet.ie/mirrors/raspbian/raspbian buster/main armhf python3-docopt all 0.6.2-2 [19.9 kB] Fetched 915 kB in 5s (189 kB/s)
Selecting previously unselected package cgroupfs-mount. (Reading database ... 166083 files and directories currently installed.) Preparing to unpack .../00-cgroupfs-mount_1.4_all.deb ... Unpacking cgroupfs-mount (1.4) ... Selecting previously unselected package python3-cached-property. Preparing to unpack .../01-python3-cached-property_1.5.1-3_all.deb ... Unpacking python3-cached-property (1.5.1-3) ... Selecting previously unselected package golang-docker-credential-helpers. Preparing to unpack .../02-golang-docker-credential-helpers_0.6.1-2_armhf.deb ... Unpacking golang-docker-credential-helpers (0.6.1-2) ... Selecting previously unselected package python3-dockerpycreds. Preparing to unpack .../03-python3-dockerpycreds_0.3.0-1_all.deb ... Unpacking python3-dockerpycreds (0.3.0-1) ... Selecting previously unselected package python3-websocket. Preparing to unpack .../04-python3-websocket_0.53.0-1_all.deb ... Unpacking python3-websocket (0.53.0-1) ... Selecting previously unselected package python3-docker. Preparing to unpack .../05-python3-docker_3.4.1-4_all.deb ... Unpacking python3-docker (3.4.1-4) ... Selecting previously unselected package python3-dockerpty. Preparing to unpack .../06-python3-dockerpty_0.4.1-1_all.deb ... Unpacking python3-dockerpty (0.4.1-1) ... Selecting previously unselected package python3-docopt. Preparing to unpack .../07-python3-docopt_0.6.2-2_all.deb ... Unpacking python3-docopt (0.6.2-2) ... Selecting previously unselected package python3-jsonschema. Preparing to unpack .../08-python3-jsonschema_2.6.0-4_all.deb ... Unpacking python3-jsonschema (2.6.0-4) ... Selecting previously unselected package python3-texttable. Preparing to unpack .../09-python3-texttable_1.6.0-1_all.deb ... Unpacking python3-texttable (1.6.0-1) ... Selecting previously unselected package python3-yaml. Preparing to unpack .../10-python3-yaml_3.13-2_armhf.deb ... Unpacking python3-yaml (3.13-2) ... Selecting previously unselected package docker-compose. Preparing to unpack .../11-docker-compose_1.21.0-3_all.deb ... Unpacking docker-compose (1.21.0-3) ... Setting up python3-cached-property (1.5.1-3) ... Setting up python3-texttable (1.6.0-1) ... Setting up python3-docopt (0.6.2-2) ... Setting up golang-docker-credential-helpers (0.6.1-2) ... Setting up python3-dockerpycreds (0.3.0-1) ... Setting up python3-yaml (3.13-2) ... Setting up python3-jsonschema (2.6.0-4) ... update-alternatives: using /usr/bin/python3-jsonschema to provide /usr/bin/jsonschema (jsonschema) in auto mode Setting up cgroupfs-mount (1.4) ... Setting up python3-websocket (0.53.0-1) ... update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode Setting up python3-dockerpty (0.4.1-1) ... Setting up python3-docker (3.4.1-4) ... Setting up docker-compose (1.21.0-3) ... Processing triggers for man-db (2.8.5-2) ... Processing triggers for systemd (241-7~deb10u5+rpi1) ...
Error getting docker version. Received permission denied error.
Try rerunning the menu with: ./menu.sh --run-env-setup
Checking for project update From https://github.com/SensorsIot/IOTstack
Existing installation detected.