installing wis2node on a raspberry pi

petersilva commented 2 years ago

User story

Not sure the best way to document this... more of a war story than a user story. will see how it goes. I want to install wis2node stack on a raspberry pi. installed on "Raspberry Pi 4 Model B Rev 1.4" with the new 64-bit Debian Bullseye image. ( https://downloads.raspberrypi.org/raspios_lite_arm64/images/raspios_lite_arm64-2022-01-28/2022-01-28-raspios-bullseye-arm64-lite.zip )

Acceptance criteria

[ ] acceptance criteria 1

functionality of wis2node should work, likely with no change from installation on linux amd64 hosts.

[ ] acceptance criteria 2
One should be able to follow the README.md instructions to get to a working state.

Definition of done

[ ] unit/functional tests added/updated
[ ] documentation updated
[ ] live demonstration provided

petersilva commented 2 years ago

installed raspberry pi OS image normally, configured SSH access. looked at README.md, began. Performed following without instruction::

apt install git docker docker-compose python3-venv
I ran python -V and noticed default was 3.9... so omitting 3 in python3 instructions.
"edit/adjust accordingly of dev.env" ... unclear... what needs to be adjusted? I did:
- mkdir wis2node-data # to match what the default value in dev.env says...

then the fun:

pi@cwnp:~/wis2node/wis2node $ python3 wis2node-ctl.py config
WARNING: The WIS2NODE_PIP3_EXTRA_PACKAGES variable is not set. Defaulting to a blank string.
ERROR: Named volume "$WIS2NODE_DATADIR/data/incoming:/data/wis2node/data/incoming:rw" is used in service "data-consumer" but no declaration was found in the volumes section.

that is that weird thing I noticed on my PC where WIS2NODE_DATADIR does not get replaced... changed METPX_SR3_DATADIR_INCOMING to WIS2NODE_DATADIR_INCOMING... in data-consumer stanza.

pi@cwnp:~/wis2node/wis2node $ python3 wis2node-ctl.py config
WARNING: The WIS2NODE_PIP3_EXTRA_PACKAGES variable is not set. Defaulting to a blank string.
ERROR: Named volume "$WIS2NODE_DATADIR/data/config:/data/wis2node/data/config:rw" is used in service "wis2node" but no declaration was found in the volumes section.

I don't see how to fix this yet...

david-i-berry commented 2 years ago

I have an issue on Windows (using PowerShell) where the PWD (first line of dev.env) isn't properly set from python, as a result the mounted volumes do not exist (although I don't get the above error). See MWE below:

    import os
    os.getenv("PWD")

Other environment variables exist, it is just PWD that is not set (or inherited from calling shell).

petersilva commented 2 years ago

@david-i-berry I replaced ${PWD} by . and ${WIS2NODE_DATADIR} in dev.env by the actual path to the data directory in the enclosing host ( in my case: ./wis2node-data ) and then config worked.

david-i-berry commented 2 years ago

It turns out that PWD isn't set on Windows. To make the config / setup OS agnostic I think the first line of dev.env should define the value of a variable to point at the working directory and to replace the use of PWD with that throughout. This could be one of the user edits to the environment file and something that need to be highlighted in the documentation.

  WD=your_path_here
  WIS2NODE_DATADIR=${WD}/tests
  WIS2NODE_DATADIR_CONFIG=${WIS2NODE_DATADIR}/data/config
  WIS2NODE_DATADIR_INCOMING=${WIS2NODE_DATADIR}/data/incoming
  WIS2NODE_DATADIR_OUTGOING=${WIS2NODE_DATADIR}/data/outgoing
  WIS2NODE_DATADIR_PUBLIC=${WIS2NODE_DATADIR}/data/public

  WIS2NODE_OSCAR_API_TOKEN=some_token
  WIS2NODE_URL=http://localhost/
  # WIS2NODE_PIP3_EXTRA_PACKAGES="elasticsearch<8"
  WIS2NODE_MQP_URL=mqtt://localhost:1883
  WIS2NODE_API_TYPE=pygeoapi
  WIS2NODE_API_URL=http://localhost:8999/pygeoapi
  WIS2NODE_API_CONFIG=${WD}/docker/pygeoapi/pygeoapi-config.yml
  WIS2NODE_API_BACKEND_TYPE=Elasticsearch
  WIS2NODE_API_BACKEND_HOST=elasticsearch
  WIS2NODE_API_BACKEND_PORT=9200
  WIS2NODE_API_BACKEND_USERNAME=wis2node
  WIS2NODE_API_BACKEND_PASSWORD=wis2node
  WIS2NODE_LOGGING_LOGLEVEL=DEBUG
  WIS2NODE_LOGGING_LOGFILE=stdout
  METPX_SR3_HOST=mosquitto
  METPX_SR3_BROKER_USERNAME=wis2node
  METPX_SR3_BROKER_PASSWORD=wis2node
  METPX_SR3_EXCHANGE=xs_wis2node_acquisition

  # optionally override default mappings
  # WIS2NODE_DATADIR_DATA_MAPPINGS=${WD}/wis2node-data-mappings.yml

where your_path_here can either be a path or the ${PWD} as appropriate.

petersilva commented 2 years ago

fwiw ... @david-i-berry on ubuntu 20.04, and debian 11, the ${WIS2NODE_DATADIR} don't get evaluated either... so I had to replace them by literal values.

tomkralidis commented 2 years ago

(moving from #126)

User story

As a wis2box user, I want wis2box images to offer at least amd64 and arm64 architectures (or more)so that I can run wis2box on an ARM computer, like a raspberry pi. This is true for pygeoapi, wis2box-ui, and data-consumer/sarracenia. @petersilva has added images tags with different architectures for sr3. I think that different architectures can be offered under the same tag using QEMU and docker buildx with Github Actions.

Acceptance criteria

[ ] all image wis2box uses offer multiple architecture support under one tag
[ ] multiple architecture support included in ghcr build action

Definition of done

[ ] unit/functional tests added/updated
[ ] documentation updated
[ ] live demonstration provided

tomkralidis commented 2 years ago

From @webb-ben (moving from #126)

As a wis2box user, I want wis2box images to offer at least amd64 and arm64 architectures (or more)so that I can run wis2box on an ARM computer, like a raspberry pi. This is true for pygeoapi, wis2box-ui, and data-consumer/sarracenia. @petersilva has added images tags with different architectures for sr3. I think that different architectures can be offered under the same tag using QEMU and docker buildx with Github Actions.

wmo-im / wis2box