Issues with installing on Apple Silicon

SonQBChau commented 2 years ago

I've recently gone through the CanDIGv2 Install Guide on my Mac M1. I noted down some issues (and solutions) here:

1) Step1: Install OS Dependencies:

Mac users can get docker desktop and skip this part. I also installed rosetta and used Docker Compose V2 as suggested at the moment.

Optional: these installations are not mentioned but might be needed:
- Install brew
- Install md5sha1sum (brew install md5sha1sum)
- Install PostgreSQL (brew install postgresql)

Step 2: Initialize CanDIGv2 Repo

# 1. initialize repo and submodules
git clone -b develop https://github.com/CanDIG/CanDIGv2.git
cd CanDIGv2
git submodule update --init --recursive

# 2. copy and edit .env with your site's local configuration
cp -i etc/env/example.env .env

Edit the .env VENV_OS=arm64mac (this is Mac M1 naming)

# 3. fetch binaries and initialize candig virtualenv
make bin-all
make init-conda

To activate conda env, do the following:

conda env list
# Copy the whole path that contains `/envs/candig`
conda activate {path_to_folder}/CanDIGv2/bin/miniconda3/envs/candig

Note: The reason we cannot activate it automatically on Mac was described in this post. If conda env is not in the root folder, it won't have a name.

Step 3: Initialize CanDIGv2 (Docker)

Make sure you are in candig virtual environment (activate it in previous step)
```
# initialize docker environment
make init-docker
```

Step 4: Deploy CanDIGv2 Services (Compose)

make compose

Step 5: Update hosts

Run this command in the terminal to get the local IP address (or google if it doesn't work for you) -dig -4 TXT +short o-o.myaddr.l.google.com @ns1.google.com
Add it to your system in /etc/hosts (not CanDIGv2's) by:
```
sudo nano /etc/hosts
```
Add it to the end of the file so it look like this:
```
# Other settings
xx.xx.xx.xx  docker.localhost
```

After this, you should see the CanDIGv2 services running in Docker :tada:

kcranston commented 2 years ago

Thanks for this! Was helpful when I was reviewing #154 by installing on my M1

SonQBChau commented 2 years ago

Update for the make init-authx. If you got this error:

Setting up Keycloak
Number of keycloak containers running:        1
    Starting setup calls to keycloak
Getting keycloak token
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/opt/miniconda3/lib/python3.9/json/__init__.py", line 293, in load
    return loads(fp.read(),
  File "/opt/miniconda3/lib/python3.9/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
  File "/opt/miniconda3/lib/python3.9/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/opt/miniconda3/lib/python3.9/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
make: *** [init-authx] Error 1

The old keycloak image katsu has right now (15.0.0) is not compatible with M1, so we need to upgrade it.

Go to lib/keycloak/docker-compose.yml and replace the - BASE_IMAGE=candig/keycloak:${KEYCLOAK_VERSION} with this:

- BASE_IMAGE=mihaibob/keycloak:18.0.2-legacy

(I found it on StackOverflow, and it worked, but @shaikh-rashid might want to look for an "official" one or build a candig version for us)

I also commented this out, not sure if it is needed:

  #   volumes:
  #   - keycloak-data:/opt/jboss/keycloak/standalone

In the .env, comment out all the WES_OPT+=….

make clean-authx and make init-authx. If you got this error:

22:45:15,749 INFO  [org.wildfly.extension.undertow] (MSC service thread 1-5) WFLYUT0006: Undertow HTTPS listener https listening on 0.0.0.0:8443
Keycloak container started.
    Starting setup calls to keycloak
Getting keycloak token
Traceback (most recent call last):
  File "<string>", line 1, in <module>
KeyError: 'access_token'
make: *** [init-authx] Error 1

Then try to replace all the keycloak passwords in tmp/secrets folder (make docker-secrets if you don't have it) with something simple like thisisasupersecretpassword, basically no special chars.

make clean-authx and make init-authx again. If you got :

>> configuring jwt stuff
Error writing data to auth/jwt/config: Error making API request.

URL: PUT http://127.0.0.1:8200/v1/auth/jwt/config
Code: 400. Errors:

* error checking oidc discovery URL: Get "http://docker.localhost:8080/auth/realms/candig/.well-known/openid-configuration": dial tcp 24.80.13.34:8080: i/o timeout
make: *** [init-authx] Error 2

Go to .env, look for # keycloak serviceand change this line:

KEYCLOAK_PRIVATE_URL=${KEYCLOAK_PRIVATE_PROTO}://${CANDIG_AUTH_DOMAIN}:${KEYCLOAK_CONTAINER_PORT}

to this

KEYCLOAK_PRIVATE_URL=${KEYCLOAK_PRIVATE_PROTO}://host.docker.internal:${KEYCLOAK_CONTAINER_PORT}

Go to lib/opa/docker-compose.yml, change idp KEYCLOAK_PUBLIC_URL to KEYCLOAK_PRIVATE_URL so it looks like this:

idp: "${KEYCLOAK_PRIVATE_URL}/auth/realms/${KEYCLOAK_REALM}"

and

IDP: "${KEYCLOAK_PRIVATE_URL}/auth/realms/${KEYCLOAK_REALM}"

Finally, go to lib/vault/vault_setup.sh and do the same for KEYCLOAK_PUBLIC_URL:

docker exec $vault sh -c "vault write auth/jwt/config oidc_discovery_url=\"${KEYCLOAK_PRIVATE_URL}/auth/realms/candig\" bound_issuer=\"${KEYCLOAK_PUBLIC_URL}/auth/realms/candig\" default_role=\"researcher\""

Try make clean-authx and make init-authx and it should worked 🎉

daisieh commented 2 years ago

There are probably a few pieces in Tyk that require this same private url/public url fix (or maybe it won't work because of idp stuff), but I need to look into it.

daisieh commented 1 year ago

@SonQBChau Does #161 fix this issue?

SonQBChau commented 1 year ago

I haven't tested it yet, was working on other stuffs and my stack is running so I didn't want to break it

daisieh commented 1 year ago

I don't know if there's any testing required; that pull request is your documentation update.

CanDIG / CanDIGv2