Open tanjix opened 1 year ago
I see that some documentation here is missing.
Please try the following steps:
openslides migrations --help
in a second terminal. This shows you the resepctive commands.openslides migrations stats
to see statistics.finalize
or to use migrate
and then, after finishing, run finalize
. This depends on how large your database is and if there is any risk that the process tooks too long. Just try it out.Hello @normanjaeckel
thanks for your advise!
However, all commands that you suggest, end up with the error message I provided in my initial post:
Error: connecting to gRPC server: creating gRPC client connection with grpc.DialContext(): context deadline exceeded
Any suggestions?
Thanks!
The error message indicates that the command line manage tool openslides
is not able to connect to a running OpenSlides instance. So the question is: Do you have a running instance (e. g. with docker-compose up
)? Which host and port? With SSL or not? Maybe you have to use a custom --address
flag.
What can you see in Docker's logs?
If you started OpenSlides using the default way we described in the README than just try to check if everything is running with openslides check-server
.
Hello @normanjaeckel!
Do you have a running instance (e. g. with
docker-compose up
)
Yes, of course!
docker ps
shows:
root@ga:/root/openslides# docker-compose up -d
Starting openslides_postgres_1 ... done
Starting openslides_redis_1 ... done
Starting openslides_media_1 ... done
Starting openslides_datastoreReader_1 ... done
Starting openslides_datastoreWriter_1 ... done
Starting openslides_autoupdate_1 ... done
Starting openslides_icc_1 ... done
Starting openslides_auth_1 ... done
Starting openslides_backendManage_1 ... done
Starting openslides_manage_1 ... done
Starting openslides_backendPresenter_1 ... done
Starting openslides_vote_1 ... done
Starting openslides_backendAction_1 ... done
Starting openslides_client_1 ... done
Starting openslides_proxy_1 ... done
root@ga:~/openslides# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS
NAMES
1febba8c9f29 ghcr.io/openslides/openslides/openslides-proxy:latest "/entrypoint caddy r…" 4 days ago Up 3 seconds 80/tcp, 443/tcp, 2019/tcp, a.b.c.d:443->8000/tcp openslides_proxy_1
f6baa3042116 ghcr.io/openslides/openslides/openslides-client:latest "/docker-entrypoint.…" 4 days ago Up 4 seconds
openslides_client_1
1a8b58dd68c9 ghcr.io/openslides/openslides/openslides-backend:latest "./entrypoint.sh pyt…" 4 days ago Up 5 seconds 9002-9003/tcp
openslides_backendAction_1
a295f8d23539 ghcr.io/openslides/openslides/openslides-backend:latest "./entrypoint.sh pyt…" 4 days ago Up 6 seconds
openslides_backendPresenter_1
5c37a2f1f00d ghcr.io/openslides/openslides/openslides-vote:latest "/vote" 4 days ago Up 6 seconds (health: starting)
openslides_vote_1
80434fd7e64d ghcr.io/openslides/openslides/openslides-manage:latest "/server" 4 days ago Up 6 seconds (health: starting)
openslides_manage_1
22238bc69f3e ghcr.io/openslides/openslides/openslides-backend:latest "./entrypoint.sh pyt…" 4 days ago Up 8 seconds
openslides_backendManage_1
5dec3d67c063 ghcr.io/openslides/openslides/openslides-auth:latest "./entrypoint.sh nod…" 4 days ago Up 7 seconds
openslides_auth_1
439313c671e3 ghcr.io/openslides/openslides/openslides-icc:latest "/icc" 4 days ago Up 7 seconds (health: starting)
openslides_icc_1
5549549d0560 ghcr.io/openslides/openslides/openslides-datastore-reader:latest "./entrypoint.sh /bi…" 4 days ago Up 9 seconds
openslides_datastoreReader_1
6c5c9cde26ca ghcr.io/openslides/openslides/openslides-media:latest "./entrypoint.sh gun…" 4 days ago Up 9 seconds
openslides_media_1
73eb3a1399e1 ghcr.io/openslides/openslides/openslides-datastore-writer:latest "./entrypoint.sh /bi…" 4 days ago Up 9 seconds
openslides_datastoreWriter_1
86f8162bf3fd redis:latest "docker-entrypoint.s…" 4 days ago Up 10 seconds
openslides_redis_1
53a1a9d0405e postgres:11 "docker-entrypoint.s…" 4 days ago Up 10 seconds
openslides_postgres_1
root@ga:~/openslides#
I've replaced my public IP through a.b.c.d on the above output.
Which host and port? With SSL or not?
The host is a public IP address, the port is 443 (SSL enabled)
If you started OpenSlides using the default way we described in the README than just try to check if everything is running with
openslides check-server
.
This fails with the same error message :(
root@ga:~/openslides# ./openslides check-server
Error: connecting to gRPC server: creating gRPC client connection with grpc.DialContext(): context deadline exceeded
root@ga:~/openslides#
Maybe you have to use a custom
--address
flag.
I've not seen a documentation on that nor anything on the help pages. How shall I use that and where?
Thanks!
Ah! I found the --address
thing
root@ga:~/openslides# ./openslides check-server -a a.b.c.d:443
Server is ready.
root@ga:~/openslides#
So, this works fine. I think I can continue from there, as the openslides --migrations migrate -a a.b.c.d:443
also does some stuff now!
Thanks!
Hi again,
migration did now finish:
(...)
Position 1026 from MI 27 to MI 34 ...
Position 1027 from MI 27 to MI 34 ...
Position 1028 from MI 27 to MI 34 ...
Position 1029 from MI 27 to MI 34 ...
Done. Finalizing is still needed.
root@ga:~/openslides# ./openslides migrations finalize -a a.b.c.d:443
Progress:
Finalize migrations.
No migrations to apply, but finalizing is still needed. Current migration index: 34
Cleaning collectionfield helper tables...
Calculate helper tables...
Deleting all migration keyframes...
Swap events and migration_events tables...
Set the new migration index to 34...
Clean up migration data...
root@ga:~/openslides#
However, after migration has been done, I still cannot login to OS. After entering my credentials and clicking 'login', there's just a circle moving but nothin happening any more, even after waiting several minutes.
The console (running just docker-compose up
) shows during my try to login:
datastoreReader_1 | [2022-11-20 14:48:18 +0000] [12] [INFO] FILTER-request: {'collection': 'user', 'filter': {'field': 'id', 'value': 2, 'operator': '='}, 'mapped_fields': ['username', 'password', 'id', 'is_active', 'meta_deleted']}
auth_1 | 2022-11-20T14:48:19.581Z http://my.hostname.com: POST -- /system/auth/login/
datastoreReader_1 | [2022-11-20 14:48:19 +0000] [12] [INFO] FILTER-request: {'collection': 'user', 'filter': {'field': 'username', 'value': 'tanjix', 'operator': '='}, 'mapped_fields': ['username', 'password', 'id', 'is_active', 'meta_deleted']}
auth_1 | 2022-11-20T14:48:19.628Z http://my.hostname.com: POST -- /system/auth/who-am-i/
datastoreReader_1 | [2022-11-20 14:48:19 +0000] [12] [INFO] FILTER-request: {'collection': 'user', 'filter': {'field': 'id', 'value': 2, 'operator': '='}, 'mapped_fields': ['username', 'password', 'id', 'is_active', 'meta_deleted']}
Does that help?
Apart from that, I also am getting tons of messages like:
proxy_1 | {"level":"error","ts":1668955704.653598,"logger":"http.log.error","msg":"dial tcp: lookup autoupdate on 127.0.0.11:53: no such host","request":{"remote_addr":"84.118.48.176:63480","proto":"HTTP/2.0","method":"GET","host":"my.hostname.com","uri":"/system/autoupdate/health","headers":{"Accept-Language":["de,en-US;q=0.7,en;q=0.3"],"Dnt":["1"],"Sec-Fetch-Dest":["empty"],"Te":["trailers"],"Ngsw-Bypass":["true"],"Sec-Fetch-Mode":["cors"],"Sec-Fetch-Site":["same-origin"],"Cookie":["refreshId=bearer%20eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzZXNzaW9uSWQiOiIxODcyOGFiNDdhMGRhMTQ2MjM3ZGVkMmM1OTdhOTBhYSIsInVzZXJJZCI6MiwiaWF0IjoxNjY4OTU1Njk5LCJleHAiOjE2NzE1NDc2OTl9.K-IGpOAaLgqCGQWuDuaz0l4ouc1Gs--P4GN0bWcUaCc"],"User-Agent":["Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:107.0) Gecko/20100101 Firefox/107.0"],"Accept":["*/*"],"Accept-Encoding":["gzip, deflate, br"],"Referer":["https://my.hostname.com/openslides-shared-worker.f761f22998231b77.js"]},"tls":{"resumed":true,"version":772,"cipher_suite":4865,"proto":"h2","proto_mutual":true,"server_name":"my.hostname.com"}},"duration":0.011335852,"status":502,"err_id":"s619j86ru","err_trace":"reverseproxy.statusError (reverseproxy.go:783)"}`
I guess, that doesn't have anything to do with the login issue, but maybe you have any advise on that, as well?
For testing purpose I did a completely new and clean install of OS4 and I encounter the same issue regarding not being able to login. I therefore suspect any other issue but nothing related to my migration / my data.
Any help would be appreciated!
Thanks!
If I - on purpose - provide wrong credentials to log in, it says:
Username or password is incorrect.
But providing the correct credentials results in the above described problem.
Sorry for the late reply. I am not able to test this myself tonight nor am I tomorrow. Wednesday might be better ...
@ostcar Please have a look at the following error message. What can this be ("autoupdate ... no such host")?
Apart from that, I also am getting tons of messages like: proxy_1 | {"level":"error","ts":1668955704.653598,"logger":"http.log.error","msg":"dial tcp: lookup autoupdate on 127.0.0.11:53: no such host","request":{"remote_addr":"84.118.48.176:63480","proto":"HTTP/2.0","method":"GET","host":"my.hostname.com","uri":"/system/autoupdate/health","headers":{"Accept-Language":["de,en-US;q=0.7,en;q=0.3"],"Dnt":["1"],"Sec-Fetch-Dest":["empty"],"Te":["trailers"],"Ngsw-Bypass":["true"],"Sec-Fetch-Mode":["cors"],"Sec-Fetch-Site":["same-origin"],"Cookie":["refreshId=bearer%20eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzZXNzaW9uSWQiOiIxODcyOGFiNDdhMGRhMTQ2MjM3ZGVkMmM1OTdhOTBhYSIsInVzZXJJZCI6MiwiaWF0IjoxNjY4OTU1Njk5LCJleHAiOjE2NzE1NDc2OTl9.K-IGpOAaLgqCGQWuDuaz0l4ouc1Gs--P4GN0bWcUaCc"],"User-Agent":["Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:107.0) Gecko/20100101 Firefox/107.0"],"Accept":["/"],"Accept-Encoding":["gzip, deflate, br"],"Referer":["https://my.hostname.com/openslides-shared-worker.f761f22998231b77.js"]},"tls":{"resumed":true,"version":772,"cipher_suite":4865,"proto":"h2","proto_mutual":true,"server_name":"my.hostname.com"}},"duration":0.011335852,"status":502,"err_id":"s619j86ru","err_trace":"reverseproxy.statusError (reverseproxy.go:783)"}`
I guess, that doesn't have anything to do with the login issue, but maybe you have any advise on that, as well?
@tanjix Please provide your docker-compose.yml so I can check if we have an (undocumented) versioning issue.
@normanjaeckel here it is.
Thanks! docker-compose.txt
The problem is, that the client can not connect to the autoupdate service. Instead of showing an error message, it tries again forever. I created an issue in the client repo, that a better error message is shown in the future.
This is the relevant part of the error message: dial tcp: lookup autoupdate on 127.0.0.11:53: no such host
. It means, that the proxy tries to redirect the connection to the autoupdate serive, but can not find out the IP-Address of the service.
If you google that error message, it seems, that many docker users have that issue. For some, it helps to turn of VPN. I also had a simular error in the past and it helped (for some reason) to change the DNS-Server of my host-maschine to '8.8.8.8'.
Hi @ostcar thanks for advising about the issue!
For some, it helps to turn of VPN
I am not using VPN on that server
to change the DNS-Server of my host-maschine to '8.8.8.8'.
My DNS servers already are on 8.8.8.8 and 8.8.4.4
I also tried adding a line to /etc/hosts
like:
127.0.0.11 autoupdate
None of all these steps helped out.
Thanks!
Since this is a docker issue, I have no idea how to help you.
@normanjaeckel Any update on this? Thanks!
Sorry for the late answer. If this is still relevant: You have an old version of the docker-compose.yml. You should create a new one using the latest version of the openslides
tool. (This way you can enable the postgres password as secret for the autoupdate service, which is required now and was not in the past.)
So, what I've done now:
openslides config .
)openslides migrations finalize -a a.b.c.d:443
This ends up with:
Running migrations.
Datastore is empty, nothing to do.
I assume, there's now an issue with the newly created docker-compose.yml file?
Can you please advise and guide me through the correct steps to get this sorted? Thank you!
@normanjaeckel Any advise please? Thanks for your help!
Sorry.
We recently changed the location of the postgres volume from a path mounted volume to a docker volume (integrated).
I think you have to revert this small thing and use your old postgres volume again.
Compare the new compose file with your old postgres section.
postgres:
image: postgres:11
environment:
<< : *default-environment
POSTGRES_DB: openslides
POSTGRES_USER: openslides
POSTGRES_PASSWORD_FILE: /run/secrets/postgres_password
PGDATA: /var/lib/postgresql/data/pgdata
networks:
- data
user: 0:0
secrets:
- postgres_password
volumes:
- ./db-data:/var/lib/postgresql/data
You will (hopefully) see some differences. Try to use your old postgres section.
I am sorry for this undocumented breaking changes and I will issue this to our team to make it better next time.
@peb-adr Maybe let us discuss this topic when there is some time ...
Hi Guys!
I recently upgraded my OS4 beta (I didn't note down which version) to OS4 RC version (4.0.0-rc-20221101-a23e23d). Now, when I'm trying to login to OS with my account, it just keeps loading and after a while, I get a message at the bottom:
"Fehler: Missing 7 migrations to apply."
I assume, some migration work needs to be done (probably on pgsql?). However, I haven't found where and how to do it.
I tried playing around and found something on the openslides command, that says "migration". Trying to run a:
root@ga:~/openslides# ./openslides migrations migrate
ends with:
Error: connecting to gRPC server: creating gRPC client connection with grpc.DialContext(): context deadline exceeded
Could anyone assist in getting this resolved? Thanks!