search

jhuckaby / Cronicle

A simple, distributed task scheduler and runner with a web based UI.
http://cronicle.net
Other
3.9k stars 390 forks source link

Live Logs connecting to wrong port on workers #426

Open rrusso1982 opened 3 years ago

rrusso1982 commented 3 years ago

Summary

When attempting to view live logs on worker nodes The system attempts to connect to https://:3012. However 3012 is the http port on the worker and not the https port. This causes live logs to not be streamed to the job as it runs. Logs become available after the job has completed.

Steps to reproduce the problem

Run any job in the system and attempt to view the live logs. Instead of logs you will get messages of the type: Log Watcher: Server Connect Error: Error: websocket error (https://10.75.6.150:3012)

Your Setup

Single master with many workers. The master is behind a load balancer. The LB has an SSL certificate installed to present https to the end user. Master and Workers are configured with http only.

Operating system and version?

Amazon Linux 2

Node.js version?

v14.17.1

Cronicle software version?

0.8.62

Are you using a multi-server setup, or just a single server? Multi

Are you using the filesystem as back-end storage, or S3/Couchbase? S3

Can you reproduce the crash consistently? Yes

Log Excerpts

Log Watcher: Connecting to server: https://10.75.6.150:3012... Log Watcher: Server Connect Error: Error: websocket error (https://10.75.6.150:3012) Log Watcher: Server Connect Error: Error: websocket error (https://10.75.6.150:3012)

justinmdudley commented 3 years ago

Yes! I have this same exact issue! Would love to see this resolved! :)

mikeTWC1984 commented 3 years ago

I'm afraid it's not possible in this configuration. Downgrading websockets from https to http is blocked by design. Hopefully in the next version live log would go through the master node (not via direct connection). As a workaround you can rewrite live log function just to use http polling vs websockets, it's actually not that complicated, see my answer from here: https://github.com/jhuckaby/Cronicle/issues/391#issuecomment-822600803

jhuckaby commented 3 years ago

Hopefully in the next version live log would go through the master node (not via direct connection).

It will 😊, and also HTTPS/SSL will be fully supported 100%.

rrusso1982 commented 3 years ago

I have worked to put together this PR that implements live log proxying through the master. Please take a look and let me know what you think. PR# 427

Also much thanks to @mikeTWC1984 for the inspiration and base code that I worked from for this.

muety commented 2 years ago

I'd highly appreciate to have this merged!