jhuckaby / Cronicle

A simple, distributed task scheduler and runner with a web based UI.
http://cronicle.net
Other
3.78k stars 387 forks source link

Shell plugin FAQ #52

Closed imran-baig-se closed 6 years ago

imran-baig-se commented 6 years ago

Summary

@jhuckaby your app is wonderful and thanks your quicke response

As I am trying setup jobs using shell plugin am running simple shell jobs.

1) Am changing the work directory from default $HOME/ubuntu or /home/ubuntu and unable to run the jobs.

2) I want default simulation for memory usage and CPU usage. for live and past jobs.

Steps to reproduce the problem

Your Setup

Operating system and version?

Node.js version?

latest

Cronicle software version?

Are you using a multi-server setup, or just a single server?

Are you using the filesystem as back-end storage, or S3/Couchbase?

Can you reproduce the crash consistently?

Log Excerpts

image

image

Please advise

jhuckaby commented 6 years ago

Am changing the work directory from default $HOME/ubuntu or /home/ubuntu and unable to run the jobs.

Unfortunately Cronicle expects the current working directory to always be set to its base install location (e.g. /opt/cronicle, unless you did a custom install somewhere else). I don't think it will work correctly if you change the CWD. You can try changing the Shell Plugin's "Executable" property to a fully qualified path, e.g. /opt/cronicle/bin/shell-plugin.js but I have no idea if this will work. Cronicle isn't designed to have its current working directory changed, so there are likely other things that will break.

I want default simulation for memory usage and CPU usage. for live and past jobs.

I'm sorry, I have no idea what you are talking about here. Can you try to rephrase?

imran-baig-se commented 6 years ago

Screen Shot 2018-01-01 at 7.55.22 PM image

I mean here i want to see cpu usage and memory usage status shows up. while running the job or completed the job

===============================================================

One other thing is am makeing my local machine is server and make salve on AWS EC2 instance. It dosnt get connected even if i have open all the traffic for both incoming and out going & secret key is the same. Server logs : [imrans-macbook-pro.local][Error][error][server][Slave connection failed: ip-172-31-47-249: timeout][] [1514821032.2][2018-01-01 19:37:12][imrans-macbook-pro.local][Error][error][server][Slave connection timeout: ip-172-31-47-249][] [1514821052.202][2018-01-01 19:37:32][imrans-macbook-pro.local][Cronicle][debug][6][Reconnecting to slave: ip-172-31-47-249][] [1514821052.202][2018-01-01 19:37:32][imrans-macbook-pro.local][Cronicle][debug][8][Connecting to slave via socket.io: http://172.31.47.249:3012

======================================================= Salve logs : If i try manually to (using local file system) (here immediately connection get closed)

[List could not be loaded: global/servers: Error: Failed to fetch key: global/servers: File not found][] -01-01 15:14:18][ip-172-31-47-249][Cronicle][debug][4][Server not found in cluster -- waiting for a master server to contact us][] [ip-172-31-47-249][Cronicle][debug][2][Startup complete, entering main loop][] 15:19:01][ip-172-31-47-249][WebServer][debug][8][New incoming HTTP connection: c1][{"ip":"::ffff:94.204.87.212","num_conns":1}] 15:19:01][ip-172-31-47-249][WebServer][debug][8][New HTTP request: POST /api/app/check_add_server (::ffff:94.204.87.212)][{"socket":"c1","version":"1.1"}] 15:19:01][ip-172-31-47-249][WebServer][debug][9][Incoming HTTP Headers:][{"content-type":"application/json","accept-encoding":"gzip, deflate","user-agent":"Cronicle 0.7.6","content-length":"130","host":"34.242.190.120:3012","connection":"close"}] 15:19:01][ip-172-31-47-249][WebServer][debug][6][Invoking handler for request: POST /api/app/check_add_server: API][] 15:19:01][ip-172-31-47-249][API][debug][6][Handling API request: POST /api/app/check_add_server][{}] [1514819941.753][2018-01-01 15:19:01][ip-172-31-47-249][API][debug][9][API Params][{"master":"imrans-macbook-pro.local","now":1514819941,"token":"5d59473deaf4824f9c0d9558cd85c06074f06bb50109ab5c20848dc59d6dbb08"}] [2018-01-01 15:19:01][ip-172-31-47-249][API][debug][9][Activating namespaced API handler: app/api_check_add_server for URI: /api/app/check_add_server][] [1514819941.759][2018-01-01 15:19:01][ip-172-31-47-249][WebServer][debug][9][Compressed text output with gzip: 62 bytes down to: 75 bytes][] 15:19:01][ip-172-31-47-249][WebServer][debug][9][Sending HTTP response: 200 OK][{"Content-Type":"application/json","Access-Control-Allow-Origin":"*","Server":"Cronicle 1.0","Content-Length":75,"Content-Encoding":"gzip"}] [1514819941.763][2018-01-01 15:19:01][ip-172-31-47-249][WebServer][debug][9][Response finished writing to socket][] 15:19:01][ip-172-31-47-249][WebServer][debug][9][Request performance metrics:][{"scale":1000,"perf":{"total":13.956,"read":1.582,"process":1.899,"write":8.98},"counters":{"bytes_in":332,"bytes_out":222,"num_requests":1}}] 15:19:01][ip-172-31-47-249][WebServer][debug][9][Closing socket: c1][] 15:19:01][ip-172-31-47-249][WebServer][debug][8][HTTP connection has closed: c1][{"ip":"::ffff:94.204.87.212","total_elapsed":19,"num_requests":1,"bytes_in":332,"bytes_out":222}]

@jhuckaby thanks in advance

jhuckaby commented 6 years ago

I mean here i want to see cpu usage and memory usage status shows up. while running the job or completed the job

CPU / memory stats are only collected every 10 seconds. Your job is completing faster than that, so it doesn't have a chance to grab a sample. Try a longer running job and you will see accurate CPU / mem graphs for it.

One other thing is am makeing my local machine is server and make salve on AWS EC2 instance. It dosnt get connected even if i have open all the traffic for both incoming and out going & secret key is the same.

Cronicle is not designed to work across the public internet. It is only designed to work on a single local LAN. AWS EC2 instances have local IPs inside the AWS datacenter, which you cannot reach from home. You probably need to setup a VPN into your AWS VPC. Or, you may be able to assign it a proper fully-qualified hostname, and use one or both of these configuration options:

server_comm_use_hostnames

Setting this parameter to 1 will force the Cronicle servers to connect to each other using hostnames rather than LAN IP addresses. This is mainly for special situations where your local server IP addresses may change, and you would prefer to rely on DNS instead. The default is 0 (disabled), meaning connect using IP addresses.

web_socket_use_hostnames

Setting this parameter to 1 will force Cronicle's Web UI to connect to the back-end servers using their hostnames rather than IP addresses. This includes both AJAX API calls and Websocket streams. You should only need to enable this in special situations where your users cannot access your servers via their LAN IPs, and you need to proxy them through a hostname (DNS) instead. The default is 0 (disabled), meaning connect using IP addresses.

Good luck!

imran-baig-se commented 6 years ago

@jhuckaby Thanks for the update I tried making one master and 2 slave on EC2 and it doesn't discover the slaves until i manually add it with UI and it using port 3012 as u can see in logs Note: all ports are open & they are in same availability zone. Is there anyway I can force discovery of slave or run a discovery process.

jhuckaby commented 6 years ago

I'm sorry, but server auto-discovery doesn't work in any public clouds like EC2. Amazon blocks all UDP broadcast traffic on the LAN, so Cronicle cannot discover nearby servers. You will have to add servers manually in the UI. Apologies!

imran-baig-se commented 6 years ago

@jhuckaby how to start running Cronicle in normal mode instead of Deamon mode as it is exiting in my container. Screen Shot 2018-01-04 at 8.31.08 PM image

If i run in Debug mode it will work fine and it wont exit. image

This is just a slave server I dont want to run setup or anyting just start. @kozmic @pierreprevoteau @jhuckaby please suggest. am running through docker-compose.yml

Please suggest in running forground

jhuckaby commented 6 years ago

Please do not repeat your question across multiple issues, and please stop using this issue as a place to send all your questions. Closing it.