Unable to start worker

[lizhanyang505]

Hi

When I configured CSD and parcel, there was a problem in cm web installation. The CDH version is 6.3.0

• echo '2019-12-22 14:54:56: Starting Airflow worker...'
• su -s /bin/bash - airflow -c 'CONF_DIR=/var/run/cloudera-scm-agent/process/213-airflow-AIRFLOW_WORKER exec /opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/bin/airflow-cm.sh worker ' /opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/lib/python2.7/site-packages/airflow/configuration.py:214: FutureWarning: The task_runner setting in [core] has the old default value of u'BashTaskRunner'. This value has been changed to u'StandardTaskRunner' in the running config, but please update your config before Apache Airflow 2.0. FutureWarning, Traceback (most recent call last): File "/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/bin/.airflow", line 32, in args.func(args) File "/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/lib/python2.7/site-packages/airflow/utils/cli.py", line 74, in wrapper return f(*args, kwargs) File "/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/lib/python2.7/site-packages/airflow/bin/cli.py", line 1090, in worker worker.run(options) File "/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/lib/python2.7/site-packages/celery/bin/worker.py", line 257, in run kwargs) File "/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/lib/python2.7/site-packages/celery/worker/worker.py", line 101, in init self.setup_instance(self.prepare_args(kwargs)) File "/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/lib/python2.7/site-packages/celery/worker/worker.py", line 122, in setup_instance self._conninfo = self.app.connection_for_read() File "/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/lib/python2.7/site-packages/celery/app/base.py", line 743, in connection_for_read return self._connection(url or self.conf.broker_read_url, kwargs) File "/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/lib/python2.7/site-packages/celery/app/base.py", line 819, in _connection 'broker_connection_timeout', connect_timeout File "/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/lib/python2.7/site-packages/kombu/connection.py", line 189, in init url_params = parse_url(hostname) File "/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/lib/python2.7/site-packages/kombu/utils/url.py", line 47, in parse_url scheme, host, port, user, password, path, query = _parse_url(url) File "/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/lib/python2.7/site-packages/kombu/utils/url.py", line 81, in url_to_parts parts.port, File "/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/lib/python2.7/site-packages/future/backports/urllib/parse.py", line 162, in port port = int(port, 10) File "/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/lib/python2.7/site-packages/future/types/newint.py", line 76, in new return super(newint, cls).new(cls, val) ValueError: invalid literal for long() with base 10: '' [22/Dec/2019 14:54:59 +0000] 749 MainThread redactor INFO Started launcher: /opt/cloudera/cm-agent/service/csd/csd.sh start_worker [22/Dec/2019 14:54:59 +0000] 749 MainThread redactor ERROR Redaction rules file doesn't exist, not redacting logs. file: redaction-rules.json, directory: /run/cloudera-scm-agent/process/213-airflow-AIRFLOW_WORKER [22/Dec/2019 14:54:59 +0000] 749 MainThread redactor INFO Re-exec watcher: /opt/cloudera/cm-agent/bin/cm proc_watcher 758

Would you like to help me Than you

[pta-]

I have exactly the same problem with 7.1.3. Did you solve it?

Edit: My experience is that you should do one trick while adding Airflow service:

1. Assign Roles and click Continue
2. Open another session and go to Clusters -> Airflow -> Configuration and fill in
   • DB_TYPE
   • DB_HOST
   • DB_PASS
   • DB_PORT
   • CELERY_BROKER_HOST
   • CELERY_BROKER_PORT <- I think that this is the key parameter to fill in
   • CELERY_BROKER_USER
   • CELERY_BROKER_PASS and Save changes
3. Go back to first session and fill in "Review Changes" form

I think that the problem is with empty broker port. I wonder why there is only DB-related settings in "Review Changes" form?