search

cloudera-labs / toolkits

Apache License 2.0
13 stars 20 forks source link

Hdp314 discovery tool bug fix #23

Closed yaoluffy closed 1 year ago

yaoluffy commented 1 year ago

Two bugs are fixed in HDP for the discovery tool

  1. Add verify=False for HTTPS request: When HTTPS is enabled, the code enforces to verify, which could be skipped. It could be a warning, but not a block error.
  2. Modify the json parse for namespaces: Fix the JSON parse error

Please confirm.

yaoluffy commented 1 year ago

@wmudge Could you please review this PR? Thanks.

yaoluffy commented 1 year ago
Click here for test details

./discovery_bundle_builder.sh /opt/toolkits/upgrade-toolkit/HDP-Discovery-Tool/.venv/lib/python3.8/site-packages/urllib3/connectionpool.py:1045: InsecureRequestWarning: Unverified HTTPS request is being made to host 'ccycloud-1.shyao-test-1677230213.root.hwx.site'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings warnings.warn( DEBUG - MainThread - utility - 2023-03-24 09:37:45,397 - Api response stored in: /tmp/output/api_diagnostics/cluster_name.json /opt/toolkits/upgrade-toolkit/HDP-Discovery-Tool/.venv/lib/python3.8/site-packages/urllib3/connectionpool.py:1045: InsecureRequestWarning: Unverified HTTPS request is being made to host 'ccycloud-1.shyao-test-1677230213.root.hwx.site'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings warnings.warn( /opt/toolkits/upgrade-toolkit/HDP-Discovery-Tool/.venv/lib/python3.8/site-packages/urllib3/connectionpool.py:1045: InsecureRequestWarning: Unverified HTTPS request is being made to host 'ccycloud-1.shyao-test-1677230213.root.hwx.site'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings warnings.warn( DEBUG - MainThread - mr_workload_extractor - 2023-03-24 09:37:58,204 - Active namenode url:http://ccycloud-2.shyao-test-1677230213.root.hwx.site:50070 DEBUG - MainThread - mr_workload_extractor - 2023-03-24 09:37:58,205 - Downloading files from /mr-history DEBUG - MainThread - mr_workload_extractor - 2023-03-24 09:37:58,205 - Collect kerberos information DEBUG - MainThread - mr_workload_extractor - 2023-03-24 09:37:58,338 - Kerberos is enabled using the requests-kerberos python module!! Do you have a valid kerberos ticket ?? ERROR - MainThread - mr_workload_extractor - 2023-03-24 09:37:58,387 - Issue in downloading /mr-history ERROR - MainThread - mr_workload_extractor - 2023-03-24 09:37:58,387 - No files to download found inside '/mr-history'. INFO - MainThread - mr_workload_extractor - 2023-03-24 09:37:58,387 - Creating tarball at /tmp/output/workload/MAPREDUCE/mr-history.tar.gz from /tmp/output/workload/MAPREDUCE INFO - MainThread - mr_workload_extractor - 2023-03-24 09:37:58,388 - Tarball created at: /tmp/output/workload/MAPREDUCE/mr-history.tar.gz /opt/toolkits/upgrade-toolkit/HDP-Discovery-Tool/.venv/lib/python3.8/site-packages/urllib3/connectionpool.py:1045: InsecureRequestWarning: Unverified HTTPS request is being made to host 'ccycloud-1.shyao-test-1677230213.root.hwx.site'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings warnings.warn( DEBUG - MainThread - spark_workload_extractor - 2023-03-24 09:38:10,468 - {'METRICS_COLLECTOR': 'ccycloud-1.shyao-test-1677230213.root.hwx.site', 'NAMENODE': 'ccycloud-2.shyao-test-1677230213.root.hwx.site', 'HIVE_METASTORE': 'ccycloud-2.shyao-test-1677230213.root.hwx.site'} DEBUG - MainThread - spark_workload_extractor - 2023-03-24 09:38:10,488 - Active namenode url:http://ccycloud-2.shyao-test-1677230213.root.hwx.site:50070 /opt/toolkits/upgrade-toolkit/HDP-Discovery-Tool/.venv/lib/python3.8/site-packages/urllib3/connectionpool.py:1045: InsecureRequestWarning: Unverified HTTPS request is being made to host 'ccycloud-1.shyao-test-1677230213.root.hwx.site'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings warnings.warn( DEBUG - MainThread - tez_workload_extractor - 2023-03-24 09:38:22,821 - {'METRICS_COLLECTOR': 'ccycloud-1.shyao-test-1677230213.root.hwx.site', 'NAMENODE': 'ccycloud-2.shyao-test-1677230213.root.hwx.site', 'HIVE_METASTORE': 'ccycloud-2.shyao-test-1677230213.root.hwx.site'} DEBUG - MainThread - tez_workload_extractor - 2023-03-24 09:38:22,844 - Active namenode url:http://ccycloud-2.shyao-test-1677230213.root.hwx.site:50070 DEBUG - MainThread - tez_workload_extractor - 2023-03-24 09:38:22,844 - Downloading files from /warehouse/tablespace/external/hive/sys.db DEBUG - MainThread - tez_workload_extractor - 2023-03-24 09:38:22,844 - Collect kerberos information DEBUG - MainThread - tez_workload_extractor - 2023-03-24 09:38:23,015 - Kerberos is enabled using the requests-kerberos python module!! Do you have a valid kerberos ticket ?? ERROR - MainThread - tez_workload_extractor - 2023-03-24 09:38:23,221 - Issue with webhdfs downloading /warehouse/tablespace/external/hive/sys.db ERROR - MainThread - tez_workload_extractor - 2023-03-24 09:38:23,221 - No files to download found inside '/warehouse/tablespace/external/hive/sys.db'. INFO - MainThread - tez_workload_extractor - 2023-03-24 09:38:23,221 - Creating tarball at /tmp/output/workload/TEZ/tez-history.tar.gz from /tmp/output/workload/TEZ/done ERROR - MainThread - tez_workload_extractor - 2023-03-24 09:38:23,222 - Issue with taring the directory /tmp/output/workload/TEZ ERROR - MainThread - tez_workload_extractor - 2023-03-24 09:38:23,222 - [Errno 2] No such file or directory: '/tmp/output/workload/TEZ/done' /opt/toolkits/upgrade-toolkit/HDP-Discovery-Tool/.venv/lib/python3.8/site-packages/urllib3/connectionpool.py:1045: InsecureRequestWarning: Unverified HTTPS request is being made to host 'ccycloud-1.shyao-test-1677230213.root.hwx.site'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings warnings.warn( DEBUG - MainThread - utility - 2023-03-24 09:38:23,391 - Api response stored in: /tmp/output/ranger_policies/ranger_policies.json DEBUG - MainThread - utility - 2023-03-24 09:38:23,394 - Api response stored in: /tmp/output/ranger_policies/ranger_policies.json DEBUG - MainThread - ranger_policy_extractor - 2023-03-24 09:38:23,394 - Completed collecting the ranger details /opt/toolkits/upgrade-toolkit/HDP-Discovery-Tool/.venv/lib/python3.8/site-packages/urllib3/connectionpool.py:1045: InsecureRequestWarning: Unverified HTTPS request is being made to host 'ccycloud-1.shyao-test-1677230213.root.hwx.site'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings warnings.warn( DEBUG - ambari_api_thread - ambari_api_extarctor - 2023-03-24 09:38:23,434 - Started collecting ambari details DEBUG - metrics_collector_thread - hdp_metrics_extractor - 2023-03-24 09:38:23,435 - Started Collecting the Metrics DEBUG - hive_ms_thread - hive_metastore_extractor - 2023-03-24 09:38:23,438 - Connecting to hive database on ccycloud-2.shyao-test-1677230213.root.hwx.site DEBUG - ambari_api_thread - utility - 2023-03-24 09:38:23,440 - Api response stored in: /tmp/output/api_diagnostics/ambari_details.json DEBUG - hdfs_report_thread - hdfs_fs_image_extractor - 2023-03-24 09:38:23,442 - Checking if HDFS is deployed on mytestcluster INFO - hdfs_report_thread - utility - 2023-03-24 09:38:23,443 - Running system command: hdfs dfsadmin -fetchImage /tmp/dfs_image_{cluster}_{datetime.utcnow().isoformat()} DEBUG - ambari_api_thread - ambari_api_extarctor - 2023-03-24 09:38:23,443 - Completed collecting the ambari(cm) details DEBUG - ambari_api_thread - ambari_api_extarctor - 2023-03-24 09:38:23,444 - Read host list. DEBUG - ambari_api_thread - ambari_api_extarctor - 2023-03-24 09:38:23,455 - Connecting to URL https://ccycloud-1.shyao-test-1677230213.root.hwx.site:8443/api/v1/clusters/mytestcluster/hosts DEBUG - ambari_api_thread - utility - 2023-03-24 09:38:23,648 - Api response stored in: /tmp/output/api_diagnostics/hosts.json DEBUG - ambari_api_thread - ambari_api_extarctor - 2023-03-24 09:38:23,710 - Connecting to URL https://ccycloud-1.shyao-test-1677230213.root.hwx.site:8443/api/v1/clusters/mytestcluster/hosts/ccycloud-1.shyao-test-1677230213.root.hwx.site DEBUG - metrics_collector_thread - utility - 2023-03-24 09:38:23,876 - Api response stored in: /tmp/output/AMB_METRICS/AMBARI_INFRA_SOLR/INFRA_SOLR/INFRA_SOLR_METRICS.json DEBUG - metrics_collector_thread - utility - 2023-03-24 09:38:23,910 - Api response stored in: /tmp/output/AMB_METRICS/AMBARI_INFRA_SOLR/INFRA_SOLR_CLIENT/INFRA_SOLR_CLIENT_METRICS.json DEBUG - hive_ms_thread - hive_metastore_extractor - 2023-03-24 09:38:24,272 - Executing query: {db_constant['query']} DEBUG - ambari_api_thread - utility - 2023-03-24 09:38:24,467 - Api response stored in: /tmp/output/api_diagnostics/ccycloud-1.shyao-test-1677230213.root.hwx.sitedetails.json DEBUG - ambari_api_thread - ambari_api_extarctor - 2023-03-24 09:38:24,468 - Connecting to URL https://ccycloud-1.shyao-test-1677230213.root.hwx.site:8443/api/v1/clusters/mytestcluster/hosts/ccycloud-2.shyao-test-1677230213.root.hwx.site DEBUG - hive_ms_thread - utility - 2023-03-24 09:38:24,480 - CSV write finished, results at: {output} DEBUG - hive_ms_thread - hive_metastore_extractor - 2023-03-24 09:38:24,480 - Hive Metastore collection finished. DEBUG - ambari_api_thread - utility - 2023-03-24 09:38:24,976 - Api response stored in: /tmp/output/api_diagnostics/ccycloud-2.shyao-test-1677230213.root.hwx.sitedetails.json DEBUG - ambari_api_thread - ambari_api_extarctor - 2023-03-24 09:38:24,976 - Connecting to URL https://ccycloud-1.shyao-test-1677230213.root.hwx.site:8443/api/v1/clusters/mytestcluster/hosts/ccycloud-3.shyao-test-1677230213.root.hwx.site DEBUG - ambari_api_thread - utility - 2023-03-24 09:38:25,356 - Api response stored in: /tmp/output/api_diagnostics/ccycloud-3.shyao-test-1677230213.root.hwx.sitedetails.json DEBUG - ambari_api_thread - ambari_api_extarctor - 2023-03-24 09:38:25,356 - Connecting to URL https://ccycloud-1.shyao-test-1677230213.root.hwx.site:8443/api/v1/clusters/mytestcluster/hosts/ccycloud-4.shyao-test-1677230213.root.hwx.site DEBUG - ambari_api_thread - utility - 2023-03-24 09:38:25,728 - Api response stored in: /tmp/output/api_diagnostics/ccycloud-4.shyao-test-1677230213.root.hwx.sitedetails.json DEBUG - ambari_api_thread - ambari_api_extarctor - 2023-03-24 09:38:25,729 - Connecting to URL https://ccycloud-1.shyao-test-1677230213.root.hwx.site:8443/api/v1/clusters/mytestcluster/hosts/ccycloud-5.shyao-test-1677230213.root.hwx.site INFO - hdfs_report_thread - utility - 2023-03-24 09:38:25,792 - Running system command: hdfs oiv -p Delimited -delimiter "," -i /tmp/dfs_image_{cluster}_{datetime.utcnow().isoformat()} -o /tmp/output/workload/mytestcluster/service/hdfs_fs.csv DEBUG - metrics_collector_thread - utility - 2023-03-24 09:38:26,083 - Api response stored in: /tmp/output/AMB_METRICS/AMBARI_METRICS/METRICS_COLLECTOR/METRICS_COLLECTOR_METRICS.json DEBUG - ambari_api_thread - utility - 2023-03-24 09:38:26,102 - Api response stored in: /tmp/output/api_diagnostics/ccycloud-5.shyao-test-1677230213.root.hwx.sitedetails.json DEBUG - ambari_api_thread - ambari_api_extarctor - 2023-03-24 09:38:26,103 - Connecting to URL https://ccycloud-1.shyao-test-1677230213.root.hwx.site:8443/api/v1/clusters/mytestcluster/hosts/ccycloud-6.shyao-test-1677230213.root.hwx.site DEBUG - metrics_collector_thread - utility - 2023-03-24 09:38:26,109 - Api response stored in: /tmp/output/AMB_METRICS/AMBARI_METRICS/METRICS_GRAFANA/METRICS_GRAFANA_METRICS.json DEBUG - metrics_collector_thread - utility - 2023-03-24 09:38:26,142 - Api response stored in: /tmp/output/AMB_METRICS/AMBARI_METRICS/METRICS_MONITOR/METRICS_MONITOR_METRICS.json DEBUG - metrics_collector_thread - utility - 2023-03-24 09:38:26,184 - Api response stored in: /tmp/output/AMB_METRICS/ATLAS/ATLAS_CLIENT/ATLAS_CLIENT_METRICS.json DEBUG - metrics_collector_thread - utility - 2023-03-24 09:38:26,209 - Api response stored in: /tmp/output/AMB_METRICS/ATLAS/ATLAS_SERVER/ATLAS_SERVER_METRICS.json DEBUG - metrics_collector_thread - utility - 2023-03-24 09:38:26,309 - Api response stored in: /tmp/output/AMB_METRICS/HBASE/HBASE_CLIENT/HBASE_CLIENT_METRICS.json DEBUG - ambari_api_thread - utility - 2023-03-24 09:38:26,453 - Api response stored in: /tmp/output/api_diagnostics/ccycloud-6.shyao-test-1677230213.root.hwx.sitedetails.json DEBUG - ambari_api_thread - ambari_api_extarctor - 2023-03-24 09:38:26,454 - Read service information. DEBUG - ambari_api_thread - ambari_api_extarctor - 2023-03-24 09:38:26,454 - Connecting to URL https://ccycloud-1.shyao-test-1677230213.root.hwx.site:8443/api/v1/clusters/mytestcluster/services Exception in thread hdfs_report_thread: Traceback (most recent call last): File "/usr/local/lib/python3.8/threading.py", line 932, in _bootstrap_inner self.run() File "/usr/local/lib/python3.8/threading.py", line 870, in run self._target(*self._args, **self._kwargs) File "/opt/toolkits/upgrade-toolkit/HDP-Discovery-Tool/mac-discovery-bundle-builder/hdfs_fs_image_extractor.py", line 68, in collect_fs_image_reports self.create_csv_report(hdfs_fs_csv_path) File "/opt/toolkits/upgrade-toolkit/HDP-Discovery-Tool/mac-discovery-bundle-builder/hdfs_fs_image_extractor.py", line 84, in create_csv_report df = pd.read_csv(hdfs_fs_csv_path, on_bad_lines='skip') TypeError: read_csv() got an unexpected keyword argument 'on_bad_lines' DEBUG - ambari_api_thread - utility - 2023-03-24 09:38:26,602 - Api response stored in: /tmp/output/api_diagnostics/services.json DEBUG - ambari_api_thread - ambari_api_extarctor - 2023-03-24 09:38:26,603 - Connecting to URL https://ccycloud-1.shyao-test-1677230213.root.hwx.site:8443/api/v1/clusters/mytestcluster/services/AMBARI_INFRA_SOLR DEBUG - ambari_api_thread - utility - 2023-03-24 09:38:26,632 - Api response stored in: /tmp/output/api_diagnostics/AMBARI_INFRA_SOLR.json DEBUG - ambari_api_thread - ambari_api_extarctor - 2023-03-24 09:38:26,632 - Connecting to URL https://ccycloud-1.shyao-test-1677230213.root.hwx.site:8443/api/v1/clusters/mytestcluster/services/AMBARI_INFRA_SOLR/components/INFRA_SOLR DEBUG - ambari_api_thread - utility - 2023-03-24 09:38:26,660 - Api response stored in: /tmp/output/api_diagnostics/INFRA_SOLR.json DEBUG - ambari_api_thread - ambari_api_extarctor - 2023-03-24 09:38:26,660 - Connecting to URL https://ccycloud-1.shyao-test-1677230213.root.hwx.site:8443/api/v1/clusters/mytestcluster/services/AMBARI_INFRA_SOLR/components/INFRA_SOLR_CLIENT DEBUG - ambari_api_thread - utility - 2023-03-24 09:38:26,689 - Api response stored in: /tmp/output/api_diagnostics/INFRA_SOLR_CLIENT.json DEBUG - ambari_api_thread - ambari_api_extarctor - 2023-03-24 09:38:26,689 - Connecting to URL https://ccycloud-1.shyao-test-1677230213.root.hwx.site:8443/api/v1/clusters/mytestcluster/services/AMBARI_METRICS DEBUG - ambari_api_thread - utility - 2023-03-24 09:38:26,725 - Api response stored in: /tmp/output/api_diagnostics/AMBARI_METRICS.json DEBUG - ambari_api_thread - ambari_api_extarctor - 2023-03-24 09:38:26,725 - Connecting to URL https://ccycloud-1.shyao-test-1677230213.root.hwx.site:8443/api/v1/clusters/mytestcluster/services/AMBARI_METRICS/components/METRICS_COLLECTOR DEBUG - metrics_collector_thread - utility - 2023-03-24 09:38:28,004 - Api response stored in: /tmp/output/AMB_METRICS/HBASE/HBASE_MASTER/HBASE_MASTER_METRICS.json DEBUG - ambari_api_thread - utility - 2023-03-24 09:38:28,679 - Api response stored in: /tmp/output/api_diagnostics/METRICS_COLLECTOR.json DEBUG - ambari_api_thread - ambari_api_extarctor - 2023-03-24 09:38:28,680 - Connecting to URL https://ccycloud-1.shyao-test-1677230213.root.hwx.site:8443/api/v1/clusters/mytestcluster/services/AMBARI_METRICS/components/METRICS_GRAFANA DEBUG - ambari_api_thread - utility - 2023-03-24 09:38:28,704 - Api response stored in: /tmp/output/api_diagnostics/METRICS_GRAFANA.json DEBUG - ambari_api_thread - ambari_api_extarctor - 2023-03-24 09:38:28,704 - Connecting to URL https://ccycloud-1.shyao-test-1677230213.root.hwx.site:8443/api/v1/clusters/mytestcluster/services/AMBARI_METRICS/components/METRICS_MONITOR DEBUG - ambari_api_thread - utility - 2023-03-24 09:38:28,731 - Api response stored in: /tmp/output/api_diagnostics/METRICS_MONITOR.json DEBUG - ambari_api_thread - ambari_api_extarctor - 2023-03-24 09:38:28,731 - Connecting to URL https://ccycloud-1.shyao-test-1677230213.root.hwx.site:8443/api/v1/clusters/mytestcluster/services/ATLAS DEBUG - ambari_api_thread - utility - 2023-03-24 09:38:28,765 - Api response stored in: /tmp/output/api_diagnostics/ATLAS.json DEBUG - ambari_api_thread - ambari_api_extarctor - 2023-03-24 09:38:28,765 - Connecting to URL https://ccycloud-1.shyao-test-1677230213.root.hwx.site:8443/api/v1/clusters/mytestcluster/services/ATLAS/components/ATLAS_CLIENT DEBUG - ambari_api_thread - utility - 2023-03-24 09:38:28,794 - Api response stored in: /tmp/output/api_diagnostics/ATLAS_CLIENT.json DEBUG - ambari_api_thread - ambari_api_extarctor - 2023-03-24 09:38:28,794 - Connecting to URL https://ccycloud-1.shyao-test-1677230213.root.hwx.site:8443/api/v1/clusters/mytestcluster/services/ATLAS/components/ATLAS_SERVER DEBUG - ambari_api_thread - utility - 2023-03-24 09:38:28,818 - Api response stored in: /tmp/output/api_diagnostics/ATLAS_SERVER.json DEBUG - ambari_api_thread - ambari_api_extarctor - 2023-03-24 09:38:28,818 - Connecting to URL https://ccycloud-1.shyao-test-1677230213.root.hwx.site:8443/api/v1/clusters/mytestcluster/services/HBASE DEBUG - ambari_api_thread - utility - 2023-03-24 09:38:28,859 - Api response stored in: /tmp/output/api_diagnostics/HBASE.json DEBUG - ambari_api_thread - ambari_api_extarctor - 2023-03-24 09:38:28,859 - Connecting to URL https://ccycloud-1.shyao-test-1677230213.root.hwx.site:8443/api/v1/clusters/mytestcluster/services/HBASE/components/HBASE_CLIENT DEBUG - ambari_api_thread - utility - 2023-03-24 09:38:28,917 - Api response stored in: /tmp/output/api_diagnostics/HBASE_CLIENT.json DEBUG - ambari_api_thread - ambari_api_extarctor - 2023-03-24 09:38:28,917 - Connecting to URL https://ccycloud-1.shyao-test-1677230213.root.hwx.site:8443/api/v1/clusters/mytestcluster/services/HBASE/components/HBASE_MASTER DEBUG - metrics_collector_thread - utility - 2023-03-24 09:38:30,544 - Api response stored in: /tmp/output/AMB_METRICS/HBASE/HBASE_REGIONSERVER/HBASE_REGIONSERVER_METRICS.json DEBUG - ambari_api_thread - utility - 2023-03-24 09:38:30,669 - Api response stored in: /tmp/output/api_diagnostics/HBASE_MASTER.json DEBUG - ambari_api_thread - ambari_api_extarctor - 2023-03-24 09:38:30,669 - Connecting to URL https://ccycloud-1.shyao-test-1677230213.root.hwx.site:8443/api/v1/clusters/mytestcluster/services/HBASE/components/HBASE_REGIONSERVER DEBUG - metrics_collector_thread - utility - 2023-03-24 09:38:32,473 - Api response stored in: /tmp/output/AMB_METRICS/HDFS/DATANODE/DATANODE_METRICS.json DEBUG - metrics_collector_thread - utility - 2023-03-24 09:38:32,505 - Api response stored in: /tmp/output/AMB_METRICS/HDFS/HDFS_CLIENT/HDFS_CLIENT_METRICS.json DEBUG - metrics_collector_thread - utility - 2023-03-24 09:38:32,924 - Api response stored in: /tmp/output/AMB_METRICS/HDFS/JOURNALNODE/JOURNALNODE_METRICS.json DEBUG - ambari_api_thread - utility - 2023-03-24 09:38:33,274 - Api response stored in: /tmp/output/api_diagnostics/HBASE_REGIONSERVER.json DEBUG - ambari_api_thread - ambari_api_extarctor - 2023-03-24 09:38:33,274 - Connecting to URL https://ccycloud-1.shyao-test-1677230213.root.hwx.site:8443/api/v1/clusters/mytestcluster/services/HDFS DEBUG - ambari_api_thread - utility - 2023-03-24 09:38:33,322 - Api response stored in: /tmp/output/api_diagnostics/HDFS.json DEBUG - ambari_api_thread - ambari_api_extarctor - 2023-03-24 09:38:33,323 - Connecting to URL https://ccycloud-1.shyao-test-1677230213.root.hwx.site:8443/api/v1/clusters/mytestcluster/services/HDFS/components/DATANODE DEBUG - metrics_collector_thread - utility - 2023-03-24 09:38:34,509 - Api response stored in: /tmp/output/AMB_METRICS/HDFS/NAMENODE/NAMENODE_METRICS.json DEBUG - metrics_collector_thread - utility - 2023-03-24 09:38:34,531 - Api response stored in: /tmp/output/AMB_METRICS/HDFS/SECONDARY_NAMENODE/SECONDARY_NAMENODE_METRICS.json DEBUG - metrics_collector_thread - utility - 2023-03-24 09:38:34,562 - Api response stored in: /tmp/output/AMB_METRICS/HDFS/ZKFC/ZKFC_METRICS.json DEBUG - metrics_collector_thread - utility - 2023-03-24 09:38:34,608 - Api response stored in: /tmp/output/AMB_METRICS/HIVE/HIVE_CLIENT/HIVE_CLIENT_METRICS.json DEBUG - metrics_collector_thread - utility - 2023-03-24 09:38:34,631 - Api response stored in: /tmp/output/AMB_METRICS/HIVE/HIVE_METASTORE/HIVE_METASTORE_METRICS.json DEBUG - metrics_collector_thread - utility - 2023-03-24 09:38:34,656 - Api response stored in: /tmp/output/AMB_METRICS/HIVE/HIVE_SERVER/HIVE_SERVER_METRICS.json DEBUG - metrics_collector_thread - utility - 2023-03-24 09:38:34,679 - Api response stored in: /tmp/output/AMB_METRICS/HIVE/MYSQL_SERVER/MYSQL_SERVER_METRICS.json DEBUG - metrics_collector_thread - utility - 2023-03-24 09:38:35,067 - Api response stored in: /tmp/output/AMB_METRICS/KAFKA/KAFKA_BROKER/KAFKA_BROKER_METRICS.json DEBUG - metrics_collector_thread - utility - 2023-03-24 09:38:35,114 - Api response stored in: /tmp/output/AMB_METRICS/KERBEROS/KERBEROS_CLIENT/KERBEROS_CLIENT_METRICS.json DEBUG - metrics_collector_thread - utility - 2023-03-24 09:38:35,155 - Api response stored in: /tmp/output/AMB_METRICS/KNOX/KNOX_GATEWAY/KNOX_GATEWAY_METRICS.json DEBUG - metrics_collector_thread - utility - 2023-03-24 09:38:35,206 - Api response stored in: /tmp/output/AMB_METRICS/MAPREDUCE2/HISTORYSERVER/HISTORYSERVER_METRICS.json DEBUG - metrics_collector_thread - utility - 2023-03-24 09:38:35,234 - Api response stored in: /tmp/output/AMB_METRICS/MAPREDUCE2/MAPREDUCE2_CLIENT/MAPREDUCE2_CLIENT_METRICS.json DEBUG - ambari_api_thread - utility - 2023-03-24 09:38:35,250 - Api response stored in: /tmp/output/api_diagnostics/DATANODE.json DEBUG - ambari_api_thread - ambari_api_extarctor - 2023-03-24 09:38:35,251 - Connecting to URL https://ccycloud-1.shyao-test-1677230213.root.hwx.site:8443/api/v1/clusters/mytestcluster/services/HDFS/components/HDFS_CLIENT DEBUG - ambari_api_thread - utility - 2023-03-24 09:38:35,281 - Api response stored in: /tmp/output/api_diagnostics/HDFS_CLIENT.json DEBUG - metrics_collector_thread - utility - 2023-03-24 09:38:35,281 - Api response stored in: /tmp/output/AMB_METRICS/PIG/PIG/PIG_METRICS.json DEBUG - ambari_api_thread - ambari_api_extarctor - 2023-03-24 09:38:35,282 - Connecting to URL https://ccycloud-1.shyao-test-1677230213.root.hwx.site:8443/api/v1/clusters/mytestcluster/services/HDFS/components/JOURNALNODE DEBUG - metrics_collector_thread - utility - 2023-03-24 09:38:35,323 - Api response stored in: /tmp/output/AMB_METRICS/RANGER/RANGER_ADMIN/RANGER_ADMIN_METRICS.json DEBUG - metrics_collector_thread - utility - 2023-03-24 09:38:35,352 - Api response stored in: /tmp/output/AMB_METRICS/RANGER/RANGER_TAGSYNC/RANGER_TAGSYNC_METRICS.json DEBUG - metrics_collector_thread - utility - 2023-03-24 09:38:35,377 - Api response stored in: /tmp/output/AMB_METRICS/RANGER/RANGER_USERSYNC/RANGER_USERSYNC_METRICS.json DEBUG - metrics_collector_thread - utility - 2023-03-24 09:38:35,426 - Api response stored in: /tmp/output/AMB_METRICS/RANGER_KMS/RANGER_KMS_SERVER/RANGER_KMS_SERVER_METRICS.json DEBUG - metrics_collector_thread - utility - 2023-03-24 09:38:35,479 - Api response stored in: /tmp/output/AMB_METRICS/SPARK2/SPARK2_CLIENT/SPARK2_CLIENT_METRICS.json DEBUG - metrics_collector_thread - utility - 2023-03-24 09:38:35,508 - Api response stored in: /tmp/output/AMB_METRICS/SPARK2/SPARK2_JOBHISTORYSERVER/SPARK2_JOBHISTORYSERVER_METRICS.json DEBUG - metrics_collector_thread - utility - 2023-03-24 09:38:35,559 - Api response stored in: /tmp/output/AMB_METRICS/SQOOP/SQOOP/SQOOP_METRICS.json DEBUG - metrics_collector_thread - utility - 2023-03-24 09:38:35,606 - Api response stored in: /tmp/output/AMB_METRICS/TEZ/TEZ_CLIENT/TEZ_CLIENT_METRICS.json DEBUG - ambari_api_thread - utility - 2023-03-24 09:38:35,673 - Api response stored in: /tmp/output/api_diagnostics/JOURNALNODE.json DEBUG - ambari_api_thread - ambari_api_extarctor - 2023-03-24 09:38:35,674 - Connecting to URL https://ccycloud-1.shyao-test-1677230213.root.hwx.site:8443/api/v1/clusters/mytestcluster/services/HDFS/components/NAMENODE DEBUG - metrics_collector_thread - utility - 2023-03-24 09:38:35,697 - Api response stored in: /tmp/output/AMB_METRICS/YARN/APP_TIMELINE_SERVER/APP_TIMELINE_SERVER_METRICS.json DEBUG - metrics_collector_thread - utility - 2023-03-24 09:38:36,469 - Api response stored in: /tmp/output/AMB_METRICS/YARN/NODEMANAGER/NODEMANAGER_METRICS.json DEBUG - metrics_collector_thread - utility - 2023-03-24 09:38:37,024 - Api response stored in: /tmp/output/AMB_METRICS/YARN/RESOURCEMANAGER/RESOURCEMANAGER_METRICS.json DEBUG - metrics_collector_thread - utility - 2023-03-24 09:38:37,054 - Api response stored in: /tmp/output/AMB_METRICS/YARN/TIMELINE_READER/TIMELINE_READER_METRICS.json DEBUG - metrics_collector_thread - utility - 2023-03-24 09:38:37,097 - Api response stored in: /tmp/output/AMB_METRICS/YARN/YARN_CLIENT/YARN_CLIENT_METRICS.json DEBUG - metrics_collector_thread - utility - 2023-03-24 09:38:37,122 - Api response stored in: /tmp/output/AMB_METRICS/YARN/YARN_REGISTRY_DNS/YARN_REGISTRY_DNS_METRICS.json DEBUG - metrics_collector_thread - utility - 2023-03-24 09:38:37,173 - Api response stored in: /tmp/output/AMB_METRICS/ZOOKEEPER/ZOOKEEPER_CLIENT/ZOOKEEPER_CLIENT_METRICS.json DEBUG - metrics_collector_thread - utility - 2023-03-24 09:38:37,199 - Api response stored in: /tmp/output/AMB_METRICS/ZOOKEEPER/ZOOKEEPER_SERVER/ZOOKEEPER_SERVER_METRICS.json DEBUG - ambari_api_thread - utility - 2023-03-24 09:38:37,372 - Api response stored in: /tmp/output/api_diagnostics/NAMENODE.json DEBUG - ambari_api_thread - ambari_api_extarctor - 2023-03-24 09:38:37,373 - Connecting to URL https://ccycloud-1.shyao-test-1677230213.root.hwx.site:8443/api/v1/clusters/mytestcluster/services/HDFS/components/SECONDARY_NAMENODE DEBUG - ambari_api_thread - utility - 2023-03-24 09:38:37,394 - Api response stored in: /tmp/output/api_diagnostics/SECONDARY_NAMENODE.json DEBUG - ambari_api_thread - ambari_api_extarctor - 2023-03-24 09:38:37,394 - Connecting to URL https://ccycloud-1.shyao-test-1677230213.root.hwx.site:8443/api/v1/clusters/mytestcluster/services/HDFS/components/ZKFC DEBUG - ambari_api_thread - utility - 2023-03-24 09:38:37,422 - Api response stored in: /tmp/output/api_diagnostics/ZKFC.json DEBUG - ambari_api_thread - ambari_api_extarctor - 2023-03-24 09:38:37,422 - Connecting to URL https://ccycloud-1.shyao-test-1677230213.root.hwx.site:8443/api/v1/clusters/mytestcluster/services/HIVE DEBUG - ambari_api_thread - utility - 2023-03-24 09:38:37,469 - Api response stored in: /tmp/output/api_diagnostics/HIVE.json DEBUG - ambari_api_thread - ambari_api_extarctor - 2023-03-24 09:38:37,469 - Connecting to URL https://ccycloud-1.shyao-test-1677230213.root.hwx.site:8443/api/v1/clusters/mytestcluster/services/HIVE/components/HIVE_CLIENT DEBUG - ambari_api_thread - utility - 2023-03-24 09:38:37,500 - Api response stored in: /tmp/output/api_diagnostics/HIVE_CLIENT.json DEBUG - ambari_api_thread - ambari_api_extarctor - 2023-03-24 09:38:37,500 - Connecting to URL https://ccycloud-1.shyao-test-1677230213.root.hwx.site:8443/api/v1/clusters/mytestcluster/services/HIVE/components/HIVE_METASTORE DEBUG - ambari_api_thread - utility - 2023-03-24 09:38:37,523 - Api response stored in: /tmp/output/api_diagnostics/HIVE_METASTORE.json DEBUG - ambari_api_thread - ambari_api_extarctor - 2023-03-24 09:38:37,523 - Connecting to URL https://ccycloud-1.shyao-test-1677230213.root.hwx.site:8443/api/v1/clusters/mytestcluster/services/HIVE/components/HIVE_SERVER DEBUG - ambari_api_thread - utility - 2023-03-24 09:38:37,554 - Api response stored in: /tmp/output/api_diagnostics/HIVE_SERVER.json DEBUG - ambari_api_thread - ambari_api_extarctor - 2023-03-24 09:38:37,554 - Connecting to URL https://ccycloud-1.shyao-test-1677230213.root.hwx.site:8443/api/v1/clusters/mytestcluster/services/HIVE/components/MYSQL_SERVER DEBUG - ambari_api_thread - utility - 2023-03-24 09:38:37,581 - Api response stored in: /tmp/output/api_diagnostics/MYSQL_SERVER.json DEBUG - ambari_api_thread - ambari_api_extarctor - 2023-03-24 09:38:37,581 - Connecting to URL https://ccycloud-1.shyao-test-1677230213.root.hwx.site:8443/api/v1/clusters/mytestcluster/services/KAFKA DEBUG - ambari_api_thread - utility - 2023-03-24 09:38:37,612 - Api response stored in: /tmp/output/api_diagnostics/KAFKA.json DEBUG - ambari_api_thread - ambari_api_extarctor - 2023-03-24 09:38:37,612 - Connecting to URL https://ccycloud-1.shyao-test-1677230213.root.hwx.site:8443/api/v1/clusters/mytestcluster/services/KAFKA/components/KAFKA_BROKER DEBUG - ambari_api_thread - utility - 2023-03-24 09:38:37,944 - Api response stored in: /tmp/output/api_diagnostics/KAFKA_BROKER.json DEBUG - ambari_api_thread - ambari_api_extarctor - 2023-03-24 09:38:37,945 - Connecting to URL https://ccycloud-1.shyao-test-1677230213.root.hwx.site:8443/api/v1/clusters/mytestcluster/services/KERBEROS DEBUG - ambari_api_thread - utility - 2023-03-24 09:38:37,975 - Api response stored in: /tmp/output/api_diagnostics/KERBEROS.json DEBUG - ambari_api_thread - ambari_api_extarctor - 2023-03-24 09:38:37,976 - Connecting to URL https://ccycloud-1.shyao-test-1677230213.root.hwx.site:8443/api/v1/clusters/mytestcluster/services/KERBEROS/components/KERBEROS_CLIENT DEBUG - ambari_api_thread - utility - 2023-03-24 09:38:38,000 - Api response stored in: /tmp/output/api_diagnostics/KERBEROS_CLIENT.json DEBUG - ambari_api_thread - ambari_api_extarctor - 2023-03-24 09:38:38,000 - Connecting to URL https://ccycloud-1.shyao-test-1677230213.root.hwx.site:8443/api/v1/clusters/mytestcluster/services/KNOX DEBUG - ambari_api_thread - utility - 2023-03-24 09:38:38,032 - Api response stored in: /tmp/output/api_diagnostics/KNOX.json DEBUG - ambari_api_thread - ambari_api_extarctor - 2023-03-24 09:38:38,032 - Connecting to URL https://ccycloud-1.shyao-test-1677230213.root.hwx.site:8443/api/v1/clusters/mytestcluster/services/KNOX/components/KNOX_GATEWAY DEBUG - ambari_api_thread - utility - 2023-03-24 09:38:38,056 - Api response stored in: /tmp/output/api_diagnostics/KNOX_GATEWAY.json DEBUG - ambari_api_thread - ambari_api_extarctor - 2023-03-24 09:38:38,056 - Connecting to URL https://ccycloud-1.shyao-test-1677230213.root.hwx.site:8443/api/v1/clusters/mytestcluster/services/MAPREDUCE2 DEBUG - ambari_api_thread - utility - 2023-03-24 09:38:38,093 - Api response stored in: /tmp/output/api_diagnostics/MAPREDUCE2.json DEBUG - ambari_api_thread - ambari_api_extarctor - 2023-03-24 09:38:38,094 - Connecting to URL https://ccycloud-1.shyao-test-1677230213.root.hwx.site:8443/api/v1/clusters/mytestcluster/services/MAPREDUCE2/components/HISTORYSERVER DEBUG - ambari_api_thread - utility - 2023-03-24 09:38:38,128 - Api response stored in: /tmp/output/api_diagnostics/HISTORYSERVER.json DEBUG - ambari_api_thread - ambari_api_extarctor - 2023-03-24 09:38:38,128 - Connecting to URL https://ccycloud-1.shyao-test-1677230213.root.hwx.site:8443/api/v1/clusters/mytestcluster/services/MAPREDUCE2/components/MAPREDUCE2_CLIENT DEBUG - ambari_api_thread - utility - 2023-03-24 09:38:38,162 - Api response stored in: /tmp/output/api_diagnostics/MAPREDUCE2_CLIENT.json DEBUG - ambari_api_thread - ambari_api_extarctor - 2023-03-24 09:38:38,163 - Connecting to URL https://ccycloud-1.shyao-test-1677230213.root.hwx.site:8443/api/v1/clusters/mytestcluster/services/PIG DEBUG - ambari_api_thread - utility - 2023-03-24 09:38:38,195 - Api response stored in: /tmp/output/api_diagnostics/PIG.json DEBUG - ambari_api_thread - ambari_api_extarctor - 2023-03-24 09:38:38,196 - Connecting to URL https://ccycloud-1.shyao-test-1677230213.root.hwx.site:8443/api/v1/clusters/mytestcluster/services/PIG/components/PIG DEBUG - ambari_api_thread - utility - 2023-03-24 09:38:38,222 - Api response stored in: /tmp/output/api_diagnostics/PIG.json DEBUG - ambari_api_thread - ambari_api_extarctor - 2023-03-24 09:38:38,223 - Connecting to URL https://ccycloud-1.shyao-test-1677230213.root.hwx.site:8443/api/v1/clusters/mytestcluster/services/RANGER DEBUG - ambari_api_thread - utility - 2023-03-24 09:38:38,253 - Api response stored in: /tmp/output/api_diagnostics/RANGER.json DEBUG - ambari_api_thread - ambari_api_extarctor - 2023-03-24 09:38:38,253 - Connecting to URL https://ccycloud-1.shyao-test-1677230213.root.hwx.site:8443/api/v1/clusters/mytestcluster/services/RANGER/components/RANGER_ADMIN DEBUG - ambari_api_thread - utility - 2023-03-24 09:38:38,277 - Api response stored in: /tmp/output/api_diagnostics/RANGER_ADMIN.json DEBUG - ambari_api_thread - ambari_api_extarctor - 2023-03-24 09:38:38,277 - Connecting to URL https://ccycloud-1.shyao-test-1677230213.root.hwx.site:8443/api/v1/clusters/mytestcluster/services/RANGER/components/RANGER_TAGSYNC DEBUG - ambari_api_thread - utility - 2023-03-24 09:38:38,304 - Api response stored in: /tmp/output/api_diagnostics/RANGER_TAGSYNC.json DEBUG - ambari_api_thread - ambari_api_extarctor - 2023-03-24 09:38:38,304 - Connecting to URL https://ccycloud-1.shyao-test-1677230213.root.hwx.site:8443/api/v1/clusters/mytestcluster/services/RANGER/components/RANGER_USERSYNC DEBUG - ambari_api_thread - utility - 2023-03-24 09:38:38,327 - Api response stored in: /tmp/output/api_diagnostics/RANGER_USERSYNC.json DEBUG - ambari_api_thread - ambari_api_extarctor - 2023-03-24 09:38:38,327 - Connecting to URL https://ccycloud-1.shyao-test-1677230213.root.hwx.site:8443/api/v1/clusters/mytestcluster/services/RANGER_KMS DEBUG - ambari_api_thread - utility - 2023-03-24 09:38:38,360 - Api response stored in: /tmp/output/api_diagnostics/RANGER_KMS.json DEBUG - ambari_api_thread - ambari_api_extarctor - 2023-03-24 09:38:38,360 - Connecting to URL https://ccycloud-1.shyao-test-1677230213.root.hwx.site:8443/api/v1/clusters/mytestcluster/services/RANGER_KMS/components/RANGER_KMS_SERVER DEBUG - ambari_api_thread - utility - 2023-03-24 09:38:38,386 - Api response stored in: /tmp/output/api_diagnostics/RANGER_KMS_SERVER.json DEBUG - ambari_api_thread - ambari_api_extarctor - 2023-03-24 09:38:38,386 - Connecting to URL https://ccycloud-1.shyao-test-1677230213.root.hwx.site:8443/api/v1/clusters/mytestcluster/services/SPARK2 DEBUG - ambari_api_thread - utility - 2023-03-24 09:38:38,427 - Api response stored in: /tmp/output/api_diagnostics/SPARK2.json DEBUG - ambari_api_thread - ambari_api_extarctor - 2023-03-24 09:38:38,427 - Connecting to URL https://ccycloud-1.shyao-test-1677230213.root.hwx.site:8443/api/v1/clusters/mytestcluster/services/SPARK2/components/SPARK2_CLIENT DEBUG - ambari_api_thread - utility - 2023-03-24 09:38:38,457 - Api response stored in: /tmp/output/api_diagnostics/SPARK2_CLIENT.json DEBUG - ambari_api_thread - ambari_api_extarctor - 2023-03-24 09:38:38,457 - Connecting to URL https://ccycloud-1.shyao-test-1677230213.root.hwx.site:8443/api/v1/clusters/mytestcluster/services/SPARK2/components/SPARK2_JOBHISTORYSERVER DEBUG - ambari_api_thread - utility - 2023-03-24 09:38:38,485 - Api response stored in: /tmp/output/api_diagnostics/SPARK2_JOBHISTORYSERVER.json DEBUG - ambari_api_thread - ambari_api_extarctor - 2023-03-24 09:38:38,485 - Connecting to URL https://ccycloud-1.shyao-test-1677230213.root.hwx.site:8443/api/v1/clusters/mytestcluster/services/SQOOP DEBUG - ambari_api_thread - utility - 2023-03-24 09:38:38,531 - Api response stored in: /tmp/output/api_diagnostics/SQOOP.json DEBUG - ambari_api_thread - ambari_api_extarctor - 2023-03-24 09:38:38,532 - Connecting to URL https://ccycloud-1.shyao-test-1677230213.root.hwx.site:8443/api/v1/clusters/mytestcluster/services/SQOOP/components/SQOOP DEBUG - ambari_api_thread - utility - 2023-03-24 09:38:38,558 - Api response stored in: /tmp/output/api_diagnostics/SQOOP.json DEBUG - ambari_api_thread - ambari_api_extarctor - 2023-03-24 09:38:38,558 - Connecting to URL https://ccycloud-1.shyao-test-1677230213.root.hwx.site:8443/api/v1/clusters/mytestcluster/services/TEZ DEBUG - ambari_api_thread - utility - 2023-03-24 09:38:38,591 - Api response stored in: /tmp/output/api_diagnostics/TEZ.json DEBUG - ambari_api_thread - ambari_api_extarctor - 2023-03-24 09:38:38,592 - Connecting to URL https://ccycloud-1.shyao-test-1677230213.root.hwx.site:8443/api/v1/clusters/mytestcluster/services/TEZ/components/TEZ_CLIENT DEBUG - ambari_api_thread - utility - 2023-03-24 09:38:38,621 - Api response stored in: /tmp/output/api_diagnostics/TEZ_CLIENT.json DEBUG - ambari_api_thread - ambari_api_extarctor - 2023-03-24 09:38:38,621 - Connecting to URL https://ccycloud-1.shyao-test-1677230213.root.hwx.site:8443/api/v1/clusters/mytestcluster/services/YARN DEBUG - ambari_api_thread - utility - 2023-03-24 09:38:38,691 - Api response stored in: /tmp/output/api_diagnostics/YARN.json DEBUG - ambari_api_thread - ambari_api_extarctor - 2023-03-24 09:38:38,692 - Connecting to URL https://ccycloud-1.shyao-test-1677230213.root.hwx.site:8443/api/v1/clusters/mytestcluster/services/YARN/components/APP_TIMELINE_SERVER DEBUG - ambari_api_thread - utility - 2023-03-24 09:38:38,735 - Api response stored in: /tmp/output/api_diagnostics/APP_TIMELINE_SERVER.json DEBUG - ambari_api_thread - ambari_api_extarctor - 2023-03-24 09:38:38,735 - Connecting to URL https://ccycloud-1.shyao-test-1677230213.root.hwx.site:8443/api/v1/clusters/mytestcluster/services/YARN/components/NODEMANAGER DEBUG - ambari_api_thread - utility - 2023-03-24 09:38:39,556 - Api response stored in: /tmp/output/api_diagnostics/NODEMANAGER.json DEBUG - ambari_api_thread - ambari_api_extarctor - 2023-03-24 09:38:39,557 - Connecting to URL https://ccycloud-1.shyao-test-1677230213.root.hwx.site:8443/api/v1/clusters/mytestcluster/services/YARN/components/RESOURCEMANAGER DEBUG - ambari_api_thread - utility - 2023-03-24 09:38:40,091 - Api response stored in: /tmp/output/api_diagnostics/RESOURCEMANAGER.json DEBUG - ambari_api_thread - ambari_api_extarctor - 2023-03-24 09:38:40,092 - Connecting to URL https://ccycloud-1.shyao-test-1677230213.root.hwx.site:8443/api/v1/clusters/mytestcluster/services/YARN/components/TIMELINE_READER DEBUG - ambari_api_thread - utility - 2023-03-24 09:38:40,119 - Api response stored in: /tmp/output/api_diagnostics/TIMELINE_READER.json DEBUG - ambari_api_thread - ambari_api_extarctor - 2023-03-24 09:38:40,119 - Connecting to URL https://ccycloud-1.shyao-test-1677230213.root.hwx.site:8443/api/v1/clusters/mytestcluster/services/YARN/components/YARN_CLIENT DEBUG - ambari_api_thread - utility - 2023-03-24 09:38:40,149 - Api response stored in: /tmp/output/api_diagnostics/YARN_CLIENT.json DEBUG - ambari_api_thread - ambari_api_extarctor - 2023-03-24 09:38:40,149 - Connecting to URL https://ccycloud-1.shyao-test-1677230213.root.hwx.site:8443/api/v1/clusters/mytestcluster/services/YARN/components/YARN_REGISTRY_DNS DEBUG - ambari_api_thread - utility - 2023-03-24 09:38:40,177 - Api response stored in: /tmp/output/api_diagnostics/YARN_REGISTRY_DNS.json DEBUG - ambari_api_thread - ambari_api_extarctor - 2023-03-24 09:38:40,177 - Connecting to URL https://ccycloud-1.shyao-test-1677230213.root.hwx.site:8443/api/v1/clusters/mytestcluster/services/ZOOKEEPER DEBUG - ambari_api_thread - utility - 2023-03-24 09:38:40,211 - Api response stored in: /tmp/output/api_diagnostics/ZOOKEEPER.json DEBUG - ambari_api_thread - ambari_api_extarctor - 2023-03-24 09:38:40,212 - Connecting to URL https://ccycloud-1.shyao-test-1677230213.root.hwx.site:8443/api/v1/clusters/mytestcluster/services/ZOOKEEPER/components/ZOOKEEPER_CLIENT DEBUG - ambari_api_thread - utility - 2023-03-24 09:38:40,243 - Api response stored in: /tmp/output/api_diagnostics/ZOOKEEPER_CLIENT.json DEBUG - ambari_api_thread - ambari_api_extarctor - 2023-03-24 09:38:40,243 - Connecting to URL https://ccycloud-1.shyao-test-1677230213.root.hwx.site:8443/api/v1/clusters/mytestcluster/services/ZOOKEEPER/components/ZOOKEEPER_SERVER DEBUG - ambari_api_thread - utility - 2023-03-24 09:38:40,269 - Api response stored in: /tmp/output/api_diagnostics/ZOOKEEPER_SERVER.json DEBUG - ambari_api_thread - ambari_api_extarctor - 2023-03-24 09:38:40,270 - collecting the blueprint of the cluster DEBUG - ambari_api_thread - ambari_api_extarctor - 2023-03-24 09:38:40,270 - Connecting to URL https://ccycloud-1.shyao-test-1677230213.root.hwx.site:8443/api/v1/clusters/mytestcluster?format=blueprint DEBUG - ambari_api_thread - utility - 2023-03-24 09:38:40,749 - Api response stored in: /tmp/output/api_diagnostics/blueprint.json DEBUG - ambari_api_thread - ambari_api_extarctor - 2023-03-24 09:38:40,749 - Collecting the cluster Info DEBUG - ambari_api_thread - ambari_api_extarctor - 2023-03-24 09:38:40,749 - Connecting to URL https://ccycloud-1.shyao-test-1677230213.root.hwx.site:8443/api/v1/clusters/mytestcluster DEBUG - ambari_api_thread - utility - 2023-03-24 09:38:41,551 - Api response stored in: /tmp/output/api_diagnostics/clusters.json DEBUG - ambari_api_thread - ambari_api_extarctor - 2023-03-24 09:38:41,553 - Collect kerberos information DEBUG - ambari_api_thread - ambari_api_extarctor - 2023-03-24 09:38:41,553 - Connecting to URL https://ccycloud-1.shyao-test-1677230213.root.hwx.site:8443/api/v1/clusters/mytestcluster/services DEBUG - ambari_api_thread - ambari_api_extarctor - 2023-03-24 09:38:41,700 - Connecting to URL https://ccycloud-1.shyao-test-1677230213.root.hwx.site:8443/api/v1/clusters/mytestcluster/configurations/service_config_versions?service_name=KERBEROS&is_current=true DEBUG - ambari_api_thread - utility - 2023-03-24 09:38:41,720 - Api response stored in: /tmp/output/api_diagnostics/kerberos.json DEBUG - ambari_api_thread - ambari_api_extarctor - 2023-03-24 09:38:41,720 - Collect kerberos information DEBUG - ambari_api_thread - ambari_api_extarctor - 2023-03-24 09:38:41,720 - Connecting to URL https://ccycloud-1.shyao-test-1677230213.root.hwx.site:8443/api/v1/clusters/mytestcluster/services DEBUG - ambari_api_thread - ambari_api_extarctor - 2023-03-24 09:38:41,919 - Connecting to URL https://ccycloud-1.shyao-test-1677230213.root.hwx.site:8443/api/v1/clusters/mytestcluster/configurations/service_config_versions?service_name=YARN&is_current=true DEBUG - ambari_api_thread - utility - 2023-03-24 09:38:41,948 - Api response stored in: /tmp/output/api_diagnostics/capacity-scheduler.json /opt/toolkits/upgrade-toolkit/HDP-Discovery-Tool/.venv/lib/python3.8/site-packages/urllib3/connectionpool.py:1045: InsecureRequestWarning: Unverified HTTPS request is being made to host 'ccycloud-1.shyao-test-1677230213.root.hwx.site'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings warnings.warn( DEBUG - ambari_api_thread - utility - 2023-03-24 09:38:41,984 - Api response stored in: /tmp/output/api_diagnostics/cluster_name.json DEBUG - ambari_api_thread - ambari_api_extarctor - 2023-03-24 09:38:41,984 - Collecting the componentlist Info DEBUG - ambari_api_thread - ambari_api_extarctor - 2023-03-24 09:38:41,984 - Connecting to URL https://ccycloud-1.shyao-test-1677230213.root.hwx.site:8443/api/v1/clusters/mytestcluster/host_components?HostRoles DEBUG - ambari_api_thread - utility - 2023-03-24 09:38:42,143 - Api response stored in: /tmp/output/api_diagnostics/componentslist.json DEBUG - ambari_api_thread - ambari_api_extarctor - 2023-03-24 09:38:42,143 - Collecting the configuration Info DEBUG - ambari_api_thread - ambari_api_extarctor - 2023-03-24 09:38:42,144 - Connecting to URL https://ccycloud-1.shyao-test-1677230213.root.hwx.site:8443/api/v1/clusters/mytestcluster/configurations/service_config_versions?is_current=true DEBUG - ambari_api_thread - utility - 2023-03-24 09:38:42,240 - Api response stored in: /tmp/output/api_diagnostics/configurations.json DEBUG - metrics_collector_thread - hdp_metrics_extractor - 2023-03-24 09:38:50,714 - {'METRICS_COLLECTOR': 'ccycloud-1.shyao-test-1677230213.root.hwx.site', 'NAMENODE': 'ccycloud-2.shyao-test-1677230213.root.hwx.site', 'HIVE_METASTORE': 'ccycloud-2.shyao-test-1677230213.root.hwx.site'} DEBUG - metrics_collector_thread - hdp_metrics_extractor - 2023-03-24 09:38:50,736 - END TIME STAMP EPOCH:1679650730 DEBUG - metrics_collector_thread - hdp_metrics_extractor - 2023-03-24 09:38:50,737 - START TIME STAMP EPOCH: 1675762730 DEBUG - metrics_collector_thread - hdp_metrics_extractor - 2023-03-24 09:38:50,972 - http://ccycloud-1.shyao-test-1677230213.root.hwx.site:6188/ws/v1/timeline/metrics?metricNames=%&appId=timeline_metric_store_watcher&hostname=%&startTime=1675762730&endTime=1679650730 DEBUG - metrics_collector_thread - utility - 2023-03-24 09:38:51,087 - Api response stored in: /tmp/output/AMS_METRICS/apps.json DEBUG - metrics_collector_thread - utility - 2023-03-24 09:38:51,395 - Api response stored in: /tmp/output/AMS_METRICS/timeline_metric_store_watcher/timeline_metric_store_watcher.json DEBUG - metrics_collector_thread - hdp_metrics_extractor - 2023-03-24 09:38:51,396 - http://ccycloud-1.shyao-test-1677230213.root.hwx.site:6188/ws/v1/timeline/metrics?metricNames=%._max&appId=timeline_metric_store_watcher&hostname=%&startTime=1675762730&endTime=1679650730 DEBUG - metrics_collector_thread - utility - 2023-03-24 09:38:51,834 - Api response stored in: /tmp/output/AMS_METRICS/timeline_metric_store_watcher/timeline_metric_store_watcher_max.json DEBUG - metrics_collector_thread - hdp_metrics_extractor - 2023-03-24 09:38:51,834 - http://ccycloud-1.shyao-test-1677230213.root.hwx.site:6188/ws/v1/timeline/metrics?metricNames=%._min&appId=timeline_metric_store_watcher&hostname=%&startTime=1675762730&endTime=1679650730 DEBUG - metrics_collector_thread - utility - 2023-03-24 09:38:52,119 - Api response stored in: /tmp/output/AMS_METRICS/timeline_metric_store_watcher/timeline_metric_store_watcher_min.json DEBUG - metrics_collector_thread - hdp_metrics_extractor - 2023-03-24 09:38:52,119 - http://ccycloud-1.shyao-test-1677230213.root.hwx.site:6188/ws/v1/timeline/metrics?metricNames=%._avg&appId=timeline_metric_store_watcher&hostname=%&startTime=1675762730&endTime=1679650730 DEBUG - metrics_collector_thread - utility - 2023-03-24 09:38:52,374 - Api response stored in: /tmp/output/AMS_METRICS/timeline_metric_store_watcher/timeline_metric_store_watcher_avg.json DEBUG - metrics_collector_thread - hdp_metrics_extractor - 2023-03-24 09:38:52,375 - http://ccycloud-1.shyao-test-1677230213.root.hwx.site:6188/ws/v1/timeline/metrics?metricNames=%&appId=datanode&hostname=%&startTime=1675762730&endTime=1679650730 DEBUG - metrics_collector_thread - utility - 2023-03-24 09:38:52,486 - Api response stored in: /tmp/output/AMS_METRICS/apps.json DEBUG - metrics_collector_thread - utility - 2023-03-24 09:38:53,157 - Api response stored in: /tmp/output/AMS_METRICS/datanode/datanode.json DEBUG - metrics_collector_thread - hdp_metrics_extractor - 2023-03-24 09:38:53,158 - http://ccycloud-1.shyao-test-1677230213.root.hwx.site:6188/ws/v1/timeline/metrics?metricNames=%._max&appId=datanode&hostname=%&startTime=1675762730&endTime=1679650730 DEBUG - metrics_collector_thread - utility - 2023-03-24 09:38:53,736 - Api response stored in: /tmp/output/AMS_METRICS/datanode/datanode_max.json DEBUG - metrics_collector_thread - hdp_metrics_extractor - 2023-03-24 09:38:53,737 - http://ccycloud-1.shyao-test-1677230213.root.hwx.site:6188/ws/v1/timeline/metrics?metricNames=%._min&appId=datanode&hostname=%&startTime=1675762730&endTime=1679650730 DEBUG - metrics_collector_thread - utility - 2023-03-24 09:38:54,199 - Api response stored in: /tmp/output/AMS_METRICS/datanode/datanode_min.json DEBUG - metrics_collector_thread - hdp_metrics_extractor - 2023-03-24 09:38:54,200 - http://ccycloud-1.shyao-test-1677230213.root.hwx.site:6188/ws/v1/timeline/metrics?metricNames=%._avg&appId=datanode&hostname=%&startTime=1675762730&endTime=1679650730 DEBUG - metrics_collector_thread - utility - 2023-03-24 09:38:54,690 - Api response stored in: /tmp/output/AMS_METRICS/datanode/datanode_avg.json DEBUG - metrics_collector_thread - hdp_metrics_extractor - 2023-03-24 09:38:54,690 - http://ccycloud-1.shyao-test-1677230213.root.hwx.site:6188/ws/v1/timeline/metrics?metricNames=%&appId=HOST&hostname=%&startTime=1675762730&endTime=1679650730 DEBUG - metrics_collector_thread - utility - 2023-03-24 09:38:54,796 - Api response stored in: /tmp/output/AMS_METRICS/apps.json DEBUG - metrics_collector_thread - utility - 2023-03-24 09:38:55,246 - Api response stored in: /tmp/output/AMS_METRICS/HOST/HOST.json DEBUG - metrics_collector_thread - hdp_metrics_extractor - 2023-03-24 09:38:55,247 - http://ccycloud-1.shyao-test-1677230213.root.hwx.site:6188/ws/v1/timeline/metrics?metricNames=%._max&appId=HOST&hostname=%&startTime=1675762730&endTime=1679650730 DEBUG - metrics_collector_thread - utility - 2023-03-24 09:38:55,716 - Api response stored in: /tmp/output/AMS_METRICS/HOST/HOST_max.json DEBUG - metrics_collector_thread - hdp_metrics_extractor - 2023-03-24 09:38:55,717 - http://ccycloud-1.shyao-test-1677230213.root.hwx.site:6188/ws/v1/timeline/metrics?metricNames=%._min&appId=HOST&hostname=%&startTime=1675762730&endTime=1679650730 DEBUG - metrics_collector_thread - utility - 2023-03-24 09:38:56,183 - Api response stored in: /tmp/output/AMS_METRICS/HOST/HOST_min.json DEBUG - metrics_collector_thread - hdp_metrics_extractor - 2023-03-24 09:38:56,183 - http://ccycloud-1.shyao-test-1677230213.root.hwx.site:6188/ws/v1/timeline/metrics?metricNames=%._avg&appId=HOST&hostname=%&startTime=1675762730&endTime=1679650730 DEBUG - metrics_collector_thread - utility - 2023-03-24 09:38:56,702 - Api response stored in: /tmp/output/AMS_METRICS/HOST/HOST_avg.json DEBUG - metrics_collector_thread - hdp_metrics_extractor - 2023-03-24 09:38:56,703 - http://ccycloud-1.shyao-test-1677230213.root.hwx.site:6188/ws/v1/timeline/metrics?metricNames=%&appId=jobhistoryserver&hostname=%&startTime=1675762730&endTime=1679650730 DEBUG - metrics_collector_thread - utility - 2023-03-24 09:38:56,850 - Api response stored in: /tmp/output/AMS_METRICS/apps.json DEBUG - metrics_collector_thread - utility - 2023-03-24 09:38:57,227 - Api response stored in: /tmp/output/AMS_METRICS/jobhistoryserver/jobhistoryserver.json DEBUG - metrics_collector_thread - hdp_metrics_extractor - 2023-03-24 09:38:57,227 - http://ccycloud-1.shyao-test-1677230213.root.hwx.site:6188/ws/v1/timeline/metrics?metricNames=%._max&appId=jobhistoryserver&hostname=%&startTime=1675762730&endTime=1679650730 DEBUG - metrics_collector_thread - utility - 2023-03-24 09:38:57,675 - Api response stored in: /tmp/output/AMS_METRICS/jobhistoryserver/jobhistoryserver_max.json DEBUG - metrics_collector_thread - hdp_metrics_extractor - 2023-03-24 09:38:57,675 - http://ccycloud-1.shyao-test-1677230213.root.hwx.site:6188/ws/v1/timeline/metrics?metricNames=%._min&appId=jobhistoryserver&hostname=%&startTime=1675762730&endTime=1679650730 DEBUG - metrics_collector_thread - utility - 2023-03-24 09:38:58,002 - Api response stored in: /tmp/output/AMS_METRICS/jobhistoryserver/jobhistoryserver_min.json DEBUG - metrics_collector_thread - hdp_metrics_extractor - 2023-03-24 09:38:58,003 - http://ccycloud-1.shyao-test-1677230213.root.hwx.site:6188/ws/v1/timeline/metrics?metricNames=%._avg&appId=jobhistoryserver&hostname=%&startTime=1675762730&endTime=1679650730 DEBUG - metrics_collector_thread - utility - 2023-03-24 09:38:58,335 - Api response stored in: /tmp/output/AMS_METRICS/jobhistoryserver/jobhistoryserver_avg.json DEBUG - metrics_collector_thread - hdp_metrics_extractor - 2023-03-24 09:38:58,335 - http://ccycloud-1.shyao-test-1677230213.root.hwx.site:6188/ws/v1/timeline/metrics?metricNames=%&appId=namenode&hostname=%&startTime=1675762730&endTime=1679650730 DEBUG - metrics_collector_thread - utility - 2023-03-24 09:38:58,442 - Api response stored in: /tmp/output/AMS_METRICS/apps.json DEBUG - metrics_collector_thread - utility - 2023-03-24 09:38:58,918 - Api response stored in: /tmp/output/AMS_METRICS/namenode/namenode.json DEBUG - metrics_collector_thread - hdp_metrics_extractor - 2023-03-24 09:38:58,918 - http://ccycloud-1.shyao-test-1677230213.root.hwx.site:6188/ws/v1/timeline/metrics?metricNames=%._max&appId=namenode&hostname=%&startTime=1675762730&endTime=1679650730 DEBUG - metrics_collector_thread - utility - 2023-03-24 09:38:59,376 - Api response stored in: /tmp/output/AMS_METRICS/namenode/namenode_max.json DEBUG - metrics_collector_thread - hdp_metrics_extractor - 2023-03-24 09:38:59,376 - http://ccycloud-1.shyao-test-1677230213.root.hwx.site:6188/ws/v1/timeline/metrics?metricNames=%._min&appId=namenode&hostname=%&startTime=1675762730&endTime=1679650730 DEBUG - metrics_collector_thread - utility - 2023-03-24 09:38:59,827 - Api response stored in: /tmp/output/AMS_METRICS/namenode/namenode_min.json DEBUG - metrics_collector_thread - hdp_metrics_extractor - 2023-03-24 09:38:59,828 - http://ccycloud-1.shyao-test-1677230213.root.hwx.site:6188/ws/v1/timeline/metrics?metricNames=%._avg&appId=namenode&hostname=%&startTime=1675762730&endTime=1679650730 DEBUG - metrics_collector_thread - utility - 2023-03-24 09:39:00,302 - Api response stored in: /tmp/output/AMS_METRICS/namenode/namenode_avg.json DEBUG - metrics_collector_thread - hdp_metrics_extractor - 2023-03-24 09:39:00,302 - http://ccycloud-1.shyao-test-1677230213.root.hwx.site:6188/ws/v1/timeline/metrics?metricNames=%&appId=kafka_broker&hostname=%&startTime=1675762730&endTime=1679650730 DEBUG - metrics_collector_thread - utility - 2023-03-24 09:39:00,405 - Api response stored in: /tmp/output/AMS_METRICS/apps.json DEBUG - metrics_collector_thread - utility - 2023-03-24 09:39:01,011 - Api response stored in: /tmp/output/AMS_METRICS/kafka_broker/kafka_broker.json DEBUG - metrics_collector_thread - hdp_metrics_extractor - 2023-03-24 09:39:01,012 - http://ccycloud-1.shyao-test-1677230213.root.hwx.site:6188/ws/v1/timeline/metrics?metricNames=%._max&appId=kafka_broker&hostname=%&startTime=1675762730&endTime=1679650730 DEBUG - metrics_collector_thread - utility - 2023-03-24 09:39:01,565 - Api response stored in: /tmp/output/AMS_METRICS/kafka_broker/kafka_broker_max.json DEBUG - metrics_collector_thread - hdp_metrics_extractor - 2023-03-24 09:39:01,565 - http://ccycloud-1.shyao-test-1677230213.root.hwx.site:6188/ws/v1/timeline/metrics?metricNames=%._min&appId=kafka_broker&hostname=%&startTime=1675762730&endTime=1679650730 DEBUG - metrics_collector_thread - utility - 2023-03-24 09:39:02,130 - Api response stored in: /tmp/output/AMS_METRICS/kafka_broker/kafka_broker_min.json DEBUG - metrics_collector_thread - hdp_metrics_extractor - 2023-03-24 09:39:02,130 - http://ccycloud-1.shyao-test-1677230213.root.hwx.site:6188/ws/v1/timeline/metrics?metricNames=%._avg&appId=kafka_broker&hostname=%&startTime=1675762730&endTime=1679650730 DEBUG - metrics_collector_thread - utility - 2023-03-24 09:39:02,686 - Api response stored in: /tmp/output/AMS_METRICS/kafka_broker/kafka_broker_avg.json DEBUG - metrics_collector_thread - hdp_metrics_extractor - 2023-03-24 09:39:02,687 - http://ccycloud-1.shyao-test-1677230213.root.hwx.site:6188/ws/v1/timeline/metrics?metricNames=%&appId=ambari-infra-solr&hostname=%&startTime=1675762730&endTime=1679650730 DEBUG - metrics_collector_thread - utility - 2023-03-24 09:39:02,789 - Api response stored in: /tmp/output/AMS_METRICS/apps.json DEBUG - metrics_collector_thread - utility - 2023-03-24 09:39:03,289 - Api response stored in: /tmp/output/AMS_METRICS/ambari-infra-solr/ambari-infra-solr.json DEBUG - metrics_collector_thread - hdp_metrics_extractor - 2023-03-24 09:39:03,289 - http://ccycloud-1.shyao-test-1677230213.root.hwx.site:6188/ws/v1/timeline/metrics?metricNames=%._max&appId=ambari-infra-solr&hostname=%&startTime=1675762730&endTime=1679650730 DEBUG - metrics_collector_thread - utility - 2023-03-24 09:39:03,715 - Api response stored in: /tmp/output/AMS_METRICS/ambari-infra-solr/ambari-infra-solr_max.json DEBUG - metrics_collector_thread - hdp_metrics_extractor - 2023-03-24 09:39:03,716 - http://ccycloud-1.shyao-test-1677230213.root.hwx.site:6188/ws/v1/timeline/metrics?metricNames=%._min&appId=ambari-infra-solr&hostname=%&startTime=1675762730&endTime=1679650730 DEBUG - metrics_collector_thread - utility - 2023-03-24 09:39:04,126 - Api response stored in: /tmp/output/AMS_METRICS/ambari-infra-solr/ambari-infra-solr_min.json DEBUG - metrics_collector_thread - hdp_metrics_extractor - 2023-03-24 09:39:04,126 - http://ccycloud-1.shyao-test-1677230213.root.hwx.site:6188/ws/v1/timeline/metrics?metricNames=%._avg&appId=ambari-infra-solr&hostname=%&startTime=1675762730&endTime=1679650730 DEBUG - metrics_collector_thread - utility - 2023-03-24 09:39:04,614 - Api response stored in: /tmp/output/AMS_METRICS/ambari-infra-solr/ambari-infra-solr_avg.json DEBUG - metrics_collector_thread - hdp_metrics_extractor - 2023-03-24 09:39:04,614 - http://ccycloud-1.shyao-test-1677230213.root.hwx.site:6188/ws/v1/timeline/metrics?metricNames=%&appId=ambari_server&hostname=%&startTime=1675762730&endTime=1679650730 DEBUG - metrics_collector_thread - utility - 2023-03-24 09:39:04,716 - Api response stored in: /tmp/output/AMS_METRICS/apps.json DEBUG - metrics_collector_thread - utility - 2023-03-24 09:39:05,169 - Api response stored in: /tmp/output/AMS_METRICS/ambari_server/ambari_server.json DEBUG - metrics_collector_thread - hdp_metrics_extractor - 2023-03-24 09:39:05,170 - http://ccycloud-1.shyao-test-1677230213.root.hwx.site:6188/ws/v1/timeline/metrics?metricNames=%._max&appId=ambari_server&hostname=%&startTime=1675762730&endTime=1679650730 DEBUG - metrics_collector_thread - utility - 2023-03-24 09:39:05,536 - Api response stored in: /tmp/output/AMS_METRICS/ambari_server/ambari_server_max.json DEBUG - metrics_collector_thread - hdp_metrics_extractor - 2023-03-24 09:39:05,537 - http://ccycloud-1.shyao-test-1677230213.root.hwx.site:6188/ws/v1/timeline/metrics?metricNames=%._min&appId=ambari_server&hostname=%&startTime=1675762730&endTime=1679650730 DEBUG - metrics_collector_thread - utility - 2023-03-24 09:39:05,905 - Api response stored in: /tmp/output/AMS_METRICS/ambari_server/ambari_server_min.json DEBUG - metrics_collector_thread - hdp_metrics_extractor - 2023-03-24 09:39:05,905 - http://ccycloud-1.shyao-test-1677230213.root.hwx.site:6188/ws/v1/timeline/metrics?metricNames=%._avg&appId=ambari_server&hostname=%&startTime=1675762730&endTime=1679650730 DEBUG - metrics_collector_thread - utility - 2023-03-24 09:39:06,377 - Api response stored in: /tmp/output/AMS_METRICS/ambari_server/ambari_server_avg.json DEBUG - metrics_collector_thread - hdp_metrics_extractor - 2023-03-24 09:39:06,378 - http://ccycloud-1.shyao-test-1677230213.root.hwx.site:6188/ws/v1/timeline/metrics?metricNames=%&appId=hiveserver2&hostname=%&startTime=1675762730&endTime=1679650730 DEBUG - metrics_collector_thread - utility - 2023-03-24 09:39:06,484 - Api response stored in: /tmp/output/AMS_METRICS/apps.json DEBUG - metrics_collector_thread - utility - 2023-03-24 09:39:07,050 - Api response stored in: /tmp/output/AMS_METRICS/hiveserver2/hiveserver2.json DEBUG - metrics_collector_thread - hdp_metrics_extractor - 2023-03-24 09:39:07,051 - http://ccycloud-1.shyao-test-1677230213.root.hwx.site:6188/ws/v1/timeline/metrics?metricNames=%._max&appId=hiveserver2&hostname=%&startTime=1675762730&endTime=1679650730 DEBUG - metrics_collector_thread - utility - 2023-03-24 09:39:07,765 - Api response stored in: /tmp/output/AMS_METRICS/hiveserver2/hiveserver2_max.json DEBUG - metrics_collector_thread - hdp_metrics_extractor - 2023-03-24 09:39:07,766 - http://ccycloud-1.shyao-test-1677230213.root.hwx.site:6188/ws/v1/timeline/metrics?metricNames=%._min&appId=hiveserver2&hostname=%&startTime=1675762730&endTime=1679650730 DEBUG - metrics_collector_thread - utility - 2023-03-24 09:39:08,268 - Api response stored in: /tmp/output/AMS_METRICS/hiveserver2/hiveserver2_min.json DEBUG - metrics_collector_thread - hdp_metrics_extractor - 2023-03-24 09:39:08,269 - http://ccycloud-1.shyao-test-1677230213.root.hwx.site:6188/ws/v1/timeline/metrics?metricNames=%._avg&appId=hiveserver2&hostname=%&startTime=1675762730&endTime=1679650730 DEBUG - metrics_collector_thread - utility - 2023-03-24 09:39:08,872 - Api response stored in: /tmp/output/AMS_METRICS/hiveserver2/hiveserver2_avg.json DEBUG - metrics_collector_thread - hdp_metrics_extractor - 2023-03-24 09:39:08,873 - http://ccycloud-1.shyao-test-1677230213.root.hwx.site:6188/ws/v1/timeline/metrics?metricNames=%&appId=amssmoketestfake&hostname=%&startTime=1675762730&endTime=1679650730 DEBUG - metrics_collector_thread - utility - 2023-03-24 09:39:08,975 - Api response stored in: /tmp/output/AMS_METRICS/apps.json DEBUG - metrics_collector_thread - utility - 2023-03-24 09:39:09,264 - Api response stored in: /tmp/output/AMS_METRICS/amssmoketestfake/amssmoketestfake.json DEBUG - metrics_collector_thread - hdp_metrics_extractor - 2023-03-24 09:39:09,265 - http://ccycloud-1.shyao-test-1677230213.root.hwx.site:6188/ws/v1/timeline/metrics?metricNames=%._max&appId=amssmoketestfake&hostname=%&startTime=1675762730&endTime=1679650730 DEBUG - metrics_collector_thread - utility - 2023-03-24 09:39:09,526 - Api response stored in: /tmp/output/AMS_METRICS/amssmoketestfake/amssmoketestfake_max.json DEBUG - metrics_collector_thread - hdp_metrics_extractor - 2023-03-24 09:39:09,527 - http://ccycloud-1.shyao-test-1677230213.root.hwx.site:6188/ws/v1/timeline/metrics?metricNames=%._min&appId=amssmoketestfake&hostname=%&startTime=1675762730&endTime=1679650730 DEBUG - metrics_collector_thread - utility - 2023-03-24 09:39:09,825 - Api response stored in: /tmp/output/AMS_METRICS/amssmoketestfake/amssmoketestfake_min.json DEBUG - metrics_collector_thread - hdp_metrics_extractor - 2023-03-24 09:39:09,825 - http://ccycloud-1.shyao-test-1677230213.root.hwx.site:6188/ws/v1/timeline/metrics?metricNames=%._avg&appId=amssmoketestfake&hostname=%&startTime=1675762730&endTime=1679650730 DEBUG - metrics_collector_thread - utility - 2023-03-24 09:39:10,098 - Api response stored in: /tmp/output/AMS_METRICS/amssmoketestfake/amssmoketestfake_avg.json DEBUG - metrics_collector_thread - hdp_metrics_extractor - 2023-03-24 09:39:10,098 - http://ccycloud-1.shyao-test-1677230213.root.hwx.site:6188/ws/v1/timeline/metrics?metricNames=%&appId=hivemetastore&hostname=%&startTime=1675762730&endTime=1679650730 DEBUG - metrics_collector_thread - utility - 2023-03-24 09:39:10,202 - Api response stored in: /tmp/output/AMS_METRICS/apps.json DEBUG - metrics_collector_thread - utility - 2023-03-24 09:39:10,732 - Api response stored in: /tmp/output/AMS_METRICS/hivemetastore/hivemetastore.json DEBUG - metrics_collector_thread - hdp_metrics_extractor - 2023-03-24 09:39:10,733 - http://ccycloud-1.shyao-test-1677230213.root.hwx.site:6188/ws/v1/timeline/metrics?metricNames=%._max&appId=hivemetastore&hostname=%&startTime=1675762730&endTime=1679650730 DEBUG - metrics_collector_thread - utility - 2023-03-24 09:39:11,267 - Api response stored in: /tmp/output/AMS_METRICS/hivemetastore/hivemetastore_max.json DEBUG - metrics_collector_thread - hdp_metrics_extractor - 2023-03-24 09:39:11,268 - http://ccycloud-1.shyao-test-1677230213.root.hwx.site:6188/ws/v1/timeline/metrics?metricNames=%._min&appId=hivemetastore&hostname=%&startTime=1675762730&endTime=1679650730 DEBUG - metrics_collector_thread - utility - 2023-03-24 09:39:11,751 - Api response stored in: /tmp/output/AMS_METRICS/hivemetastore/hivemetastore_min.json DEBUG - metrics_collector_thread - hdp_metrics_extractor - 2023-03-24 09:39:11,751 - http://ccycloud-1.shyao-test-1677230213.root.hwx.site:6188/ws/v1/timeline/metrics?metricNames=%._avg&appId=hivemetastore&hostname=%&startTime=1675762730&endTime=1679650730 DEBUG - metrics_collector_thread - utility - 2023-03-24 09:39:12,278 - Api response stored in: /tmp/output/AMS_METRICS/hivemetastore/hivemetastore_avg.json DEBUG - metrics_collector_thread - hdp_metrics_extractor - 2023-03-24 09:39:12,279 - http://ccycloud-1.shyao-test-1677230213.root.hwx.site:6188/ws/v1/timeline/metrics?metricNames=%&appId=resourcemanager&hostname=%&startTime=1675762730&endTime=1679650730 DEBUG - metrics_collector_thread - utility - 2023-03-24 09:39:12,386 - Api response stored in: /tmp/output/AMS_METRICS/apps.json DEBUG - metrics_collector_thread - utility - 2023-03-24 09:39:12,879 - Api response stored in: /tmp/output/AMS_METRICS/resourcemanager/resourcemanager.json DEBUG - metrics_collector_thread - hdp_metrics_extractor - 2023-03-24 09:39:12,880 - http://ccycloud-1.shyao-test-1677230213.root.hwx.site:6188/ws/v1/timeline/metrics?metricNames=%._max&appId=resourcemanager&hostname=%&startTime=1675762730&endTime=1679650730 DEBUG - metrics_collector_thread - utility - 2023-03-24 09:39:13,383 - Api response stored in: /tmp/output/AMS_METRICS/resourcemanager/resourcemanager_max.json DEBUG - metrics_collector_thread - hdp_metrics_extractor - 2023-03-24 09:39:13,384 - http://ccycloud-1.shyao-test-1677230213.root.hwx.site:6188/ws/v1/timeline/metrics?metricNames=%._min&appId=resourcemanager&hostname=%&startTime=1675762730&endTime=1679650730 DEBUG - metrics_collector_thread - utility - 2023-03-24 09:39:13,860 - Api response stored in: /tmp/output/AMS_METRICS/resourcemanager/resourcemanager_min.json DEBUG - metrics_collector_thread - hdp_metrics_extractor - 2023-03-24 09:39:13,861 - http://ccycloud-1.shyao-test-1677230213.root.hwx.site:6188/ws/v1/timeline/metrics?metricNames=%._avg&appId=resourcemanager&hostname=%&startTime=1675762730&endTime=1679650730 DEBUG - metrics_collector_thread - utility - 2023-03-24 09:39:14,420 - Api response stored in: /tmp/output/AMS_METRICS/resourcemanager/resourcemanager_avg.json DEBUG - metrics_collector_thread - hdp_metrics_extractor - 2023-03-24 09:39:14,421 - http://ccycloud-1.shyao-test-1677230213.root.hwx.site:6188/ws/v1/timeline/metrics?metricNames=%&appId=journalnode&hostname=%&startTime=1675762730&endTime=1679650730 DEBUG - metrics_collector_thread - utility - 2023-03-24 09:39:14,518 - Api response stored in: /tmp/output/AMS_METRICS/apps.json DEBUG - metrics_collector_thread - utility - 2023-03-24 09:39:14,975 - Api response stored in: /tmp/output/AMS_METRICS/journalnode/journalnode.json DEBUG - metrics_collector_thread - hdp_metrics_extractor - 2023-03-24 09:39:14,976 - http://ccycloud-1.shyao-test-1677230213.root.hwx.site:6188/ws/v1/timeline/metrics?metricNames=%._max&appId=journalnode&hostname=%&startTime=1675762730&endTime=1679650730 DEBUG - metrics_collector_thread - utility - 2023-03-24 09:39:15,477 - Api response stored in: /tmp/output/AMS_METRICS/journalnode/journalnode_max.json DEBUG - metrics_collector_thread - hdp_metrics_extractor - 2023-03-24 09:39:15,478 - http://ccycloud-1.shyao-test-1677230213.root.hwx.site:6188/ws/v1/timeline/metrics?metricNames=%._min&appId=journalnode&hostname=%&startTime=1675762730&endTime=1679650730 DEBUG - metrics_collector_thread - utility - 2023-03-24 09:39:16,058 - Api response stored in: /tmp/output/AMS_METRICS/journalnode/journalnode_min.json DEBUG - metrics_collector_thread - hdp_metrics_extractor - 2023-03-24 09:39:16,059 - http://ccycloud-1.shyao-test-1677230213.root.hwx.site:6188/ws/v1/timeline/metrics?metricNames=%._avg&appId=journalnode&hostname=%&startTime=1675762730&endTime=1679650730 DEBUG - metrics_collector_thread - utility - 2023-03-24 09:39:16,571 - Api response stored in: /tmp/output/AMS_METRICS/journalnode/journalnode_avg.json DEBUG - metrics_collector_thread - hdp_metrics_extractor - 2023-03-24 09:39:16,572 - http://ccycloud-1.shyao-test-1677230213.root.hwx.site:6188/ws/v1/timeline/metrics?metricNames=%&appId=yarn-timeline-storage-hbase&hostname=%&startTime=1675762730&endTime=1679650730 DEBUG - metrics_collector_thread - utility - 2023-03-24 09:39:16,672 - Api response stored in: /tmp/output/AMS_METRICS/apps.json DEBUG - metrics_collector_thread - utility - 2023-03-24 09:39:17,204 - Api response stored in: /tmp/output/AMS_METRICS/yarn-timeline-storage-hbase/yarn-timeline-storage-hbase.json DEBUG - metrics_collector_thread - hdp_metrics_extractor - 2023-03-24 09:39:17,205 - http://ccycloud-1.shyao-test-1677230213.root.hwx.site:6188/ws/v1/timeline/metrics?metricNames=%._max&appId=yarn-timeline-storage-hbase&hostname=%&startTime=1675762730&endTime=1679650730 DEBUG - metrics_collector_thread - utility - 2023-03-24 09:39:17,827 - Api response stored in: /tmp/output/AMS_METRICS/yarn-timeline-storage-hbase/yarn-timeline-storage-hbase_max.json DEBUG - metrics_collector_thread - hdp_metrics_extractor - 2023-03-24 09:39:17,828 - http://ccycloud-1.shyao-test-1677230213.root.hwx.site:6188/ws/v1/timeline/metrics?metricNames=%._min&appId=yarn-timeline-storage-hbase&hostname=%&startTime=1675762730&endTime=1679650730 DEBUG - metrics_collector_thread - utility - 2023-03-24 09:39:18,337 - Api response stored in: /tmp/output/AMS_METRICS/yarn-timeline-storage-hbase/yarn-timeline-storage-hbase_min.json DEBUG - metrics_collector_thread - hdp_metrics_extractor - 2023-03-24 09:39:18,338 - http://ccycloud-1.shyao-test-1677230213.root.hwx.site:6188/ws/v1/timeline/metrics?metricNames=%._avg&appId=yarn-timeline-storage-hbase&hostname=%&startTime=1675762730&endTime=1679650730 DEBUG - metrics_collector_thread - utility - 2023-03-24 09:39:18,990 - Api response stored in: /tmp/output/AMS_METRICS/yarn-timeline-storage-hbase/yarn-timeline-storage-hbase_avg.json DEBUG - metrics_collector_thread - hdp_metrics_extractor - 2023-03-24 09:39:18,991 - http://ccycloud-1.shyao-test-1677230213.root.hwx.site:6188/ws/v1/timeline/metrics?metricNames=%&appId=applicationhistoryserver&hostname=%&startTime=1675762730&endTime=1679650730 DEBUG - metrics_collector_thread - utility - 2023-03-24 09:39:19,091 - Api response stored in: /tmp/output/AMS_METRICS/apps.json DEBUG - metrics_collector_thread - utility - 2023-03-24 09:39:19,642 - Api response stored in: /tmp/output/AMS_METRICS/applicationhistoryserver/applicationhistoryserver.json DEBUG - metrics_collector_thread - hdp_metrics_extractor - 2023-03-24 09:39:19,643 - http://ccycloud-1.shyao-test-1677230213.root.hwx.site:6188/ws/v1/timeline/metrics?metricNames=%._max&appId=applicationhistoryserver&hostname=%&startTime=1675762730&endTime=1679650730 DEBUG - metrics_collector_thread - utility - 2023-03-24 09:39:20,182 - Api response stored in: /tmp/output/AMS_METRICS/applicationhistoryserver/applicationhistoryserver_max.json DEBUG - metrics_collector_thread - hdp_metrics_extractor - 2023-03-24 09:39:20,183 - http://ccycloud-1.shyao-test-1677230213.root.hwx.site:6188/ws/v1/timeline/metrics?metricNames=%._min&appId=applicationhistoryserver&hostname=%&startTime=1675762730&endTime=1679650730 DEBUG - metrics_collector_thread - utility - 2023-03-24 09:39:20,676 - Api response stored in: /tmp/output/AMS_METRICS/applicationhistoryserver/applicationhistoryserver_min.json DEBUG - metrics_collector_thread - hdp_metrics_extractor - 2023-03-24 09:39:20,677 - http://ccycloud-1.shyao-test-1677230213.root.hwx.site:6188/ws/v1/timeline/metrics?metricNames=%._avg&appId=applicationhistoryserver&hostname=%&startTime=1675762730&endTime=1679650730 DEBUG - metrics_collector_thread - utility - 2023-03-24 09:39:21,065 - Api response stored in: /tmp/output/AMS_METRICS/applicationhistoryserver/applicationhistoryserver_avg.json DEBUG - metrics_collector_thread - hdp_metrics_extractor - 2023-03-24 09:39:21,066 - http://ccycloud-1.shyao-test-1677230213.root.hwx.site:6188/ws/v1/timeline/metrics?metricNames=%&appId=nodemanager&hostname=%&startTime=1675762730&endTime=1679650730 DEBUG - metrics_collector_thread - utility - 2023-03-24 09:39:21,173 - Api response stored in: /tmp/output/AMS_METRICS/apps.json DEBUG - metrics_collector_thread - utility - 2023-03-24 09:39:21,849 - Api response stored in: /tmp/output/AMS_METRICS/nodemanager/nodemanager.json DEBUG - metrics_collector_thread - hdp_metrics_extractor - 2023-03-24 09:39:21,849 - http://ccycloud-1.shyao-test-1677230213.root.hwx.site:6188/ws/v1/timeline/metrics?metricNames=%._max&appId=nodemanager&hostname=%&startTime=1675762730&endTime=1679650730 DEBUG - metrics_collector_thread - utility - 2023-03-24 09:39:22,396 - Api response stored in: /tmp/output/AMS_METRICS/nodemanager/nodemanager_max.json DEBUG - metrics_collector_thread - hdp_metrics_extractor - 2023-03-24 09:39:22,397 - http://ccycloud-1.shyao-test-1677230213.root.hwx.site:6188/ws/v1/timeline/metrics?metricNames=%._min&appId=nodemanager&hostname=%&startTime=1675762730&endTime=1679650730 DEBUG - metrics_collector_thread - utility - 2023-03-24 09:39:22,868 - Api response stored in: /tmp/output/AMS_METRICS/nodemanager/nodemanager_min.json DEBUG - metrics_collector_thread - hdp_metrics_extractor - 2023-03-24 09:39:22,869 - http://ccycloud-1.shyao-test-1677230213.root.hwx.site:6188/ws/v1/timeline/metrics?metricNames=%._avg&appId=nodemanager&hostname=%&startTime=1675762730&endTime=1679650730 DEBUG - metrics_collector_thread - utility - 2023-03-24 09:39:23,358 - Api response stored in: /tmp/output/AMS_METRICS/nodemanager/nodemanager_avg.json DEBUG - metrics_collector_thread - hdp_metrics_extractor - 2023-03-24 09:39:23,359 - http://ccycloud-1.shyao-test-1677230213.root.hwx.site:6188/ws/v1/timeline/metrics?metricNames=%&appId=hbase&hostname=%&startTime=1675762730&endTime=1679650730 DEBUG - metrics_collector_thread - utility - 2023-03-24 09:39:23,462 - Api response stored in: /tmp/output/AMS_METRICS/apps.json DEBUG - metrics_collector_thread - utility - 2023-03-24 09:39:23,965 - Api response stored in: /tmp/output/AMS_METRICS/hbase/hbase.json DEBUG - metrics_collector_thread - hdp_metrics_extractor - 2023-03-24 09:39:23,966 - http://ccycloud-1.shyao-test-1677230213.root.hwx.site:6188/ws/v1/timeline/metrics?metricNames=%._max&appId=hbase&hostname=%&startTime=1675762730&endTime=1679650730 DEBUG - metrics_collector_thread - utility - 2023-03-24 09:39:24,601 - Api response stored in: /tmp/output/AMS_METRICS/hbase/hbase_max.json DEBUG - metrics_collector_thread - hdp_metrics_extractor - 2023-03-24 09:39:24,601 - http://ccycloud-1.shyao-test-1677230213.root.hwx.site:6188/ws/v1/timeline/metrics?metricNames=%._min&appId=hbase&hostname=%&startTime=1675762730&endTime=1679650730 DEBUG - metrics_collector_thread - utility - 2023-03-24 09:39:25,118 - Api response stored in: /tmp/output/AMS_METRICS/hbase/hbase_min.json DEBUG - metrics_collector_thread - hdp_metrics_extractor - 2023-03-24 09:39:25,119 - http://ccycloud-1.shyao-test-1677230213.root.hwx.site:6188/ws/v1/timeline/metrics?metricNames=%._avg&appId=hbase&hostname=%&startTime=1675762730&endTime=1679650730 DEBUG - metrics_collector_thread - utility - 2023-03-24 09:39:25,631 - Api response stored in: /tmp/output/AMS_METRICS/hbase/hbase_avg.json DEBUG - metrics_collector_thread - hdp_metrics_extractor - 2023-03-24 09:39:25,631 - http://ccycloud-1.shyao-test-1677230213.root.hwx.site:6188/ws/v1/timeline/metrics?metricNames=%&appId=ams-hbase&hostname=%&startTime=1675762730&endTime=1679650730 DEBUG - metrics_collector_thread - utility - 2023-03-24 09:39:25,735 - Api response stored in: /tmp/output/AMS_METRICS/apps.json DEBUG - metrics_collector_thread - utility - 2023-03-24 09:39:26,490 - Api response stored in: /tmp/output/AMS_METRICS/ams-hbase/ams-hbase.json DEBUG - metrics_collector_thread - hdp_metrics_extractor - 2023-03-24 09:39:26,491 - http://ccycloud-1.shyao-test-1677230213.root.hwx.site:6188/ws/v1/timeline/metrics?metricNames=%._max&appId=ams-hbase&hostname=%&startTime=1675762730&endTime=1679650730 DEBUG - metrics_collector_thread - utility - 2023-03-24 09:39:27,254 - Api response stored in: /tmp/output/AMS_METRICS/ams-hbase/ams-hbase_max.json DEBUG - metrics_collector_thread - hdp_metrics_extractor - 2023-03-24 09:39:27,254 - http://ccycloud-1.shyao-test-1677230213.root.hwx.site:6188/ws/v1/timeline/metrics?metricNames=%._min&appId=ams-hbase&hostname=%&startTime=1675762730&endTime=1679650730 DEBUG - metrics_collector_thread - utility - 2023-03-24 09:39:27,948 - Api response stored in: /tmp/output/AMS_METRICS/ams-hbase/ams-hbase_min.json DEBUG - metrics_collector_thread - hdp_metrics_extractor - 2023-03-24 09:39:27,948 - http://ccycloud-1.shyao-test-1677230213.root.hwx.site:6188/ws/v1/timeline/metrics?metricNames=%._avg&appId=ams-hbase&hostname=%&startTime=1675762730&endTime=1679650730 DEBUG - metrics_collector_thread - utility - 2023-03-24 09:39:28,633 - Api response stored in: /tmp/output/AMS_METRICS/ams-hbase/ams-hbase_avg.json DEBUG - metrics_collector_thread - hdp_metrics_extractor - 2023-03-24 09:39:28,643 - Completed collecting the Metrics INFO - MainThread - discovery_bundle_builder - 2023-03-24 09:39:28,644 - Finished discovery bundle extraction, results available at: /tmp/output

  1. KeyNameError solved;
  2. HTTPS verify false error solved
yaoluffy commented 1 year ago
Click here for an HDFS configuration json parse Example

{ "dfs.block.access.token.enable":"true", "dfs.blockreport.initialDelay":"120", "dfs.blocksize":"134217728", "dfs.client.block.write.locateFollowingBlock.retries":"10", "dfs.client.failover.proxy.provider.jpw1ns1":"org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider", "dfs.client.read.shortcircuit":"true", "dfs.client.read.shortcircuit.streams.cache.size":"4096", "dfs.client.retry.policy.enabled":"false", "dfs.cluster.administrators":" hdfs", "dfs.content-summary.limit":"5000", "dfs.data.transfer.protection":"authentication,privacy", "dfs.datanode.address":"0.0.0.0:1019", "dfs.datanode.balance.bandwidthPerSec":"6250000", "dfs.datanode.data.dir":"/hadoop/hdd01/hdfs,/hadoop/hdd02/hdfs,/hadoop/hdd03/hdfs,/hadoop/hdd04/hdfs,/hadoop/hdd05/hdfs,/hadoop/hdd06/hdfs,/hadoop/hdd07/hdfs,/hadoop/hdd08/hdfs,/hadoop/hdd09/hdfs,/hadoop/hdd10/hdfs,/hadoop/hdd11/hdfs,/hadoop/hdd12/hdfs,[SSD]file:///hadoop/ssd01/hdfs,[SSD]file:///hadoop/ssd02/hdfs", "dfs.datanode.data.dir.perm":"750", "dfs.datanode.du.reserved":"96913891328", "dfs.datanode.failed.volumes.tolerated":"6", "dfs.datanode.http.address":"0.0.0.0:1022", "dfs.datanode.https.address":"0.0.0.0:50475", "dfs.datanode.ipc.address":"0.0.0.0:8010", "dfs.datanode.kerberos.principal":"dn/_HOST@JP_STG.HADOOP.RAKUTEN.COM", "dfs.datanode.keytab.file":"/etc/security/keytabs/dn.service.keytab", "dfs.datanode.max.transfer.threads":"16384", "dfs.domain.socket.path":"/var/lib/hadoop-hdfs/dn_socket", "dfs.encrypt.data.transfer.cipher.suites":"AES/CTR/NoPadding", "dfs.encryption.key.provider.uri":"kms://http@stg-ranger6001.haas.jpw1a.dcnw.rakuten;stg-ranger6002.haas.jpw1a.dcnw.rakuten:9292/kms", "dfs.ha.automatic-failover.enabled":"true", "dfs.ha.fencing.methods":"shell(/bin/true)", "dfs.ha.namenodes.jpw1ns1":"nn1,nn2", "dfs.heartbeat.interval":"3", "dfs.hosts.exclude":"/etc/hadoop/conf/dfs.exclude", "dfs.http.policy":"HTTP_ONLY", "dfs.https.port":"50470", "dfs.internal.nameservices":"jpw1ns1", "dfs.journalnode.edits.dir":"/hadoop/hdfs/journal_jpw1ns1", "dfs.journalnode.http-address":"0.0.0.0:8480", "dfs.journalnode.https-address":"0.0.0.0:8481", "dfs.journalnode.kerberos.internal.spnego.principal":"HTTP/_HOST@JP_STG.HADOOP.RAKUTEN.COM", "dfs.journalnode.kerberos.principal":"jn/_HOST@JP_STG.HADOOP.RAKUTEN.COM", "dfs.journalnode.keytab.file":"/etc/security/keytabs/jn.service.keytab", "dfs.ls.limit":"250", "dfs.namenode.accesstime.precision":"0", "dfs.namenode.acls.enabled":"true", "dfs.namenode.audit.log.async":"true", "dfs.namenode.avoid.read.stale.datanode":"true", "dfs.namenode.avoid.write.stale.datanode":"true", "dfs.namenode.checkpoint.dir":"/hadoop/hdfs/namesecondary", "dfs.namenode.checkpoint.edits.dir":"${dfs.namenode.checkpoint.dir}", "dfs.namenode.checkpoint.period":"5400", "dfs.namenode.checkpoint.txns":"1000000", "dfs.namenode.fslock.fair":"false", "dfs.namenode.handler.count":"400", "dfs.namenode.http-address.jpw1ns1.nn1":"stg-nn6001.haas.jpw1a.dcnw.rakuten:50070", "dfs.namenode.http-address.jpw1ns1.nn2":"stg-nn6002.haas.jpw1a.dcnw.rakuten:50070", "dfs.namenode.https-address.jpw1ns1.nn1":"stg-nn6001.haas.jpw1a.dcnw.rakuten:50470", "dfs.namenode.https-address.jpw1ns1.nn2":"stg-nn6002.haas.jpw1a.dcnw.rakuten:50470", "dfs.namenode.inode.attributes.provider.class":"org.apache.ranger.authorization.hadoop.RangerHdfsAuthorizer", "dfs.namenode.kerberos.internal.spnego.principal":"HTTP/_HOST@JP_STG.HADOOP.RAKUTEN.COM", "dfs.namenode.kerberos.principal":"nn/_HOST@JP_STG.HADOOP.RAKUTEN.COM", "dfs.namenode.keytab.file":"/etc/security/keytabs/nn.service.keytab", "dfs.namenode.name.dir":"/hadoop/hdfs/namenode", "dfs.namenode.name.dir.restore":"true", "dfs.namenode.rpc-address.jpw1ns1.nn1":"stg-nn6001.haas.jpw1a.dcnw.rakuten:8020", "dfs.namenode.rpc-address.jpw1ns1.nn2":"stg-nn6002.haas.jpw1a.dcnw.rakuten:8020", "dfs.namenode.safemode.threshold-pct":"0.99", "dfs.namenode.shared.edits.dir":"qjournal://stg-jz6001.haas.jpw1a.dcnw.rakuten:8485;stg-jz6002.haas.jpw1a.dcnw.rakuten:8485;stg-jz6003.haas.jpw1a.dcnw.rakuten:8485;stg-jn6004.haas.jpw1a.dcnw.rakuten:8485;stg-jn6005.haas.jpw1a.dcnw.rakuten:8485/jpw1ns1", "dfs.namenode.stale.datanode.interval":"240000", "dfs.namenode.startup.delay.block.deletion.sec":"3600", "dfs.namenode.write.stale.datanode.ratio":"1.0f", "dfs.nameservices":"jpw1ns1", "dfs.permissions.ContentSummary.subAccess":"true", "dfs.permissions.enabled":"true", "dfs.permissions.superusergroup":"hdfs", "dfs.qjournal.select-input-streams.timeout.ms":"300000", "dfs.qjournal.start-segment.timeout.ms":"300000", "dfs.qjournal.write-txns.timeout.ms":"300000", "dfs.replication":"3", "dfs.replication.max":"50", "dfs.web.authentication.kerberos.keytab":"/etc/security/keytabs/spnego.service.keytab", "dfs.web.authentication.kerberos.principal":"HTTP/_HOST@JP_STG.HADOOP.RAKUTEN.COM", "dfs.webhdfs.enabled":"true", "fs.permissions.umask-mode":"077", "ha.failover-controller.new-active.rpc-timeout.ms":"300000", "ha.health-monitor.rpc-timeout.ms":"300000", "hadoop.caller.context.enabled":"true", "manage.include.files":"false", "nfs.exports.allowed.hosts":"* ro", "nfs.file.dump.dir":"/hadoop/.hdfs-nfs", "nfs.kerberos.principal":"nfs/_HOST@JP_STG.HADOOP.RAKUTEN.COM", "nfs.keytab.file":"/etc/security/keytabs/nfs.service.keytab" }

The Namespace Info can be fetch even with Multiple-Namespaces disabled.

wmudge commented 1 year ago

@yaoluffy - could you sign your commits and push again?

yaoluffy commented 1 year ago

@wmudge

could you sign your commits and push again?

Done. Please check.