clemlabprojects / ambari

Fork of Apache Ambari maintained by Clemlab Company
https://www.clemlab.com
Apache License 2.0
39 stars 15 forks source link

Failed to Start ResourceManager Curl out of memory #55

Closed Meikelrizkyhartawan closed 3 months ago

Meikelrizkyhartawan commented 5 months ago

im trying to start resource manager after install the odp in ubuntu22 but failed because of curl is out of memory, how to fix this error , i already set the /etc/security/limits.conf and add the setting into unlimited in ~/.bashrc but this error still exists

Free -h show my ram has available space

image

/etc/security/limits.conf

image

~/.bashrc

image

Error curl in ambari-server ui

image

Error Logs : resource_management.core.exceptions.ExecutionFailed: Execution of 'curl -sS -L -w '%{http_code}' -X PUT --data-binary @/var/lib/ambari-agent/tmp/yarn-ats/1.2.1.0-134/hbase.tar.gz -H 'Content-Type: application/octet-stream' 'http://hdp-namenode01.maas.ph:50070/webhdfs/v1/odp/apps/1.2.1.0-134/hbase/hbase.tar.gz?op=CREATE&user.name=hdfs&overwrite=True&permission=444' 1>/tmp/tmp4NHtEg 2>/tmp/tmpYhjVnK' returned 2. curl: option --data-binary: out of memory curl: try 'curl --help' or 'curl --manual' for more information

when i try to run the command manually it's looks like syntax error am i correct ? image

Arni82 commented 4 months ago

Same error with @/var/lib/ambari-agent/tmp/yarn-ats/1.2.2.0-105/hbase.tar.gz

🤷🏻‍♂️

lucasbak commented 4 months ago

@Arni82 @Meikelrizkyhartawan ,

Sorry for the delay. It can happen when you have low memory on host. We are updating the option of curl in order to make it pass on every system @Meikelrizkyhartawan , can you take the curl command and replace --data-binary by -F data=@/var/lib/ambari-agent/tmp/yarn-ats/1.2.1.0-134/hbase.tar.gz

Meikelrizkyhartawan commented 4 months ago

@lucasbak yeah it's work, i change the /usr/lib/ambari-agent/lib/resource_management/libraries/providers/hdfs_resource.py

from this :

cmd += ["--data-binary", "@"+file_to_put, "-H", "Content-Type: application/octet-stream"]

into this :

cmd += ["-F", "data=@"+file_to_put, "-H", "Content-Type: application/octet-stream"]