pkExec commented 4 years ago

[X] I've read the guidelines for Contributing to Kaltura Projects
[X] I've read the installation guides and troubleshooting FAQ
[X] This is not a support request or question that should be posted on the Kaltura forum

Bug report

*Please provide steps to reproduce, including errors from /opt/kaltura/log/kaltura_api_v3.log, /opt/kaltura/log/kaltura_apache_errorslog and /opt/kaltura/log/kaltura_prod.log:**

1) Start a new Amazon Lightsail instance with ubuntu 16.04 2) Modify hosts file for your domain 3) Install Percona as per the instructions https://github.com/kaltura/platform-install-packages/blob/Orion-15.11.0/doc/install-percona-deb.md 4) Follow the instructions here https://github.com/kaltura/platform-install-packages/blob/Orion-15.11.0/doc/install-kaltura-xenial.md

Expected outcome: Installed platform

Actual outcome: Failed on the installing kaltura-db step.

Cause: No databases are created

Root cause: /opt/kaltura/bin/db_actions.rc has variable OP_DBS, while script expects DBS

Is this an all in one instance [single server] or a cluster?

[X] All in one
[ ] Cluster

jessp01 commented 4 years ago

Hello @pkExec,

Thanks for reporting. This was already fixed and the updated kaltura-db package is available in the nightly repo. It will become the stable repo tomorrow.

For the time being, you can edit /etc/apt/sources.list.d/kaltura.list So that it reads:

deb [arch=amd64] http://installrepo.origin.kaltura.org/repo/apt/xenial-nightly orion main

Then invoke:

# apt update
# apt purge kaltura-db

and re-run the deployment script.

Cheers,

pkExec commented 4 years ago

Thanks. Indeed this step was fixed. Now it fails on the "Waiting for the ElasticSearch daemon to start..." step. It displays "We've waited a minute and the ES daemon is still unreachable, we're giving up: (" sed: can't read /configurations/base.ini: No such file or directory

thecaribbean commented 4 years ago

I've received the Elastic Search error same as @pkExec

Setting up kaltura-elasticsearch (1.0.0-1+b6) ... -> Downloading analysis-icu from elastic [=================================================] 100% -> Installed analysis-icu Waiting for the ElasticSearch daemon to start... We've waited a minute and the ES daemon is still unreachable, we're giving up:( Processing triggers for ureadahead (0.100.0-19.1) ... Processing triggers for systemd (229-4ubuntu21.23) ... sed: can't read /configurations/base.ini: No such file or directory

used a Fresh Install of Ubuntu 16.04 minimal iso and MySQL 5.5 Percona deb package only other package installed wis openssh-server

lsb_release -a No LSB modules are available. Distributor ID: Ubuntu Description: Ubuntu 16.04.6 LTS Release: 16.04 Codename: xenial

mysql -V mysql Ver 14.14 Distrib 5.5.62-38.14, for debian-linux-gnu (x86_64) using readline 5.1

jessp01 commented 4 years ago

Hello @pkExec, @thecaribbean,

Kaltura CE 15.14.0 was just released. Kindly try to upgrade. If the issue persists, please edit /var/lib/dpkg/info/kaltura-elasticsearch.postinst so that:

#!/bin/bash

becomes:

#!/bin/bash -x

then run:

# apt install -f

or, depending on the package's current state:

# dpkg-reconfigure kaltura-elasticsearch

And paste the output here.

Thanks,

thecaribbean commented 4 years ago

Started another fresh install and got up to:

Setting up kaltura-elasticsearch (1.0.0-1+b7) ... -> Downloading analysis-icu from elastic [=================================================] 100% -> Installed analysis-icu Waiting for the ElasticSearch daemon to start... We've waited a minute and the ES daemon is still unreachable, we're giving up:( Processing triggers for ureadahead (0.100.0-19.1) ... Processing triggers for systemd (229-4ubuntu21.23) ... sed: can't read /configurations/base.ini: No such file or directory

edited /var/lib/dpkg/info/kaltura-elasticsearch.postinst

then run:

apt install -f Reading package lists... Done Building dependency tree
Reading state information... Done 0 upgraded, 0 newly installed, 0 to remove and 1 not upgraded.

Then ran:

root@video:~# dpkg-reconfigure kaltura-elasticsearch

KALTURA_FUNCTIONS_RC=/opt/kaltura/bin/kaltura-functions.rc
'[' '!' -r /opt/kaltura/bin/kaltura-functions.rc ']'
. /opt/kaltura/bin/kaltura-functions.rc ++ RC_FILE=/etc/kaltura.d/system.ini ++ '[' -r /etc/kaltura.d/system.ini ']' ++ . /etc/kaltura.d/system.ini +++ WEB_DIR=/opt/kaltura/web +++ LOG_DIR=/opt/kaltura/log +++ APP_DIR=/opt/kaltura/app +++ BASE_DIR=/opt/kaltura +++ PHP_BIN=/usr/bin/php +++ OS_KALTURA_USER=kaltura +++ MAILTO=myemail@example.com +++ ADMIN_CONSOLE_ADMIN_MAIL=myemail@example.com +++ DB1_HOST=127.0.0.1 +++ DB1_NAME=kaltura +++ DB1_USER=kaltura +++ DB1_PASS= +++ DWH_HOST=127.0.0.1 +++ DWH_PORT=3306 +++ DWH_USER=etl +++ DWH_PASS= +++ KAVA_DB_HOST=@KAVA_DB_HOST@ +++ KAVA_DB_PORT=@KAVA_DB_PORT@ +++ KAVA_DB_USER=@KAVA_DB_USER@ +++ KAVA_DB_PASS=@KAVA_DB_PASS@ +++ SERVICE_URL=http://video.example.com +++ SPHINX_HOST=127.0.0.1 +++ DB1_PORT=3306 +++ SUPER_USER=root +++ SUPER_USER_PASSWD= +++ KALTURA_VIRTUAL_HOST_NAME=video.example.com +++ RED5_HOST= +++ dirname /var/lib/dpkg/info/kaltura-elasticsearch.postinst ++ COLORS_RC=/var/lib/dpkg/info/colors.sh ++ '[' -r /var/lib/dpkg/info/colors.sh ']' ++ '[' -r /etc/sysconfig/clock ']' ++ '[' -z '' ']' +++ get_tz +++ '[' -r /etc/sysconfig/clock ']' +++ '[' -n '' ']' ++++ awk -F ' ' '{print $1}' ++++ md5sum /etc/localtime +++ LOCALTIME_MD5=6a95f4e0602e0869a03a18a7501c6675 ++++ sed s@/usr/share/zoneinfo/@@g ++++ tail -1 ++++ awk -F ' ' '{print $2}' ++++ grep 6a95f4e0602e0869a03a18a7501c6675 ++++ xargs md5sum ++++ find /usr/share/zoneinfo -type f +++ TZGUESS=America/St_Thomas +++ echo America/St_Thomas ++ ZONE=America/St_Thomas ++ export ZONE ++ '[' -r /opt/kaltura/app/configurations/consent_msgs ']' ++ CONSENT_FILE=/opt/kaltura/bin/contact.rc ++ SPHINX_DAEMON=kaltura-sphinx ++ BATCH_DAEMON=kaltura-batch ++ MEMCACHE_DAEMON=memcached +++ lsb_release -i -s ++ DISTRO=Ubuntu ++ '[' Ubuntu = Ubuntu -o Ubuntu = Debian ']' ++ APACHE_DAEMON=apache2 ++ MYSQL_DAEMON=mysql ++ MONIT_DAEMON=monit ++ ALL_DAEMONS='apache2 kaltura-batch monit memcached' ++ INIT_CONFIG_UTIL=sysv-rc-conf ++ PACKAGE_TYPE=deb ++ QUERY_COMMAND='dpkg -l' ++ MAIL_LOG=/var/log/mail.log ++ POST_INST_MAIL_SUBJECT='Your new Kaltura server is up! and 6 Quick Tips For Smooth Kaltura Maintenance' ++ POST_INST_MAIL_TMPL=/opt/kaltura/app/configurations/post_inst_mail ++ ANALYTICS_SERVER_URI=https://installytics.kaltura.org/report/add_rec.php ++ NEWSLETTER_URL=https://agentcontribs.kaltura.org/newsletter_subscribe.php ++ NEWSLETTER_MSG=' To keep up with the bi-weekly Kaltura releases, and stay up to date with the latest news, security and bug updates, and connect with the global Kaltura community - sign up for our regular Kaltura Community updates. Please enter your email to sign up, or enter NO to pass. '
'[' configure = configure ']'
update-rc.d kaltura-elastic-populate defaults ++ date +%Y_%m
TIME_STAMP=2020_01
ES_HOST=127.0.0.1
ES_PORT=9200
set -e
mv /etc/elasticsearch/elasticsearch.yml /etc/elasticsearch/elasticsearch.yml.orig ++ hostname
sed -e s#@BASE_DIR@#/opt/kaltura#g -e s#@LOG_DIR@#/opt/kaltura/log#g -e s#@HOSTNAME@#video#g /opt/kaltura/app/configurations/elastic/elasticsearch.template.yml
chown -R elasticsearch /opt/kaltura/log/elasticsearch /opt/kaltura/var/lib/elasticsearch
invoke-rc.d elasticsearch restart
'[' '!' -d /usr/share/elasticsearch/plugins/analysis-icu ']'
mkdir -p /opt/kaltura/app/configurations/elastic/populate
chown kaltura /opt/kaltura/app/configurations/elastic/populate
set +e
echo 'Waiting for the ElasticSearch daemon to start...' Waiting for the ElasticSearch daemon to start...
ATTEMPT=0
curl -f 'http://127.0.0.1:9200/_cluster/health?wait_for_status=yellow&timeout=10s'
sleep 5
'[' 0 -eq 22 ']' ++ expr 0 + 1
ATTEMPT=1
curl -f 'http://127.0.0.1:9200/_cluster/health?wait_for_status=yellow&timeout=10s'
sleep 5
'[' 1 -eq 22 ']' ++ expr 1 + 1
ATTEMPT=2
curl -f 'http://127.0.0.1:9200/_cluster/health?wait_for_status=yellow&timeout=10s'
sleep 5
'[' 2 -eq 22 ']' ++ expr 2 + 1

========truncated-output==============

ATTEMPT=16
curl -f 'http://127.0.0.1:9200/_cluster/health?wait_for_status=yellow&timeout=10s'
sleep 5
'[' 16 -eq 22 ']' ++ expr 16 + 1
ATTEMPT=17
curl -f 'http://127.0.0.1:9200/_cluster/health?wait_for_status=yellow&timeout=10s'
sleep 5
'[' 17 -eq 22 ']' ++ expr 17 + 1
ATTEMPT=18
curl -f 'http://127.0.0.1:9200/_cluster/health?wait_for_status=yellow&timeout=10s'
sleep 5
'[' 18 -eq 22 ']' ++ expr 18 + 1
ATTEMPT=19
curl -f 'http://127.0.0.1:9200/_cluster/health?wait_for_status=yellow&timeout=10s'
sleep 5
'[' 19 -eq 22 ']' ++ expr 19 + 1
ATTEMPT=20
curl -f 'http://127.0.0.1:9200/_cluster/health?wait_for_status=yellow&timeout=10s'
sleep 5

after attempt 20. Ctl-D to kill

thecaribbean commented 4 years ago

I noticed: sed: can't read /configurations/base.ini: No such file or directory

so ran

find / -name base.ini

and got it at this location

/opt/kaltura/app/configurations/base.ini

thecaribbean commented 4 years ago

Looking through the installation script

vi install_kaltura_all_in_1.sh

I note the last line is

for an all in one deployment, this needs to be set to true

sed -i 's@(exec_elastic\s=\s)false@\1true@' $APP_DIR/configurations/base.ini

so despite the error Elastic Search is installed and I'm able to log into KMC and upload/server media. Not sure how Elastic Search error will impact operation.

pkExec commented 4 years ago

Tried too, same errors as @thecaribbean on https://github.com/kaltura/platform-install-packages/issues/659#issuecomment-572049513

The first error (elasticsearch not starting) is because: a) $JAVA_HOME is not set for root user. b) Permissions are wrong as per https://stackoverflow.com/a/48390311/1543677 After following the instructions on stackoverflow and manually setting $JAVA_HOME, elasticsearch does start.

The second error (with the sed command in the install script), is because for some reason the $APP_DIR variable is not set. After replacing it with the installation path (/opt/kaltura/app), it finishes without errors.

jessp01 commented 4 years ago

Hi @pkExec, all,

These ENV vars are defined in /etc/profile.d/kaltura-base.sh but of course, they will only be auto sourced when a new session is established, therefore, see this commit: https://github.com/kaltura/platform-install-packages/commit/7d36b82d303bc081b8e6a9119eecd674b016bd6f which should take care of that.

With regards to

Not sure how Elastic Search error will impact operation.

ES is actually only utilised by the esearch service (https://developer.kaltura.com/api-docs/service/eSearch) and, at present time, KMC and KMCng's code do not make use of it so it isn't a critical component for Kaltura CE.

Thanks all, for your reports,

kaltura / platform-install-packages

Installation fail - Ubuntu Xenial [16.04] all-in-one Kaltura server installation #659

Bug report

for an all in one deployment, this needs to be set to true