Batch script execution failed after yum update

[ifproject]

Hello all

I tried to update Kaltura with yum a few days ago. Update went smooth and no error reported but since then the batch script execution is failing and of course no new media is processed.

If I restart Batch from control panel monitoring it return

Execution failed - Pending restart

and it never restart.

Had a look in shell

/etc/init.d/kaltura-batch status Service Batch Manager isn't running [ FAILED ]

/etc/init.d/kaltura-batch start Starting Batch Manager.... Batch Manager started with PID [ OK ]

No PID is returned after start and batch manager remain stopped.

Could you help with this issue? I worked a lot during the past few months to make this Kaltura installation work as needed and now no media is processed due to the batch script failure.

Many thanks!

[jessp01]

Hello, Did you re-run the config scripts post the upgrade as described here? https://github.com/kaltura/platform-install-packages/blob/master/doc/install-kaltura-redhat-based.md#upgrade-kaltura

I suspect not:) if not, please do and if you're still having issues, post at forum.kaltura.org describing exactly what they are and providing relevant log entries. For batch specifically, look at /opt/kaltura/log/kaltura_batch.log but in general, use:

kaltlog

which will output errors to STDOUT and then look at the relevant logs looking at the lines leading to the error.

[ifproject]

Yes I did re-run the config script as stated and apparently all was ok, no error reported.

Below the last lines of /opt/kaltura/log/kaltura_batch.log

tail kaltura_batch.log 2016-02-17 14:52:36 [0.000000] [641611149] [1] [BATCH] [KSchedulerConfig->load] NOTICE: loading configuration /opt/kaltura/app/configurations/batch at 1453830755 PHP Fatal error: Uncaught exception 'Zend_Config_Exception' with message 'Section 'videohost.netv.it' cannot be found in /opt/kaltura/app/cache//batch/config.ini' in /opt/kaltura/app/vendor/ZendFramework/library/Zend/Config/Ini.php:150 Stack trace:

0 /opt/kaltura/app/batch/scheduler/KSchedulerConfig.class.php(59): Zend_Config_Ini->__construct('/opt/kaltura/ap...', 'videohost.netv....', true)

1 /opt/kaltura/app/batch/scheduler/KSchedulerConfig.class.php(42): KSchedulerConfig->load()

2 /opt/kaltura/app/batch/scheduler/KGenericScheduler.class.php(88): KSchedulerConfig->__construct('/opt/kaltura/ap...')

3 /opt/kaltura/app/batch/scheduler/KGenericScheduler.class.php(70): KGenericScheduler->loadConfig('/opt/kaltura/ap...')

4 /opt/kaltura/app/batch/KGenericBatchMgr.class.php(37): KGenericScheduler->__construct('/usr/bin/php', '/opt/kaltura/ap...')

5 {main}

thrown in /opt/kaltura/app/vendor/ZendFramework/library/Zend/Config/Ini.php on line 150

Checking /opt/kaltura/app/cache//batch/config.ini I couldn't find really the missing section (Section 'videohost.netv.it' ). Could you please tell me where this should be in the script?

I could also posto on forum.kaltura.org if necessary.

Thank you again.

[jessp01]

The error is here:

PHP Fatal error: Uncaught exception 'Zend_Config_Exception' with message 'Section 'videohost.netv.it' cannot be found in /opt/kaltura/app/cache//batch/config.ini' in /opt/kaltura/app/vendor/ZendFramework/library/Zend/Config/Ini.php:150 Stack trace:

In /opt/kaltura/app/configurations/batch/batch.ini, you should have: [$YOUR_BATCH_HOSTNAME : mainTemplate]

This is generated from /opt/kaltura/app/configurations/batch/batch.ini.template

which has: [@INSTALLED_HOSNAME@ : mainTemplate]

That is done with: /opt/kaltura/bin/kaltura-batch-config.sh sed -i "s#@INSTALLED_HOSNAME@#hostname#" -i $BATCH_MAIN_CONF

So, you need to check why that does not happen correctly when you run this script.

Is your hostname when trying the command: $ hostname correct and resolvable?

May the source be with you,

Jess Portnoy

Kaltura Connect Virtual Summits 2015 - now on VOD! [connect.mediaspace.kaltura.com]

On Wed, 17 Feb 2016, ifproject wrote:

Yes I did re-run the config script as stated and apparently all was ok, no error reported.

Below the last lines of /opt/kaltura/log/kaltura_batch.log

tail kaltura_batch.log 2016-02-17 14:52:36 [0.000000] [641611149] [1] [BATCH] [KSchedulerConfig->load] NOTICE: loading configuration /opt/kaltura/app/configurations/batch at 1453830755 PHP Fatal error: Uncaught exception 'Zend_Config_Exception' with message 'Section 'videohost.netv.it' cannot be found in /opt/kaltura/app/cache//batch/config.ini' in /opt/kaltura/app/vendor/ZendFramework/library/Zend/Config/Ini.php:150 Stack trace:

0 /opt/kaltura/app/batch/scheduler/KSchedulerConfig.class.php(59): Zend_Config_Ini->__construct('/opt/kaltura/ap...',

'videohost.netv....', true)

1 /opt/kaltura/app/batch/scheduler/KSchedulerConfig.class.php(42): KSchedulerConfig->load()

2 /opt/kaltura/app/batch/scheduler/KGenericScheduler.class.php(88): KSchedulerConfig->__construct('/opt/kaltura/ap...')

3 /opt/kaltura/app/batch/scheduler/KGenericScheduler.class.php(70): KGenericScheduler->loadConfig('/opt/kaltura/ap...')

4 /opt/kaltura/app/batch/KGenericBatchMgr.class.php(37): KGenericScheduler->__construct('/usr/bin/php', '/opt/kaltura/ap...')

5 {main}

thrown in /opt/kaltura/app/vendor/ZendFramework/library/Zend/Config/Ini.php on line 150

Checking /opt/kaltura/app/cache//batch/config.ini I couldn't find really the missing section (Section 'videohost.netv.it' ). Could you please tell me where this should be in the script?

I could also posto on forum.kaltura.org if necessary.

Thank you again.

— Reply to this email directly or view it on GitHub.[AFBqvRDWqrWEwVgppLu214a9ZB13HYiBks5plHc6gaJpZM4Hb7lH.gif]

[ifproject]

Jess

the hostname is resolvable and correct.

I decided to run again the post install scripts but before I did a yum update to 11.8.0 and then post install scripts and reinserted all config data from scratch. No single error reported and "Setup completed successfully!"

Checked in shell for batch status

[root@videohost kaltura]# /etc/init.d/kaltura-batch status Batch Manager running with PID [ OK ]

But Now I cannot access the admin console

An error occurred (error code: Runtime:0)

Kaltolog in http section reported

PHP Warning: SimpleXMLElement::__construct(): Entity: line 1: parser error : Start tag expected, '<' not found in /opt/kaltura/app/admin_console/lib/Kaltura/Client/UserService.php on line 178, referer: http://videohost.netv.it/admin_console/index.php/

Sanity test return all ok until

[Check memcached daemon is started by Monit] [PASSED, RC: 0] - [52.919132422] [check daemon memcached init status] [PASSED, RC: 0] - [.003007207] [kaltura-html5lib ver in KMC config.ini] [PASSED, RC: 0] - [.056952055] [kaltura-kdp3 ver in KDP3 config.ini] [PASSED, RC: 0] - [.010422104] [kaltura-kmc ver in KMC config.ini] [PASSED, RC: 0] - [.014156152] [Get KMC SWFs] [PASSED, RC: 0] - [.120932115] [check_testme_page] [FAILED, RC: 1] - [.032193084] [check_kmc_index_page] [PASSED, RC: 0] - [.013382364] [check_admin_console_index_page] [PASSED, RC: 0] - [.120580998] [check_studio_index_page] [PASSED, RC: 0] - [.016036960] [check_clipapp_index_page] [PASSED, RC: 0] - [.371713134] [Create Partner] [FAILED, RC: 255] - [.070936344] Partner creation failed. I will skip all tests that require it. [Red5 file upload] [FAILED, RC: 2] - [.416108913]

Any other command I should run to be able to repair and acces the admin ?

Thanks again! Sil

[ifproject]

Runned again kaltura-config-all.sh and now the admin console is accessible again, but monit shows again

batch Execution failed

So initial problem is not solved. :(

Any idea how to proceed other than uninstall kaltura?

You said

"So, you need to check why that does not happen correctly when you run this script."

Anything I can do to check this and understand why this part of the script is not working? sed -i "s#@INSTALLED_HOSNAME@#hostname#" -i $BATCH_MAIN_CONF

Any help is greatly appreciated.

[jessp01]

Hi,

Since you say hostname returns a resolvable, correct hostname, what do you actually see in /opt/kaltura/app/configurations/batch/batch.ini: [.* : mainTemplate]

'.*' should be your hostname. What is actually there?

[ifproject]

Hi Jess

the string is correct indeed...

/opt/kaltura/app/configurations/batch/batch.ini line 23

[videohost.netv.it : mainTemplate] id = 31901 name
= Default Scheduler

and hostname return the same hostname

[root@videohost ~]# hostname videohost.netv.it

Thank you very much for taking the time to help with this issue.

[ifproject]

Jess, excuse me if I keep posting here but I believe this could reasonable be a bug. It looks to me no PID is created and no lock file starting kaltura-batch.

[root@videohost ~]# /etc/init.d/kaltura-batch status Batch Manager running with PID [ OK ] [root@videohost ~]# vi /etc/init.d/kaltura-batch [root@videohost ~]# /etc/init.d/kaltura-batch force-stop Stopping Batch Manager.... Killing Batch Manager with PID and related workers [ OK ] [root@videohost ~]# /etc/init.d/kaltura-batch status Batch Manager running with PID [ OK ]

[jessp01]

@ifproject - it is not a general issue. Like I wrote in https://github.com/kaltura/platform-install-packages/issues/525 I just installed 11.8.0 from scratch and had no problem at all. Our CI also does this on daily basis and reported no issues.

Check whether /opt/kaltura/app/cache//batch/config.ini is somehow different than what you have in /opt/kaltura/app/configurations/batch/batch.ini, if so, try to clear cache with:

find /opt/kaltura/app/cache -type f -exec rm {} \;

and restart the daemon. If it is the same, post to forum.kaltura.org and lets take it there.