New Appliance: ZoneMinder

[ghost]

Completed work on the ZoneMinder Application (no artwork yet). Repo is here https://github.com/MPTMG/ZoneMinder. Ready to be built and tested. Got a question for the experts here though.

Question 1: Application requires timezone to be set in /etc/php5/apache2/php.ini During the initial setup (inithooks phase) will cat /etc/timezone reflect the timezone of the installation? If so I can script it to place the correct/current timezone in /etc/php5/apache2/php.ini. I can see UTC potentially being used though, so should I prompt for the timezone during first boot? This would require the user to enter a correctly formatted time zone "America/New_York". Let me know how you want this handled, it is easy to script either way.

[DocCyblade]

@MPTMG - I am not "an expert" but I'll put in my 2 cents: Leaving it to UTC at build time, then a first boot inithook would work. You could to ensure correct format use a python script to make a pretty list to select from then call it from bash script. I am pretty certain that the time zone is not set on the system, as thats one of the things I do when I install a TKL server.

I am testing a new version of confconsole that allows plug-ins of sorts for allowing first boot and after the fact changing of system settings. Timezone would be a good one to do!

Also when I have time later tonight, I'll give it a build and look over, however Jeremy should be back at it in in 6-7 hours from now if my math is right.

@JedMeister - Setting the time zone in general during first boot using the new confconsole would be a good idea if not already on the road map.

[ghost]

@DocCyblade is there any objection to simply using tzselect command? It would not have the UI as it is just simply a command line tool, but I could call it on firstboot inside my own script and then make the necessary changes to /etc/php5/apache2/php.ini. I could always prompt with ui and "hit enter to run tzselect to set timezone" so users would not be suddenly surprised by a prompt. Naturally this all only applies to the interactive startup.

I will look at the complexity of some sort of python drop down for this, but it seems like a lot of work to avoid calling tzselect.

The confconsole plugin would be great as well if it had a tie in to secondary scripts. I.E. When timezone is set it automatically calls a script if it exists. Hence any app that required a timezone setting could use an overlay to modify the script called and perform necessary actions.

I.E. tzselect is already an interactive command that prevents user error in entering timezone. Would accomplish two things at once.... Setting correct system timezone and setting timezone for ZoneMinder in /etc/php5/apache2/php.ini.

[DocCyblade]

I am thinking that during headless deployments like the hub, the initscripts look for a preseed file since it's not interactive, so initscipts need to look for this file and add its variables, then the script would not prompt and use the preseeded variables.

Look at any of the python scripts in common that change a password like for the root user in MySQL as an example.

I would say as long as a preseed and or default setting is used for headless deployments, and it prompts you if there is no preseed file that would be ok.

@JedMeister should be able to expand further as my knowledge is limited. I am actually working on an first book That changed password and email for Mayan and I am still learning.

[DocCyblade]

Also if I remember correctly, I think they use a wrapper for dialog. that allows you to create custom dialogs pretty easy.

[ghost]

I have been using wrapper for dialog for Mumble, but did not have a good example for yes/no dialog. I basically have been adapting previous implementations by others for my own purposes. I got a little creative and created a yes/no dialog (adapting from documentation making it work for TKL) for selecting timezone via tzselect or proceeding with default of UTC. I am still including the try catch block from other implementations even though I am not passing parameters (I suppose it is nice to have it if I ever decide to pass a parameter). Anyhow timezone selection is implemented now and in the current master branch on github. I have been having to do a bit of learning lately as my programming experience has been c++, c#, java.... and now I have to get smart on python.

Once again thanks @DocCyblade for the quick responses. Oh and as far as preseed goes I am just using UTC as my preseed configured in conf.d/main and avoiding any unnecessary inithooks if it is just going to be a default value.

[JedMeister]

Firstly apologies on my radio silence. I really need to put some time aside to look over all your code properly. I had hoped to do that this week, but I'm almost out of week now and not sure if I'll get time today. Please be assured that I haven't forgotten about you nor am ignoring you...

Anyway, directly relevant to this thread. Here's another thought (not sure if it's a good one) but you could use the default dpkg interface for setting the tzdata. I.e.: dpkg-reconfigure tzdata. Ideally it would be nice if you could skip it (i.e. the yes/no you've been disccusing first).

Also regarding inithooks, I haven't checked you code on this yet, but in case you haven't already, if you add new non standard inithook options, then they will need logic to provide a default if the instance is preseeded but that particular value is not set. This is really important for the headless builds and servers launched from the Hub.

You could use the outmail (outgoing email address) from the Bugzilla appliance as an example of that. See the bash and python scripts.

[ghost]

I will take a look at bugzilla appliance for preseed info after the holiday. No rush on looking the code over I am just trying to contribute a bit as I think turnkey is useful for the masses. Hopefully when these are finished and commensurate with your standards your group contacts the projects and gets a link posted to drive some traffic your way.

[ghost]

I took a look at the code for outmail in the bugzilla appliance. I think this use case is a bit different, but I have taken your advice on using dpkg-reconfigure tzdata which does bypass the yes/no requirement nicely. Essentially pre-seed will always default to UTC (chose to do this in conf.d/main as it is just a php config setting) which incidentally is also what default system timezone is. Perhaps pre-seed is a bad way of putting it and may be causing confusion. I have hard coded UTC into the php config file. If the user wants to "pre-seed (given my understanding of definition of pre-seed in this context)" they can do so through the /etc/inithooks.conf file and everything will be setup the way they want on first boot.

As far as pre-seeding all values via this method:

cat>/etc/inithooks.conf<<EOF
export ROOT_PASS=supersecretrootpass
export DB_PASS=supersecretmysqlpass
export APP_EMAIL=admin@example.com
export APP_PASS=webappadminpassword
export SEC_ALERTS=admin@example.com
export SEC_UPDATES=FORCE
export HUB_APIKEY=SKIP
EOF

I have updated the Mumble Appliance and the ZoneMinder appliance to accept pre-seeds via this method as well. I assumed I am at liberty to create my own variable names. I.E. export ICE_SECRET, export MUMBLE_PW, export PHP_MAPW for Mumble and export TIME_ZONE for ZoneMinder. Obviously the firstboot scripts would pass a variable by the same name to the python call to essentially bypass any manual entry of these values because the python script looks for a parameter in order to bypass the prompt in the event it is pre-seeded.

On ZoneMinder the pre-seed non-interactive for DB password is all done without any input from user (I think it is what you are looking for using a "random value"). On Mumble there are no non-interactive setup requirements. If icesecretwrite is not set or left blank in the config file it will just not allow connections (essentially secure). The MyMumb web interface uses icesecretwrite as the password, which if not set will not allow access so essentially no login is possible without configuration. For phpMumbleAdmin the password field and the icesecretwrite configuration will be left unconfigured. So until interactively configured no access is possible as well. As currently setup one could pre-seed these values via /etc/inithooks.conf. 30turnkey-init-fence would take over before any of these would even matter anyhow (headless) on the Mumble App.

[JedMeister]

More apologies on my lack of input lately... I haven't gotten as far as fast with stuff as i might have hoped...

That all sounds reasonable to me. However I think that there is some misunderstanding. Apologies that it's not explicitly noted very well.

The non-interactive inithooks variable names need to remain consistent with the ones noted in the pre-seeding process (included in your last post). Whilst the method you note (using any variable name you like) will certainly work; to minimise overhead for many of our hosting partners (and our own Hub) we aim to keep them standardized to that list of names.

Ideally we like to keep the interactive inithooks to a minimum too; however if there is clear value in asking user's additional questions, then that's fine. But the the non-interactive inithooks need to only require the current defaults. Optional extras

To use your Mumble appliance (nice work BTW, I've been testing it out) as a practical example, it's great that each of the different Mumble related passwords can be interactively set individually. But for preseeding, we need to only provide one (APP_PASS). That's what I was alluding to when I posted:

new non standard inithook options [...] will need logic to provide a default if the instance is preseeded but that particular value is not set. This is really important for the headless builds and servers launched from the Hub.

in reference to Bugzilla (sorry I didn't provide that context clearly enough).

If you have time and inclination to look at that, then great. Otherwise we can take care of it. I was hoping to add your new appliances to the library this year (before the next v14.2 release) however, this year is now looking pretty unlikely... Sorry again that this process has taken so long.

[ghost]

I have edited the Mumble Appliance to ensure that all non-standard inithooks pass APP_PASS as the parameter thus ensuring that they can be pre-seeded via the standard variable set outlined here:

cat>/etc/inithooks.conf<<EOF export ROOT_PASS=supersecretrootpass export DB_PASS=supersecretmysqlpass export APP_EMAIL=admin@example.com export APP_PASS=webappadminpassword export SEC_ALERTS=admin@example.com export SEC_UPDATES=FORCE export HUB_APIKEY=SKIP EOF

I will work on the Zoneminder appliance shortly. That still leaves the question of how to pre-seed the timezone if I am only able to use the standard variable list. Sorry if any of this is trivial to you, I am new to the TKL game and learning as I go.

[JedMeister]

You rock! :+1:

Preseeding timezone is a really good question. I wonder if we could somehow use geolocation to guess it by default when preseeding?

PS nothing is trivial if you don't know! I have no issue taking time to assist those that help themselves. You've been really responsive and also really patient and I've been really busy and slow to respond. TBH, with minimal guidance and sub-par docs, you've done as good (if not better) job than I would have done a few years ago...

[ghost]

We can use geolocation via IP address. Quite simply we can query via curl http://ip-api.com/json to get a json which will include current timezone based off IP. grep and sed could reduce the json to just the timezone and voila use this value for setting the time. Let me know if this is acceptable to you and I will implement as part of the inithooks. Given this option do you want to setup a non-interactive inithook to set timezone instead of doing it manually or leave both options? My personal opinion is just switch it to a non-interactive inithook. I would still leave UTC setup in conf.d so application would be operational even if the timezone was for some reason not detected properly (i.e. no internet access during setup). A user could then run turnkey-init later if for some reason the timezone was wrong.

[ghost]

Went ahead and changed the init scripts for ZoneMinder to make align them with the changes @qrntx made to my Mumble init scripts (everything in python no additional calls from python). Made the whole process non-interactive with timezone auto detected as explained above. Will commit the code later today after I can thoroughly test it. FYI to @qrntz I saw you pulled the ZoneMinder code and don't want you working on anything I may have already fixed regarding init scripts.

[a3s7p]

@MPTMG I faced a strange bug which won't let me build ZoneMinder with my changes so I held off pushing for now. I will try to take another look at it soon.

[ghost]

@qrntz I had a problem when I added the back ports repo and tried to install from the repo. I ended up downloading the ZoneMinder package directly and installing it (Would Not Build Installed From Backports). I then added the repo in conf.d main. With the back ports repo added the user would be able to update ZoneMinder via apt. Your problem arise in a similar situation?

[ghost]

Take another look when the opportunity presents itself. I have committed changes and tested and everything is functional. I will get the artwork finished tomorrow and clean up the readme.

[JedMeister]

A bit of a quick update.

I did think that I was almost finished with this, but I've hit a bit of a roadblock. :cry:

I'll merge what I have shortly, but for now you can see what I've done in my 14.2-update dev branch

Things I've done so far:

• moved the install back to an apt install (rather than downloading and using dpkg); my suspicion is that they uploaded a broken package at some point, but it's now fixed. I also pinned it so that you don't need to use -t jessie-backports (see relevant commit).
• enable user authentication by default (see relevant commit).
• refactored inithooks to regenerate the zm DB password non-interactively in line with our other appliances - leverages the bin/mysql.py inithook that is included in all LAMP apps via common. It also regenerates the ZM_AUTH_HASH_SECRET value (see relevant commit)

The only thing I'm stuck on is setting the ZM 'admin' user password. It's admin by default, but we avoid default passwords because of the inherent security implications. All my googling suggests that it's hashed with MD5, but it's obviously not just an MD5 hash, as I've tried that and it doesn't work. I passed it on to one of our other devs who has been pretty handy with these sort of things but he is stumped too. So I'm going to need to get someone else to look at that one. And the guy I've got in mind is already flat out...

[ghost]

I'll see if I can get someone on the Zoneminder project to respond about exactly how passwords are stored. If the artwork is still needed I can get it to you as well.

[ghost]

@JedMeister @DocCyblade @qrntz @lamby

Thought I would tag everyone on this. Password issue seems to be a non issue and executing a few simple MYSQL commands can get it setup at boot. Password hash is automatically taken care of by PASSWORD function. Reference the following thread.

https://askubuntu.com/questions/110841/how-to-reset-zoneminder-password#259452

I just cloned and built the turnkey repo for zoneminder and executed the appropriate commands via MySQL and it worked perfectly including automatically hashing the password. Naturally I executed the commands post install and config, but it should work during first boot as well.

[JedMeister]

Nice detective work @MPTMG! You rock! I can't believe it's that simple... I'll sort that ASAP, hopefully by early next week at the latest.

[JedMeister]

FWIW, I just added a logo: https://github.com/turnkeylinux-apps/zoneminder/blob/master/.art/logo.png

[ghost]

On a side note there is an easier way for retrieving timezone via api call that does not require json parsing. I don't know if the API changed between when I first wrote the code or I just missed the simpler way to retrieve the required data. Doing it the simpler way would get rid of a dependency specifically -- jq #For Parsing JSON on command line for TIMEZONE

From zoneminder/overlay/usr/lib/inithooks/firstboot.d/20timezone This: "More complex version getting entire json" curl http://ip-api.com/json > /opt/timezonedetected TIMEZONE=$(cat /opt/timezonedetected | jq -r '.timezone')

Simpler Version: "Get just timezone field via api" TIMEZONE=$(curl http://ip-api.com/line/?fields=timezone)

Figured I would mention this if you wanted to integrate the changes considering it is a quick copy and paste replacement and when the code is merged would reflect best method for getting the timezone only.

[ghost]

Getting a build error after cloning current repo.

W: Failed to fetch http://httpredir.debian.org/debian/dists/jessie-backports/main/binary-amd64/Packages Hash Sum mismatch

E: Some index files failed to download. They have been ignored, or old ones used instead. /usr/share/fab/product.mk:476: recipe for target 'build/stamps/root.patched' failed make: *** [build/stamps/root.patched] Error 100

It appears that it was a connectivity issue for this on my end.

[ghost]

Found error on build in enabling authentication. No DB specified. might be as easy as adding a zm.Config instead of just config. Will edit and get back to you.

# Enable authentication (i.e. force users to log in to ZM)
$MYSQL_BATCH -e "update Config set Value=1 where Id=3 and Name='ZM_OPT_USE_AUTH';"

# Enable authentication (i.e. force users to log in to ZM)
$MYSQL_BATCH -e "update zm.Config set Value=1 where Id=3 and Name='ZM_OPT_USE_AUTH';"

my sql is rusty though. I will figure it out in a bit.

[JedMeister]

Found error on build in enabling authentication. No DB specified.

Oops... Thanks for checking and reporting. You're right, it should be:

# Enable authentication (i.e. force users to log in to ZM)
$MYSQL_BATCH -e "update zm.Config set Value=1 where Id=3 and Name='ZM_OPT_USE_AUTH';"

I really should have done a final test build.

[JedMeister]

I also just noticed that there's an Apache warning at startup about overlapping script alias in 000-default. We should probably get rid of that. This is a little dirty (leaves multiple blank lines) but acceptable:

sed -i "/ScriptAlias/d" /etc/apache2/sites-available/000-default.conf

Or perhaps it'd be better to comment it out? Something like this:

SITE=/etc/apache2/sites-available/000-default.conf
sed -i "/ScriptAlias/ s|^|# overlaps with ZM script alias\n#|" $SITE

[JedMeister]

Also re zoneminder/overlay/usr/lib/inithooks/firstboot.d/20timezone (i.e. https://github.com/turnkeylinux/tracker/issues/735#issuecomment-318651111) - yeah, let's implement that too.

I've still got a bit of a backlog of builds which will need to be done in smaller batches so it'll probably be early next week before I actually do the proper build of this. I wouldn't like to do any major refactoring this close to publishing (unless of course some serious bug comes to light - but hopefully that's unlikely).

But any minor polishing before we release is good! :smile:

[JedMeister]

I've just applied those and opened a pull request. I haven't merged it yet, just in case you find anything else over the weekend.

If you do and manage to fix them, just stack any new commits on top of mine and open a new PR (mine will auto merge then when I merge yours).

You probably already know how, but just in case; here's what I'd do:

cd zoneminder
git remote add jeremy https://github.com/JedMeister/ZoneMinder.git
git fetch jeremy
git checkout -b polishing -t jeremy/14.2-update

Then just push the polishing branch back to GH and open a new PR against the TurnKey repo.

[JedMeister]

Hey @MPTMG - I'm really sorry to have to do this, but we're going to have to wait for v15.0 to release this. :cry:

Personally, I'm a bit gutted because we were so close (I have built the official ISO ready for release). But the security issues that have come to light (as noted by Chris here look a bit nasty. So we'll have to hold off.

Chris is deeply involved with Debian, so if an updated (security bug free) version can get into stretch-backports, then he's the man for the job! :)

[JedMeister]

Seeing as this one was so close for v14.2 but got missed out in the end, we should make a push for v15.0! Zoneminder is in stretch-backports.

I haven't extensively tested (actually haven't tested at all) but I have updated the packages so it builds.

[JedMeister]

:point_up: @MPTMG - I've just opened a pull request "V15.0 update". If you get a chance and want to test it out, that'd be awesome! :smile:

[ghost]

I will test today and if I find anything that needs to be adjusted I will make the necessary changes.

[JedMeister]

Would it be easier if I merged my changes so you could just pull from TurnKey?

[ghost]

Yeah merge them.

[JedMeister]

Done.

Because of the comment, this issue got auto-closed. I'll reopen it until you've confirmed we're all good. If there are changes required, please issue your own PR, or if it's super simple, please just post and I can take care of it.

[ghost]

I will do the screenshots after testing.

[ghost]

Will submit a pull request. Added the art fixed a config error in php.ini (referenced 5 instead of 7 php.ini location) and "typo stuff" in timezone firstboot. 100% functional now.

[ghost]

You should be able to see the changes and merge when reviewed.

[JedMeister]

Brilliant man. Thanks for that.

I've just reviewed your changes and they looked awesome so I've merged. I'll do a final build and test today so it can be part of the first batch of the v15.0 release (I was hoping we could do that this week, but again it looks like it'll be delayed as I'm waiting on my colleague Alon to publish the ISOs I've built).

I don't have any IP cameras (or spare hardware that I could use with a webcam) so won't be able to do a full test, but I'll at least test that the webUI works as expected.

[JedMeister]

Closing this issue as ZoneMinder has finally been released :tada: :clinking_glasses: (bit slow on the update to this issue...):

https://www.turnkeylinux.org/zoneminder