structureddynamics / Open-Semantic-Framework-Installer

Command line tool used for installing, updating and configuring an Open Semantic Framework instance
http://opensemanticframework.org
45 stars 11 forks source link

Need help with the Drupal installation #15

Open ponsfrilus opened 8 years ago

ponsfrilus commented 8 years ago

I'm testing the OSF installation in a Ubuntu 14.04.4 LTS Virtual Machine. I followed the README but I still get some errors.

Environment

NAME="Ubuntu"
VERSION="14.04.4 LTS, Trusty Tahr"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 14.04.4 LTS"
VERSION_ID="14.04"
HOME_URL="http://www.ubuntu.com/"
SUPPORT_URL="http://help.ubuntu.com/"
BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"
PHP 5.5.9-1ubuntu4.17 (cli) (built: May 19 2016 19:05:57) 
Copyright (c) 1997-2014 The PHP Group
Zend Engine v2.5.0, Copyright (c) 1998-2014 Zend Technologies
    with Zend OPcache v7.0.3, Copyright (c) 1999-2014, by Zend Technologies

Installation process

I've made video of installation process (sorry it's RAW video, so please skip to interesting part instead of waiting downloads and installation complete)

OSF Installation

OpenSementicFramework Initial Installation on Ubuntu 14.04.4 LTS That part went well.

./osf-installer -d --install-osf-drupal -v

OpenSementicFramework install-osf-drupal 1/2 Ends at sh: 1: drush: not found and apt-get install drush does not help.

./osf-installer -d --install-osf-drupal -v | drush

OpenSementicFramework install-osf-drupal 2/2 Trying to Install Drush (https://www.drupal.org/node/1248790 / http://docs.drush.org/en/master/install/) and hitting "https://www.drupal.org/node/1428638" bug (https://www.drupal.org/node/1428638)

Questions

Sorry for the long issue, any help would be appreciated.

fgiasson commented 8 years ago

Hi Nicolas!

  ./osf-installer -d --install-osf-drupal -v

OpenSementicFramework install-osf-drupal 1/2 https://www.youtube.com/watch?v=5ttyiBX9wjA Ends at |sh: 1: drush: not found| and |apt-get install drush| does not help.

Yeah, Drush is not installed via apt-get, but the right version is installed using Composer.

Here is the OSF Installer code that deploys Drush on an instance:

// Install composer
$this->chdir('/tmp/');

$this->exec('curl -sS https://getcomposer.org/installer | php');
$this->mv('composer.phar', '/usr/bin/composer');

// Install Drush
$this->exec('composer global require drush/drush:7.1.0');

$this->ln('/root/.composer/vendor/drush/drush/drush', 
'/usr/bin/drush');

You can try to run these steps directly in your terminal to see if you can notice an issue with these.

My first question would be "How can I properly install drush in my
system ?". First it seems that the repository version just doesn't
work,

No, it won't, and this is why we use the specific one deployed with Composer

that the composer version just want to be ran as standard user
altough the ost-installer drupal installer needs the root's rights
to set up apache2 stuff.

This I am not sure to understand. I always deploy using root with Composer. What is the specific error related to this?

My second question is "How to properly set up OSF in a Virtual
Machine ?"; As drush want to connect to the mysql socket and
localhost heads to the Unix socket (see Forcing 127.0.0.1 instead
of localhost for drush <https://www.drupal.org/node/1428638>) what
would be your recommendation here ?

It has been quite a long time since I deployed OSF on VirtualBox. However, normally what I do is to get a domain name that I (or a client) want for its installation in the future, and I setup the domain in the /etc/hosts file that points to the local host before running the OSF installer. Then I specify that domain name when I install OSF.

In my first attempts, I re-run the |./osf-installer -c
--install-osf -v| because of some errors. Is there a way to clean
up files in order to be able to re-run the installer with
different value ? Or |osf-installer| is asking for few users and
passwords (that can be set in |installer.ini| too) - what would be
the procedure to change them (reinstalling is ok).

The cleanest way is to rerun a new image and restart the OSF installation. This is one area that needs to be improved with the OSF Installer: rollback and auto-cleaning of the installation in case of issues.

In some previous attempts I put some Dollars sign ($) in mysql
password and it seems that was a reason of script failure.

Hoo maybe so, do you remember where exactly? (when the script asks for a password?)

My last question is "Is the CentOS installation more
straight-forward ?"

Should be about the same. I personally use Ubuntu, but many other uses CentOS. But both installation is different in multiple tiny ways but the overall process is about the same.

ponsfrilus commented 8 years ago

HI @fgiasson, thanks for your reply !

I always deploy using root with Composer.

There are no errors but a warning "Do not run Composer as root/super user! See https://getcomposer.org/root for details"

Anyhow my drush seems ok:

# drush core-status version
 Drush version   :  8.1.3 

It has been quite a long time since I deployed OSF on VirtualBox. However, normally what I do is to get a domain name that I (or a client) want for its installation in the future, and I setup the domain in the /etc/hosts file that points to the local host before running the OSF installer. Then I specify that domain name when I install OSF.

Well I will test something similar which can fit to our network limitation. If you can share your VM the next time you try, it would be awesome ;)

The cleanest way is to rerun a new image and restart the OSF installation. This is one area that needs to be improved with the OSF Installer: rollback and auto-cleaning of the installation in case of issues.

Now I get a snapshot of the VM before the installation, so I can try again. If I remember well, the installer complain about already existing file which it can't remove.

Hoo maybe so, do you remember where exactly? (when the script asks for a password?)

Unfortunately I lost the logfile, but I think it was as the database creation part or connection string, regarding the mysql root passwd.

I will try again later when I get some time, but thanks again for your reply !

fgiasson commented 8 years ago

Well I will test something similar which can fit to our network limitation. If you can share your VM the next time you try, it would be awesome ;)

Unfortunately I am not planning to do so on VM anytime soon.

Now I get a snapshot of the VM before the installation, so I can try again. If I remember well, the installer complain about already existing file which it can't remove.

Yes, starting from a clean image is the best way to go for the moment.

Unfortunately I lost the logfile, but I think it was as the database creation part or connection string, regarding the mysql root passwd. I will try again later when I get some time, but thanks again for your reply !

Good thanks!

In mean time, tell if you you can solve your issues and... how :)

ponsfrilus commented 8 years ago

I've tried again with a fresh install. I set my hostname to "osf" and assure myself that /etc/hosts match it. ./osf-installer --install-osf -v runs well.

For the drupal part (./osf-installer -d --install-osf-drupal -v):

Drupal installation path: (current: /usr/share/drupal): 
Drupal's instance's domain or IP: (current: localhost): osf
Drupal's instance's domain or IP: (current: localhost): osf
Drupal's instance's domain or IP: (current: localhost): osf
Drupal's instance's domain or IP: (current: localhost): http://osf/
Drupal's instance's domain or IP: (current: localhost): 
----------------------------------
| Installing Drupal & OSF Drupal |
----------------------------------

dpkg-query: no packages found matching git
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name                                                  Version                                             Architecture Description
+++-=====================================================-===================================================-============-===============================================================================
ii  php5-curl                                             5.5.9+dfsg-1ubuntu4.17                              amd64        CURL module for php5
sh: 1: drush: not found

Git

So regarding OSFInstaller_Ubuntu_14_04.php#L537:

root@osf:/usr/share/osf-installer# dpkg-query -l git
dpkg-query: no packages found matching git
root@osf:/usr/share/osf-installer# git
The program 'git' is currently not installed. You can install it by typing:
apt-get install git

=> no git installed

Drush

Composer was properly installed:

root@osf:/usr/share/osf-installer# composer -V
Do not run Composer as root/super user! See https://getcomposer.org/root for details
Composer version 1.2.0 2016-07-19 01:28:52
root@osf:/usr/share/osf-installer# which composer
/usr/bin/composer

But on my system drush was installed in /home/USER/.composer/vendor/drush/drush/drush instead of /root/.composer/vendor/drush/drush/drush so the symbolic link /usr/bin/drush -> /root/.composer/vendor/drush/drush/drush fails.

So I guess that explain why my install fails. I will try again tomorrow by installing drush and git before running the drupal install.

Cherrs, Nicolas

fgiasson commented 8 years ago

@ponsfrilus I rechecked the code of the OSF installer with this new analysis you did. I think that in some setup the check to install Git and php-curl may lead to not be installed.

Please retry with the following fix (just re-install on a new instance, and this commit should be downloaded automatically by the installer):

https://github.com/structureddynamics/Open-Semantic-Framework-Installer/commit/2f5ce03a7afae7f5ce9a31fc5789dd2921cf6bea

That may be what is at cause for at least one of your issue here.

However, I don't know why that would be the case on your setup. I used the installer last week for deploying another instance on Ubuntu 14.04 without any issues (it was an AWS image created by Cannonical). Don't know, but try that :)

ponsfrilus commented 8 years ago

Thanks Frederick (@fgiasson)! I've made a quick test again with a fresh install.

  1. I run this script to install osf (all good)
  2. I set up and launch the osf-installer with this script

=> few error but ok

What is still not working with the installer

drush still fail to install (error sh: 1: drush: not found). Both with root or sudo, composer install drush in $HOME (which is always the user HOME instead of the root's home):

nbo@osf:~$ sudo echo $HOME
/home/nbo
nbo@osf:~$ sudo -s
root@osf:~# echo $HOME
/home/nbo

Consequently, the symbolic link at line 530 of OSFInstaller_Ubuntu_14_04.php still fail. A good option (but not sure it will work here would be to use ~ or the $HOME environment variable.

For now, my workaround is the following (as in last line of the setup script):

env "PATH=$PATH:/home/nbo/.composer/vendor/drush/drush/" /usr/share/osf-installer/osf-installer -d --install-osf-drupal -v

With the env command the osf-installer is run in a modified environment and drush is found.

That's all for today !

fgiasson commented 8 years ago

@ponsfrilus any update on this on your side?

ghost commented 8 years ago

@fgiasson I've had alot of issues installing on ubuntu.

Script doesn't exist yet for ubuntu 16.04 and the installer for 14.x has issues with the drupal install / composer / drush permissions.

Whilst i've been messing around with it, i've not come-up with an easy fix to the problems i've found along the way.

Now trying on CentOS hoping it's a little quicker to get a working outcome. nb: the hacks by @ponsfrilus certainly helped, yet, i think a fix is needed.

Re: Environment; i'm using a local, virtualised environment for testing.

The last piece i got to, was trying to get the /ws/ end-point configured; which i failed to do. I suspect it had something to do with failed install attempts, due to the install script issues.

fgiasson commented 8 years ago

@mediaprophet great thanks for reporting.

About your environment, were is it virtualised? (what is the vertualization system you are using?)

I Installed the whole thing on Ubuntu 14.x a few weeks back without any issues related to the OSF for Drupal deployment, so this is why I am asking. If I can reproduce locally, I will be able to fix this issue for good.

No 16.04 is not currently supported. It may work out of the box, but in my experience when we upgraded from 12.04 to 14.04, things changed in Ubuntu which needed some attention. Should be the same here.

Now trying on CentOS hoping it's a little quicker to get a working outcome. nb: the hacks by @ponsfrilus certainly helped, yet, i think a fix is needed.

Have you tried yet? Is it working properly?

The last piece i got to, was trying to get the /ws/ end-point configured; which i failed to do. I suspect it had something to do with failed install attempts, due to the install script issues.

Retry with a new vanilla server. The web services should be working without any issues I guess (since this current error is related to the OSF for Drupal deployment). If all the unit tests pass, then it means that the web service endpoints works properly and that you can use the PHP or Clojure APIs to interact with the endpoints, or any language you want if you query the endpoints directly.

ghost commented 7 years ago

Hi @fgiasson - yes, install environment is virtualised.

Re: Ubuntu 14.0X: I find the install requires SUDO and I think DRUSH has issues with SUDO?

Can we figure out how to create a 'few clicks' based install process? I've tried again today, to follow the install process listed in the .md - still doesn't work as described.

What's the default environment? I really don't care if its ubuntu or centos; i just want to download the source packages, run some scripts provided by your repo, using virtual-box, and set it up to test it out and start making a bigger mess (so we can start to talk about more interesting problems ;)

I know this is basic 101 stuff; Yet, I've found it difficult (time-consuming) to get past this initial process.

fgiasson commented 7 years ago

Hi Timothy,

Hi @fgiasson https://github.com/fgiasson - yes, install environment is virtualised.

Re: Ubuntu 14.0X: I find the install requires SUDO and I think DRUSH has issues with SUDO?

Can we figure out how to create a 'few clicks' based install process? I've tried again today, to follow the install process listed in the .md - still doesn't work as described.

What's the default environment? I really don't care if its ubuntu or centos; i just want to download the source packages, run some scripts provided by your repo, using virtual-box, and set it up to test it out and start making a bigger mess (so we can start to talk about more interesting problems ;)

I know this is basic 101 stuff; Yet, I've found it difficult (time-consuming) to get past this initial process.

I agree with that. However due to the nature of the installation, many things can break if things are just a little different from one system to another. Right now the best we have is installer and is known to work for the supported versions with the AWS AMI provided by Canonical.

Other than that, I guess we would have to support a complete debian package that would deploy everything the right way automatically, but I am not enough of a devop to do this properly :)

Thanks,

Fred