search

cccs-web / core

CCCS' customized django web application
4 stars 11 forks source link

help to complete server redeployment / migration #97

Closed cccs-ip closed 10 years ago

cccs-ip commented 10 years ago

I stopped at the point of creating a cccs virtual wrapper:

mkvirtualenv cccs

and then cloning in Mezzanine. Here's the full history:

ubuntu@ip-10-183-58-29:~$ history
    1  sudo apt-get install libblas3gf libc6 libgcc1 libgfortran3 liblapack3gf libstdc++6 build-essential gfortran libatlas-dev python python-all-dev gcc g++ libblas-dev liblapack-dev libevent-dev
    2  sudo apt-get update --fix-missing
    3  sudo apt-get update
    4  sudo apt-get upgrade
    5  sudo apt-get install libblas3gf libc6 libgcc1 libgfortran3 liblapack3gf libstdc++6 build-essential gfortran libatlas-dev python python-all-dev gcc g++ libblas-dev liblapack-dev libevent-dev
    6  sudo ln -s /usr/lib/x86_64-linux-gnu/libjpeg.so /usr/lib
    7  sudo ln -s /usr/lib/x86_64-linux-gnu/libfreetype.so /usr/lib
    8  sudo ln -s /usr/lib/x86_64-linux-gnu/libz.so /usr/lib
    9  sudo apt-get install binutils libproj-dev gdal-bin libgeo-proj4-perl libjson0-dev
   10  sudo apt-add-repository ppa:ubuntugis/ppa
   11  sudo apt-get update
   12  sudo apt-get install postgresql-9.3-postgis-2.1
   13  sudo apt-get install postgresql-server-dev-9.3
   14  locale -a
   15  sudo locale-gen en_US
   16  sudo locale-gen en_AU
   17  sudo locale-gen en_AU.utf8
   18  sudo update-locale
   19  sudo dpkg-reconfigure locales
   20  sudo apt-get install language-pack-en
   21  sudo apt-get install language-pack-fr
   22  sudo apt-get install language-pack-de
   23  sudo apt-get install language-pack-ru
   24  sudo apt-get install language-pack-cn
   25  sudo apt-get install language-pack-zh
   26  sudo apt-get install language-support-fonts-zh
   27  sudo apt-get install libcurl4-gnutls-dev libexpat1-dev gettext libz-dev libssl-dev
   28  sudo apt-get install git
   29  wget http://python-distribute.org/distribute_setup.py
   30  sudo python distribute_setup.py
   31  which easy_install
   32  sudo easy_install pip
   33  which pip
   34  sudo apt-get install virtualenvwrapper
   35  sudo adduser aaron
   36  sudo su aaron
   37  sudo adduser paul
   38  sudo adduser webcore
   39  visudo
   40  sudo su root
   41  sudo su webcore
   42  history
webcore@ip-10-183-58-29:~$ history
    1  cd
    2  ls
    3  mkvirtualenv cccs
    4  git clone https://github.com/cccs-web/mezzanine
    5  ls
    6  deactivate
    7  history

I will load in our ssh credentials in a moment... At that point, I'd like your help to finish the rest of the site setup and migration.

cccs-ip commented 10 years ago

New server is up at http://migrate.crossculturalconsult.com.

pwhipp commented 10 years ago

No worries - just to confirm: no problems so far?

cccs-ip commented 10 years ago

no problems so far. i just have no idea how to move 'production' vs. 'staging'...

cccs-ip commented 10 years ago

p.s. we can move over the corresponding nginx settings; after it's all set up, i'll switch over all the dns settings

pwhipp commented 10 years ago

There is no need to do anything with dns, the elastic ip can be reassigned. On 10/08/2014 7:23 AM, "cccs-ip" notifications@github.com wrote:

p.s. we can move over the corresponding nginx settings; after it's all set up, i'll switch over all the dns settings

— Reply to this email directly or view it on GitHub https://github.com/cccs-web/core/issues/97#issuecomment-51699014.

cccs-ip commented 10 years ago

Our IP is also blocked by some lists (in addition to the domain) so I figured I'd start afresh.

pwhipp commented 10 years ago

no worries. As long as you don't mind reconfiguring the DNS and potentially losing page rank we can change IP address as often as you like. It will upset the ssh logins a little but you just need to clear known_hosts of the old entries to deal with that.

On 10 August 2014 07:36, cccs-ip notifications@github.com wrote:

Our IP is also blocked (in addition to the domain) so I figured I'd start afresh.

— Reply to this email directly or view it on GitHub https://github.com/cccs-web/core/issues/97#issuecomment-51699341.

cccs-ip commented 10 years ago

i am not too worried about the page rankings for the moment, and removing known_hosts is a trivial matter that we'd also have to deal with even if not changing IPs.

cccs-ip commented 10 years ago

just to be clear: I am interested only in moving our django-based sites:

we also need to migrate a couple of tools:

mayan.crossculturalconsult.com jenkins.crossculturalconsult.com

(anything I am missing?)

pwhipp commented 10 years ago

respect.crossculturalconsult.com

I'll document setting up the multiple sites/users for future reference.

On 10 August 2014 12:02, cccs-ip notifications@github.com wrote:

just to be clear: I am interested only in moving our django-based sites:

production.crossculturalconsult.com [which will become ' crossculturalconsult.com'] staging.crossculturalconsult.com abadi.crossculturalconsult.com mayan.crossculturalconsult.com

(anything I am missing?)

— Reply to this email directly or view it on GitHub https://github.com/cccs-web/core/issues/97#issuecomment-51703874.

cccs-ip commented 10 years ago

wonderful, thanks. i think we're also at a point where respect.crossculturalconsult.com can become 100respect.com

pwhipp commented 10 years ago

Just point the domain and it should work - it is all set up. I can then remove the old url from the allowed hostnames if you want to prevent access that way.

On 10 August 2014 12:09, cccs-ip notifications@github.com wrote:

wonderful, thanks. i think we're also at a point where respect.crossculturalconsult.com can become 100respect.com

— Reply to this email directly or view it on GitHub https://github.com/cccs-web/core/issues/97#issuecomment-51703968.

cccs-ip commented 10 years ago

not sure what you mean by 'old URL'. I have switched over the DNS. Please move 'production.crossculturalconsult.com' to 'https://crossculturalconsult.com'/

Also, did we implement our updated ssh protocol and https?

cccs-ip commented 10 years ago

... It does not appear that the nginx configs have been moved over?

cccs-ip commented 10 years ago

I am on the server now and don't see the abadi user. webcore user only has the mezzanine folder that I created. where did you move everything? the new server remains accessible via http://migrate.crossculturalconsult.com. I have otherwise reverted the DNS for all working applications to our original server and IP address. I cannot see any of the migration work.

cccs-ip commented 10 years ago

btw: I forgot jenkins on my list of components that need to be migrated... updated now

cccs-ip commented 10 years ago

it appears that you migrated all the server components to the same server....

cccs-ip commented 10 years ago

http://migrate.crossculturalconsult.com was the only subdomain previously linked the the new blank server, and should have been your access portal.

cccs-ip commented 10 years ago

... this remains the case, by the way. i have re-set all DNS, so that only http://migrate.crossculturalconsult.com is linked to the new blank server. all other subdomains link to the original server.

pwhipp commented 10 years ago

I'm really sorry for the confusion, Aaron. I was away for the rest of Sunday. When I said it is 'all set up' I was just referring to the 100respect.com working on the old server. I've not done any of the server migration work yet.

I can look at it this afternoon (in seven hours).

cccs-ip commented 10 years ago

No worries, Paul. Glad to hear you didn't accidentally duplicate your work ;) My read of the history logs had me worried that something got crossed.

Enjoy your weekend!

cccs-ip commented 10 years ago

Looks like I sidetracked you from this one, but let's please keep it as a priority. I want to shut down the old CCCS server ASAP.

Please note, however, that I made several changes to the database in the last 22 hours, so please make a new dump of the database when migrating...

pwhipp commented 10 years ago

respect site is done and all the db, locales etc. are sorted. The others should be quicker.

pwhipp commented 10 years ago

production.crossculturalconsult.com is now done. It is reconfigured to serve crossculturalconsult.com.

Can you test this (flick the elastic IP over to it and it should serve everything nicely)?

After testing the respect site and the production site, you can either flick it back so all sites are covered or leave the other sites down (assuming it all works OK) - I'll aim to have them sorted tomorrow.

cccs-ip commented 10 years ago

When I switched the elastic IP over, everything seems to be OK. I just re-configured the DNS and currently I am experiencing an issue where the https:// is routing to staging and asking for a login? Maybe that will resolve after the DNS change request goes through and it's just a random bug I am seeing until then?

cccs-ip commented 10 years ago

Everything seems to be working well with the DNS changes fully populated. I disabled some nginx configs, and we're linking to the old IP for our other applications. Hopefully this will slow down the hackers and give us time to complete the migrations.

pwhipp commented 10 years ago

That sounds perfect. I'll continue the migrations of the other sites today.

So - we're live with crossculturalconsult.com in Django!

pwhipp commented 10 years ago

staging.crossculturalconsult.com is done.

cccs-ip commented 10 years ago

Thanks. I have changed the DNS values; the site should switch over in a few hours.

cccs-ip commented 10 years ago

I am getting a 403 "Forbidden" error when trying to log into https://staging.crossculturalconsult.com/

cccs-ip commented 10 years ago

Also, if you could migrate http://abadi.crossculturalconsult.com next, that would be terrific. Something went wrong with the DNS and it's forwarding traffic to https://crossculturalconsult.com, so I can't access the site... hopefully the entries will correct themselves soon.

pwhipp commented 10 years ago

I'll do abadi next.

The 403 was because I forgot to move the htpasswd file, oops! Done.

pwhipp commented 10 years ago

abadi.crossculturalconsult.com is now migrated. You can point the domain appropriately.

Any changes to the database in the meantime will not be reflected in the migration unless you let me know to resync the database.

cccs-ip commented 10 years ago

I just changed the DNS. When you speak of the database, I assume you mean with regard to page content? This hasn't changed in some time.

Ben's pushes to GitLabs won't be affected, I assume? Once the DNS switches over, the old abadi site will essentially be no more...

cccs-ip commented 10 years ago

next up is what? moving Jenkins and Mayan?

pwhipp commented 10 years ago

I'm moving mayan now. Then I'll start handling jenkins but I've moved most of the jenkins work to a separate issue - #129. I've not marked it as high priority but feel free to do so if you see it as such.

All I'll do regarding Jenkins in this issue is indicate when I no longer require it on the old server so we can stop that instance and close this issue as complete (we don't need to terminate the old server; we can always restart it to revisit anything we might have missed - the cost of this is negligible).

pwhipp commented 10 years ago

mayan.crossculturalconsult.com is now migrated. You can point the domain appropriately.

cccs-ip commented 10 years ago

I will change over the DNS for mayan, but the CCCS website is down...

pwhipp commented 10 years ago

I have the jenkins stuff downloaded. You can close this and turn off the server.

cccs-ip commented 10 years ago

excellent. thanks!