search

dansan / spring-replay-site

Website to upload, comment and download replays from/for the SpringRTS engine
http://replays.springrts.com/
GNU General Public License v3.0
11 stars 1 forks source link

migrate replay site to lobby VM #98

Closed dansan closed 5 years ago

dansan commented 8 years ago

not before fixing #94 and #97

abma commented 8 years ago

can the disk space easily be increased after migration?

dansan commented 8 years ago

Not without shutting down the VM :/ I could try to add a new device to the VM... OK - I just tried with another VM: # virsh attach-disk testvm /dev/vg_ex40/testvm1 vdb --targetbus virtio --live --config It attached a new bus to the VMs PCI bus and other stuff and the disk :)

I did a kernel upgrade, so I would actually like to reboot the server at some point... I wouldn't be necessary for the migration though.

I suggest for the migration to shutdown the replays-VM and attach its disk to the lobby-VM. Then no additional disk space or downtime will be needed :)

dansan commented 8 years ago

The [semprini] autohost would need to move too.

abma commented 8 years ago

I suggest for the migration to shutdown the replays-VM and attach its disk to the lobby-VM. Then no additional disk space or downtime will be needed :)

for the migration currently enough disk space should be available. its just that there will be few free space available -> it should be slightly increased after migration. -> imo no need for attaching it to the spring vm (it would only speed up rsync / copying)

abma commented 8 years ago

in other words: a script to migrate data and database already exists: just the replays site doesn't run because of unresolved python deps / broken install of python modules.

abma commented 8 years ago

whats missing for replays.springrts.com? does it now run with "standard" python modules?

i can migrate the autohost but i need help with the replay site.

dansan commented 8 years ago

It's in the process of a code overhaul to work with a current Django version. I will also make it run in gunicorn, so that the site can be updated without restarting the web server. The web server can then serve static files and reverse proxy gunicorn.

dansan commented 7 years ago

OK - with #97 closed, migration can (almost) take place.

It still logs a lot of warnings for code that will become deprecated with Django 1.10, but I can continue updating it to 1.10 etc on the main server without interrupting the other sites by using the gunicorn setup.

I now just have to actually pull all the changes into the live system and see what happens. If all goes well, we can move the replay site to the springrts VM. I'll do that next week.

dansan commented 7 years ago

@abma : we can move the site now. It is now live, running Django 1.10 on gunicorn as my non-privileged user, behind an apache. Python modules are in a virtualenv. With that configuration it should be really easy and quick to migrate it to the main springrts server VM. It uses MySQL - what do you use in the main server?

There are still two other services on that host, that IMHO make no sense anymore: backups of the main springrts server and a springfiles mirror. What should we do about them?

abma commented 7 years ago

It uses MySQL - what do you use in the main server?

mysql

There are still two other services on that host, that IMHO make no sense anymore: backups of the main springrts server and a springfiles mirror. What should we do about them?

the backup makes no sense. I currently backup the springrts.com server locally so moving the services to the springrts.com server would include it into the backup.

moving the springfiles mirror to springrts.com makes a lot of sense... IMHO the important stuff of springfiles (api.springfiles.com) should be moved to springrts.com, too. I can/will do that.

for moving replays.springrts.com imo we should do:

  1. write a script which copies all data (files + db) to springrts.com (not sure if the script at /home/replays/sync.sh still works)
  2. configure the service(s) on springrts server + configure a proxy for the service on the replays server
  3. change dns
dansan commented 7 years ago

There are currently 83GB data (14GB maps, 69GB demofiles) in the replay site and 32GB in the springfiles mirror. Is there enough disk space available in the springrts.com VM, or should we just attach the /var disk of the replay VM to the springrts.com VM? (You wrote above, that there is enough space, but I hadn't provided numbers back then :)

abma commented 7 years ago

Filesystem Size Used Avail Use% Mounted on /dev/mapper/lobby--vg-root 337G 248G 72G 78% /

du -hs /home/replays/ 68G /home/replays/ (thats the folder from the first try to move the service) -> ~140GB available

abma commented 7 years ago

time is running...

currently ~160GB are free on springrts.com.

my migration plan is in on the lobby server: /home/replays/sync.sh

this hopefully syncs all files for replays.springrts.com. i would prefer to rsync as unneeded stuff

how shall we switch the service? setup a http proxy at replays.springrts.com to springrts.com/replays and then wait for the dns change ?!

i've played a bit with django already, are the installation instructions basicly up to date?

any plans of purging old replays? imo its not worth to keep very old replays as possible the engine doesn't work.

dansan commented 5 years ago

Migration was done.