KanColleTool / kcsrv

An open source KanColle server
MIT License
41 stars 16 forks source link

Running KCSrv without the VM configuration? #1

Closed SoftwareGuy closed 10 years ago

SoftwareGuy commented 10 years ago

This isn't an issue as per se, but just curious - can one run the server without needing the VM setup? For example, I could deploy and test the server on a low-power ARM based development board (Raspberry Pi, ODROID, etc) which would get crushed under the weight of VirtualBox requires.

All I'd need to know is the package list and I'd just apt-get the list. Hopefully.

EDIT: Newbie FIRST! issue created :smiley:

EDIT 2: So far so good but stuck with a SyntaxError. Dump as follows:

~/kcsrv/kcsrv-master$ ./manage.py db upgrade
Traceback (most recent call last):
  File "./manage.py", line 5, in <module>
    from kcsrv import app
  File "/home/coburn/kcsrv/kcsrv-master/kcsrv.py", line 5, in <module>
    from db import *
  File "/home/coburn/kcsrv/kcsrv-master/db.py", line 4, in <module>
    import util
  File "/home/coburn/kcsrv/kcsrv-master/util.py", line 14
SyntaxError: Non-ASCII character '\xe6' in file /home/coburn/kcsrv/kcsrv-master/util.py on line 14, but no encoding declared; see http://www.python.org/peps/pep-0263.html for details

Stuck this in util.py...

# -*- coding: utf-8 -*-

...and tried again. And then this happened!

~/kcsrv/kcsrv-master$ ./manage.py db upgrade
Traceback (most recent call last):
  File "./manage.py", line 5, in <module>
    from kcsrv import app
  File "/home/coburn/kcsrv/kcsrv-master/kcsrv.py", line 15, in <module>
    app.config.from_object('config')
  File "/usr/local/lib/python2.7/dist-packages/flask/config.py", line 162, in from_object
    obj = import_string(obj)
  File "/usr/local/lib/python2.7/dist-packages/werkzeug/utils.py", line 426, in import_string
    sys.exc_info()[2])
  File "/usr/local/lib/python2.7/dist-packages/werkzeug/utils.py", line 408, in import_string
    return __import__(import_name)
werkzeug.utils.ImportStringError: import_string() failed for 'config'. Possible reasons are:

- missing __init__.py in a package;
- package or module path not included in sys.path;
- duplicated package or module name taking precedence in sys.path;
- missing module, class, function or variable;

Debugged import:

- 'config' not found.

Original exception:

ImportError: No module named config

EDIT 3: I'm a idiot - didn't realize you had to rename config.example.py to config.py. :frowning: EDIT 4: Got it running. Nice job, dev! mwhahaahha Shall I buy you a beer?

liclac commented 10 years ago

Looks like you got it working, though perhaps not the way it's intended… starting with how you seem to be running it on Python 2 (going by the encoding error), when it's built for Python 3 ^^

The easiest way to do this would actually be to imitate a Vagrant deployment on the host machine. I assume (from the mention of apt-get) that you're on some kind of Debian-based system, which is perfect - the Vagrantfile is configured to set up an Ubuntu 14.04 system, since there's a nice prepared Vagrantbox for it.

Clone the sources into /vagrant (yes, in the system root, that's the way Vagrant does it), and run the Ansible playbook in provisioning/playbook.yml, which should be all you need to get it fully set up.

The one thing that might not work for you is that since the Vagrantfile specifically targets Ubuntu, the startup script for the uWSGI Emperor is an Upstart service rather than a SysVInit script, so you're gonna have to write of those yourself if you're using a SysVInit-based system such as Debian. Do feel free to send a pull request swapping the Upstart service out for an initscript though, I hate Upstart with a passion, and only used it for this because uWSGI's website has a nice template for it. Upstart's service command can handle initscripts as well, so it'd be nothing but a win to have one instead.

To add it, put it in provisioning/init/, then just edit provisioning/playbook.yml and change the symlink step for the Upstart job to link in the initscript in the right place. Might need to execute something to make it start on boot as well, but I can't quite remember - it was a long time since I last had to touch a SysVInit system, my own Debian server uses systemd from wheezy-backports, and all of my clients' servers use Upstart.

I'll have to decline the beer though, not only because I don't drink alcohol, but because I don't want to feel obligated to anyone before I've gotten this in a working state ^^;;

SoftwareGuy commented 10 years ago

It was intended for python3? Bugger, Python 2.7 or 2.8 is what Ubuntu is shipping with (14.04), so I might install the 3.0 branch and tell it to use that instead.

I'll consider the vagrant paths next time to boot up the server. Still, I'm happy to run the server privately on my own network because I can then play around and break the game. And that's what we ultimately want to do, give ourselves unlimited Shima/Amatsu-Kaze shipgirls, no?

liclac commented 10 years ago

Yeah, Python 3, some parts will work on 2.7 as well, but it'll break the moment I start to use Py3-exclusive stuff (like pyenv, for starters). Python 3 is available in Ubuntu's repositories (the development VM uses 14.04 as well), so you just have to install it. Look at the playbook for the exact packages used.

But the ultimate goal is more to provide a KanColle server without DMM's banhammers and poor performance, nearly identical to the official game in every way, as well as to understand the game better… I already have a Shimakaze too >w>

SoftwareGuy commented 10 years ago

Alright, I've noted that and I will install python3 today when I push stuff to that machine.

I second the goal. I want to learn how the game works, and how it does API stuff. I know we may not be able to do public unofficial servers (I bet the legal team will get on our tails real quick) but for private use and playing with your friends, it's fine I guess. The only thing wrong with the public unofficial is if one charged money to access the server - that wouldn't fly by DMM, because they lose profits.

This was just food for thought, but I will test as much as I can as I'm really curious to see the internal layers between the pretty shipgirls and the engine that runs the game.

EDIT: Apparently mySQLDB doesn't work with Python3.x and SQLAlchemy doesn't like SQLite (well, SQLite is kinda basic). So it's either PostgreSQL or bust...

EDIT 2: I said stuff it and went to brief myself on PostgreSQL. Now it's running on that DB like a boss.

liclac commented 10 years ago

I'm definitely going to host a public server, the more people are on one server, the more accurately I can emulate PvP. Definitely not charging for it though, at most putting up some advertisements to help cover the server bills.

And you're going to enjoy watching this engine grow. The game is a massive clusterfuck under the hood with mind-bogglingly stupid design mixed with completely unnecessary complications and information that isn't even needed. It's absolutely hilarious to watch.

(And yes, PostgreSQL is the only supported database backend. Seriously, read the playbook - that exact setup is the only one I'm supporting, and is what I'll be using on my server once it's ready to go live.)

SoftwareGuy commented 10 years ago

Well, we can safely say that the devs don't know how to do proper game code. Honestly, for a game like this, the simpler the code is I guess it's better because I wouldn't want to run a public server that uses up all my resources.

I might do the same and host a public server just for kicks, mainly because I'm not real fond of DMM and I'm an Australian, so ping times and network dropouts are a factor.

As for supported SQL backends, I've got it working on PostgreSQL, but I can confirm it can do MySQL as well. I know you have your own supported configuration, I'm just a person that likes to tinker with the setup, and sometimes, help to find bugs in various layers of the code.

liclac commented 10 years ago

MySQL is going to work until you trip over one of its bizarre bugs. It's going to break in possibly quite subtle ways once I start using PostgreSQL-specific features, or maybe just start performing like ass, because I'll be making no attempt to work around MySQL's limitations…

If you're interested in running a server, I should look into some way to have multiple servers collaborate… multiple frontend servers and one backend server, maybe? Master/slave databases? World transfer options (with cryptographic authentication of datablobs to prevent cheating)?

SoftwareGuy commented 10 years ago

Sorry for the absence, but that would be nice. And that also works somewhat with load balancing, in the event that your server is down or does up in smoke, mine can keep the open source version running until yours is back in the game.

I also consider using MariaDB which takes MySQL and unfucks what Oracle fucks up. (How does that work?)

PostgreSQL seems like another language to me but I can get the geist of it.

liclac commented 10 years ago

Please don't run it on anything but Postgres. Once I have time to actually work on it again, I'm going to start using PostgreSQL and Python 3-specific features, at which point this will break for you.

And MariaDB doesn't unfuck MySQL as much as it improves it; it's a drop-in replacement, and thus needs to be 100% compatible with the original, including the gigantic pile of design flaw. For some fun, google "MySQL server has gone away", that's my latest favorite one, currently affecting a work project.

SoftwareGuy commented 10 years ago

I've had that issue, but mine was related to query packet sizes. I have, as I said, PostgreSQL at my disposal ready to test. I might check out the code and see what I can do to help, I am fluent in PHP as I'm a web developer, but I know little bits of Python.

That and I am sick and tired of DMM's servers taking /forever/ to feed my client resources.

SoftwareGuy commented 10 years ago

Going to try the Vagrant stuff tomorrow, however I'll have to use a beefier machine (ie. laptop) because pip from python3.2 fucked up the install on my Raspberry Pi. I'm going to just play it by your rules.

I have also been capturing data going back and forth from DMM's servers to my KanColle, so I can see what requests are being put where. You said the error cats work - how do you get them to trigger?

SoftwareGuy commented 10 years ago

Going to close this, redundant.