Master server ?

[jhbruhn]

So, I thought it would be awesome if actually everything would run on some kind of master server which manages the games and acts as a lobby for everyone who wants to play etc. Sadly i haven't really got in experience in Networked Programming, but what do you think about the idea?

[spaceemotion]

i already programmed something like a central point, but since nobody answered in the forums i stopped working on it. Sadly the forum is currently closed but you can access my work here: http://spaceemotion.zapto.org/ci/ The plans were a central location for map and server listing (for maps also downloading) which data can be also accessed in-game.

[jhbruhn]

That looks really neat but I thought about some kind of "server-browser" ingame with pair-matching (like Portal 2 or so). I think it would be really hard to implement, so it's just a thought.

[spaceemotion]

well, my system is API friendly so this wouldn't be a problem (server-side). But I also think that this whole project kinda went down, so I think implementing sth. like this would be more effort than success.

[Mebibyte]

Although, it would potentially encourage more people to play it. Making it super simple to just hop into an online game would be much funner than only being able to play with someone on your network.

[spaceemotion]

If the wish is there i can code it right into my system, nothing big to do :D

Edit: According to my site stats there are still 1k visitors each week. Maybe it won't result in such a fail than i thought :)

[jhbruhn]

Yeah. It would be nice to start with a simple map-downloading-feature from http://spaceemotion.zapto.org/ci/ and then implementing some kind of master-server-concept

[spaceemotion]

i will be not at home next week so the site will be down (its my localhost btw) but when i will get back i can start working on the API (list of maps, downloads, etc.)

[jhbruhn]

Is it possible for you to put the code on GitHub so we can take a look at it etc.?

[spaceemotion]

that would be possible indeed :D I am currently storing it on a subversion server (also local), i can copy it over to github if you want.

[Mebibyte]

Yes, please do! I would love to see what you have created so far.

[spaceemotion]

okay its up:https://github.com/spaceemotion/CatacombSnatch-Repo please mind that the database could not be uploaded (of course)

for the api: i thought using the REST structure, what do you think about?

oh and if you want you could also help me out solving this problem (i wanted to create a map preview using the image lib and the raw xml data...): http://stackoverflow.com/questions/10613996/decompressing-tiled-tmx-file-contents-with-php

[Maescool]

About the master server, i found something that could help http://smartfoxserver.com/ check it out and post what you think about it..

[spaceemotion]

so this smartfox server is basically just an extension for the server but still can get all the data from the website? or does it have its own way of storing information? anyways, i think this is a great tool for both, server-side and client-side. It just takes some time to recode the parts in the game i think.

[Maescool]

I'm currently researching and trying to add to the code. here is some good reading material http://www.smartfoxserver.com/downloads/sfs2x/documents/SFS2X_FPS_Walkthrough.pdf i also recommend http://docs2x.smartfoxserver.com/

this can do everything we need for a lobby master server system. smartfox community (free) is licensed for 100 users to play at once.

[jhbruhn]

Have you maybe already got any results?

[Maescool]

I'm looking into it.. but I best get some help from @Flet

[Flet]

I took a quick look, @Maescool but not sure what you're looking for. let me know!

[spaceemotion]

i also took a look at it and installed the server on my own. the thing that i am worried about is the database. how should i access it when i cannot find it? where does SFS saves its data?

btw we are onl getting about 10-20 people on the website atm. i think its because the forum has been closed :(

[Maescool]

I did some initial writing and including the client API @spaceemotion I upgraded the forum yesterday, asked @Austin01 to fix the theme (got borked with upgrade) He did this, i'll see if i can set the forum live again.

To test this, you need to locally install smartfoxserver (for now) i guess this is not a problem for you guys.. just drop it in /opt/ (make sure you also upgrade after the initial install, which is just untarring the file xD) The documentation is great, but we need a stratagie how to implement, cause we will also have to write extensions for the master..

[spaceemotion]

i had my birthday today so lot of family and friends were around but i can make some simple sketches tomorrow. for the extensions: which ones do we need? for the custom protocoll data?

[Maescool]

First of all @spaceemotion Happy bday! :D

I notice you guys saw the new branch i added.

There is a lot to be done.. a short list, yes i could go more in detail, but i guess this needs to be done in a wikipage or something (easier to update) or i can make new issues with sfs milestone..

• some kind of login system.. (registering can be done on the site)
• design for the lobby screen, chat-screens needs to be in there, you need to be able to find your friendlist, etc..
• if you have read the sfs docs you know there is a gameroom per game, and that also supports in game chat ;) and all the gamecalls that happen, the gamecalls are passed to the extensions to handle, so the gameroom simulates the game, so it can confirm the calls (work against cheaters) Also that way we can keep ranking between the gamers, also will be easier to add more players, i guess we can go up to 4.

this is a lot of work to be done.. cause the gamecalls need to be handled by an extension on the serverside.. but i have no clue how to start on this..

Yes we are starting this back up! :D

[Maescool]

To offload the smartfox work a bit, it can make webservice calls to verify login, handle a part of the friends, handle the ranking system.. so the game also can call that service. Note: we need to be very carefully about the security in this..

[spaceemotion]

the server side stuff will be easy (the repo i mean) but we need sth like and api auth key and session timeouts. ive never done sth like this so far...

[spaceemotion]

since the api will be rest based i found a way to authorize: http encrypted aurhentication (http://en.m.wikipedia.org/wiki/Basic_access_authentication) the username will be encrypted thought and the pass will be a sh1 salted encryption.

[Maescool]

Btw, there is also an alternative server: http://www.electrotank.com/es5.html but i'm not sure which to take.. smartfox or es5

I'll see what you guys think before we commit to for smartfox..

[godleydemon]

I can see a huge advantage in having the master server with user registration and game host lobby and I'll kind of list them out a little bit.

1. With user name we can display user name plates above characters heads when in game.
2. This will allow us to create a simple achievement system with a viewable achievement list on the main website.
3. For the way off into the distance future, we can possibly see implementation of customer character sprites.
4. people can easily distribute and play custom maps
5. This will pique interest in the game as it has waned since the humble bundle days
6. implementation of lobby chat, options menu, ect will finally be able to be realized.

With all of this, this allows us to bring in a new wave of viewers and possible programmers to the project. To think this project has gone this far on just volunteer work is amazing. We need to keep this going.

[spaceemotion]

before we dive too much into the SFS stuff: @Verexa made a good point when we talked about the master server.

About SFS:

• API-friendly
• multi-language
• no access to Db (we hadnt found one)
• no dynamic or flexible way to handle custom stuff (except the extensions)
• no master-server API (huge minus for us)

So we thought about using StreamSocketServers in PHP (example: http://www.php.net/manual/en/function.stream-socket-server.php - see comment 1 for a ssl login)

• DB-friendly
• master-server is flexible
• repo-integration made easy

So far we have not found any negative points for the PHP solution.

What do you guys think?

For the stuff that @godleydemon said: Achievements can be easily stored in the db then, with readable output on the user pages custom maps can be downloaded via PHP very easily (via the stream or a URL)

[jhbruhn]

I'd prefer the PHP solution, although it would probably be alot more work, or are there any libraries supporting the development of a Masterserver? Additionally, I could maybe help a bit with the programming because I've already got some experience in PHP (might need to refresh it a bit).

[spaceemotion]

well there is some kind of library already in PHP: http://php.net/manual/de/book.stream.php @Verexa is currently trying it out, but has some issues with multi-threading the streams

The "friends" function works on my local copy of the repo already, ill upload the changes later.

[ljfio]

It is possible to create this solution using PHP. The only few problems that I've currently come across is that multiple connections will need a bit of work. I know that if we had a REST like API in place for the client to use we could do it very easily. But I think that dealing with a lot of information and requests we're better to use streams. As it would mean only authenticating once.

[spaceemotion]

@Verexa started an awsome test code for read/write sockets in PHP. Because I had trouble to re-use ports once the script has been stopped I changes some things.

The work can be seen here: https://github.com/spaceemotion/CSMS (Lots of credit to @Verexa :) )

The best thing is: I opened the game just for fun to see if the game can connect to the script - and it does! The server currently does not recieve the right packages and does not send sth back, but the game recognizes the PHP "server" as a legit game-server! Currently its only working on localhosts, but we are sure that this can be fixed very soon. If this thing really works we made a huge step forward!

[godleydemon]

holy crap dude, did we literally just jump foreward in time!? I thought it would have taken a month to get this far at least, lol. Nice going Verexa and SpaceEmotion! :D

I think some of the first few things that should be worked on though after figuring out how to make the client handshake the server. We need to design the login screen and or launcher screen. We can start with a simple user name to begin with. But this will allow us to begin programming the db to accept incoming connections based on user names. This will also let us experiment a little with the handshaking process. I've been playing around with some images last night and a little today. I'm designing a few screens for MP. Just so we can kind of get an idea what we're going for here. We can also change little things here and there to adjust it. So that way everyone will be happy with the final outlook. Alot of things won't be able to be programmed for MP until the master server and client hand shake is figured out through the php. Because I'm deffinantly seeing some benefits of using the php.

[spaceemotion]

so far we made the server OOP based. we have classes for commands and packets just like in java - for better handling. we already can read and write packets using this new system (still locally). new stuff can be added very easily now :)

[spaceemotion]

hat we definitly need is a better packagestructure. for example sth like the handshake paket, then a clientinformation paket (game version,...) and then login pakets.

we now have devided the server in: master and game server. the game server handles the game. the master handles the creation of a new game, authentication, map listing and so on. the way of communicating can be done via 1) pakets or 2) commands. the pakets are only being sent by the client and server, commands are morr like admin stuff vis the console to halt/debug the server. thats why we devided the server into server and client parts: client folder holds anything coient related, the server folder anything server related (also db stuff)

[Maescool]

The reason I propose Smartfox is that it can handle massive amounts of connections and handle more than a php-server could. The smartfox doesn't need direct contact to the database, but can do it over json calls to a php webservice script, which can handle the database part etc..

The main reason i stand behind this server is that it can first of all, do decent TCP and UDP connections, provides an client-server API, and has proven to handle a lot of gamers at the same time.. which i'm sure php read write sockets can handle that well.. and it saves a lot of development time :)

Really no offense to you php guys.. but like i said, there will be still a lot of php work anyway, so we need to write out first what the dataflow will be, i'll try to write something up this week how i have it in mind and believe can scale.

[godleydemon]

I think what Maes purposing is a good way to look at. Making the json calls to a php webservice script I think might be the best route to take. I was actually talking about the server with a friend last night and was expressing the same concerns of the php being able to handle a massive intake of connection at once. The conclusion... theres going to be a lot of dropped packets and or commands not reaching the server. But if we use smart fox and program a json call to php, we can deffinantly see an improvement in connection and less packets dropped.

I think what one of the things we should kind of start on though is a .exe launching script thats akin to minecraft. The actual jar is stored in %appdata% you run the game through the .exe which handles the hand shaking to the master server for authentication. Once authenticated, the .exe launches the jar with the appropriated information from the master db information on user.

[ljfio]

I can see you have expressed concerns an made very valid points. The only issue I can see is how fast it deals witch connections. There is only a limitation of sockets available as there is to your computer. I believe that we wouldn't have issues with lost packets but on the rare occasion (usually client fault though) we could deal with it well. I can't exactly test a system that isn't finished, but I see many other advantages to using PHP.

With Smartfox server I can't seem to find API documentatio for the server side and believe there might be some licensing issues. As far as it goes I believe we can do what we're doing in PHP but in Java and have it a multi-threaded server to increase performance. I'm just thinking that we can produce a more custom masterserver to what we need.

[spaceemotion]

The problems I saw with using SFS are very important ones: Database integration (like I said, I don't know how i can get the data of open rooms from it without having a client that sends me the data to a php script). the second is the user numbers, the free edition is very limited in my eyes, so SFS could handle this stuff but then stops because the server is full. If only 100 players can be connected that would make 4 waiting in the lobby and 96 others are playing (that makes 48 games). Maybe you are right when you say that the server my not be able to handle all the sockets, but what we currently are making is only the master server (for getting rooms, maps and stuff). the game server itself is another thing that maybe is not even based on php.

[godleydemon]

hmmm, you have a major point there SpaceEmotion. My thought though is this, what about instead of having everything hosted in a master server. Let the client handle the server side calculations but make periodic calls to the master server on event hooks. Basically the only thing a master server would do is keep track of live games, players and so on. It will also record user data in the database for achievements (like how many snakes have I killed) This way we minimize the amount of stress on the server and instead divide it onto the client computer.

[jhbruhn]

That's a good idea too, but then we'd have to deal with the ports the player who is hosting the game would need to open, won't we?

[ljfio]

Yeah me and @spaceemotion had decided to make the masterserver a REST API. We're currently working on it in PHP just now. https://github.com/spaceemotion/SnatchAPI

We think that it should be down to a dedicated server or the client with the server classes to host a game lobby. Then report back to the master server. This is what I thought you guys wanted originally but was lead astray thinking it was meant to be something else haha.

Yeah we'd have to deal with that so... We thought 3031 for the game port, and a HTTP 80 request for the master server aka API?

[godleydemon]

3031 is pretty easy to remember and its obscure enough nothing is going to interfere with it. -does a google search for port 3031 then face desks- apparently that port is used on Remote AppleEvents/PPC Toolbox. I'm not sure how obscure or how used this is >.0. Let me do a little more research on it just to confirm.

[godleydemon]

ok this is what I've dug up on port 3031 http://www.speedguide.net/port.php?port=3031

[spaceemotion]

Then let us do port 3040: http://www.speedguide.net/port.php?port=3040 Already used by other games, so it would fit in there :D (and I dont want to let the game be recognized as a virus)

[spaceemotion]

By the way: I just wanted to say that the API is almost finished (server side). The only things left are filling it up with commands and implementing an authorization library (like OAuth). The next thing would be the Java code then I think :D

[godleydemon]

sounds awesome Space. I can't wait to see a little dev build of this to test her out, any idea on where we are going to run the master server at for the server lists?

[spaceemotion]

umm, i dont think that you got it quite right. there is no real server for only the lists. there are only two sites to handle the requests (not the games): the repo and the api. the repository handles the data while the api handles the requests from the game (login, getting maps, servers and so on). The repo is only a website utility to display and edit stuff. It also is the home of the users - to play online you have to register in order to get a UUID. So this whole thing is running on the homepage server while the games are handled somewhere else.

And for the repository: may be sound a bit hard - but i am thinking about a rewrite. the way its now is too complex and not flexible. i am also seeing code igniter as a not-fast framework for this kind of website so it may be a custom site...

[jhbruhn]

Another alternative would be writing the repo with Rails in Ruby. I don't know if you know ruby, but that would be quite cool too. There's a plugin (gem) which's only purpose is to serve APIs. Here's a good tutorial for it: http://railscasts.com/episodes/348-the-rails-api-gem Maybe you could make a small list of requirements of the API and a can start to work on it in rails. Help is really appreciated if anyone knows rails/ruby. :D But you can use PHP too, this is just an idea.

[spaceemotion]

I am coding in PHP/JS/CSS and HTML for 5-7 years now. Using Ruby would be a bit of a problem because I would have to learn it first, however I know that ruby is preferred when using APIs. If the PHP system fails (too much traffic, other problems) we can still think about it. For the repo i would like to code a system that is specialized in creating "game-based" communities and managements.

[spaceemotion]

Just wanted to say that the framework is nearly done :) Had some free hours today, so I started working on a lightweight framework with nearly no footprint, but the efficiency of our SnatchAPI and the flexibility of CodeIgniter in mind (I only have 9 files so far - and it still works ^^).

I will commit the first version when a single website output can be generated :)

The framework is only a base though. Once its running stable and got some plugins and helpers I will rewrite the site (you can help too!). Like I said, the reason for this is the massive weight of CodeIgniter - it runs benchmarking and some other (for me) useless stuff that I am not using anyways which takes some more time to load the site.

edit: basic output works - only models are left

edit 2: I finally committed the first parts of the framework: https://github.com/spaceemotion/CodeFire. Please mind that this is only the framework - not the repository code!