Using best practices

[timothyhahn]

We are currently doing things the way servers/web development have worked for a decade or two. This needs to be fixed, of course.

In all seriousness, I'll get to work on making this "easy". Screw sleep/homework, right?

[santiontanon]

“best practices” are fine. But as you guys saw yesterday, I could not even make it run on my computer. So, the options are: 1) a project that I cannot use and follows best practices 2) a project that does not follow best practices, but that I can use

So, the option is clear :)

I guess I’ll have to get used to use a virtual machines. But please remove all the dependencies from as many 3rd party software as possible. If we can remove the need of Vagrant, I’d be very happy.

Also, can you point me to where do you have the architectural design? so I can see why do you need to do DNS changes in the server machine. It strikes me as really weird, and never came up in any previous online game development I’ve ever done. As I mentioned many times, video games work very differently from traditional server/web apps (typically even writing their own network protocols).

On Mar 19, 2014, at 3:16 PM, Timothy Hahn notifications@github.com wrote:

We are currently doing things the way servers/web development have worked for a decade or two. This needs to be fixed, of course.

In all seriousness, I'll get to work on making this "easy". Screw sleep/homework, right?

— Reply to this email directly or view it on GitHub.

[timothyhahn]

1) I've done some more research into why Vagrant didn't work, and it is due to homebrew and macports both being installed. The reason I think this isn't a "typical" user's case and that in that scenario, homebrew does warn very loudly that something is wrong, and we can't expect it to work on a system that doesn't want to install things.

2) I also know a way to remove every single external dependency (including scala), so you don't even need homebrew/ansible or anything like that. Unfortunately, it still uses Vagrant, since it will essentially provide a virtual machine acting as a local server with everything set up and configured for you. However, nothing else will need to be installed.

3) The design reasons for having nginx/changing some routes to make it look like it's all unified under a single domain is due to Cross-origin resource sharing. http://en.wikipedia.org/wiki/Cross-origin_resource_sharing#How_CORS_works

This is a browser/HTTP limitation. This is unique to how web games work, since we are using HTTP/websockets to communicate between the browser and the server (and this is why there is a difference between this and the normal server/client game setupl. As you can see on that wikipedia article, there is a work around, but it is not only not "best practice" as I mentioned yesterday, it forces you to drop several features, such as HTTP Authentication (which we use), SSL (which we would need to provide secure logins) and cookies being sent between the client and the server (while this is time consuming, you can get around it by manually sending a command to set a cookie in both directions).

The cross domain issue comes from the fact that the game server and the server that serves up the static html content (we had python serving this purpose before) are running on two different ports. This is essentially two different domains, and the browser will treat the two different ports as two different domains.

This design has been in our design document and is essentially just a limitation of how web services have worked since HTTP was first drafted. Nothing in the next version of HTTP will address these issues.

The other solution that I have come up with (other than having an improved Vagrant setup) is to have our game server also serve as our web server, and it will provide the static content. The issue with this, however is that:

1) Our game server is not well equipped for serving static content. While it can, it will come at a high performance cost, since it is nowhere near as good as nginx at serving static content. I didn't think it would be a good idea for a person simply looking at our web page to cause noticeable slowdowns in the game itself.

2) The frontend code would have to be moved into backend. The backend would need to be able to have a copy of the frontend at all times to serve it. Putting a git repo inside another git repo is extremely difficult and time consuming (http://codingkilledthecat.wordpress.com/2012/04/28/why-your-company-shouldnt-use-git-submodules/), so that wouldn't work. Simply putting our frontend's code into our backend (not as a repo) would work better, but we then have to consider how to "work around" each other, since the distinction between client work and server work has blurred greatly. While this is beneficial for the end result, it creates extra work when committing code, and you already know this team has had lots of difficulties already in committing code to the right place.

The difficulties in creating a web based MMO were raised since the very first day our group was formed, and I raised them with the entire student group at the time (outside of the class room, after our group first got together). While there have been other web based MMOs, they either use many more extra dependencies than we do (Everquest) or are not intended to be performant or extensible (BrowserQuest). Many of the things that I know my team members would have complained about having to use I have done my best to cut back on, however, this final requirement I had no easy way to overcome without either creating our own browser or using additional plugins/software (such as Flash).

I would also like to apologize about being snippy and rude to you. While I am the person on the team that wanted to do this specific MMO project the least, I am also the person who has put in the most amount of code contribution (just by a bit if you count the commits on my personal GitHub page that are used in our project and by a lot if you count the commits on extra repositories that are not used in our project but have been used to teach or prototype ideas for our group, including Scala, AI, Akka, QuadTrees, ECS, etc.), the most amount of monetary contribution (server time and I have to travel a farther distance than the rest of our group), and I have been working on this system's design nearly every single day since we started. This is simply because I want this project to succeed and I want it to be something we can be proud of.

Additionally, I was extremely defensive about my choices, as on our student team, those who contribute the least are my most vocal criticizers, and while they sometimes do have a point, it is infuriating to listen to a few fellow "teammates" complain about every change I make, and not only spread dissenting comments against me amongst others when they think I am not listening, but to voice their displeasure at my work when the amount of work they put in is sometimes not even reaching the bare minimum.

There is a vast difference between those of my teammates who have challenged my designs in order to help me find holes and improve on the design and those who have challenged my designs simply because they don't care enough to attempt to understand it.

For the most part, however, I've been keeping my complaints to myself, but I've already been hospitalized due to stress related issues since our project started, and voicing my complaints in an infrequent, but sarcastic manner has been more stress relieving than anything I've found so far, since keeping them in have been the source of my stress to begin with.

When we approached you yesterday, my impression of you was that you were dismissive and uninterested in the "why" of the changes to the server. I was also upset since you kept emphasizing you wanted as few external dependency, and the solution I wanted to offer was both fewer dependencies in number and fewer things to actually do. The issue we had with Vagrant yesterday was not due to Vagrant, but instead Ansible and the way homebrew and macports were conflicting, and it was a very unlikely discrepancy that could have been easily fixed if I had a few more minutes to research the subject. I understand that you may not have been intending to sound dismissive, but every time I tried to offer a clear technical answer as to why the changes were made, your only response was either "I'm not convinced" or "But this is more complicated", and in neither case elaborating on your specific issue with it.

Immediately after that, at our Salvage meeting, the group nominated me to start presenting the project (without letting me know beforehand) and followed up with groans of displeasure when I initially hesitated (as I wasn't sure where to start), causing Salvage to move onto the next group. While the entire group is not at fault, the fact that I'm being actively antagonized by the group is getting grating.

At any rate, I've talked with our group leader and he's decided that I should take a break from this, which I think you'll understand I will accept readily.

If any of you other members of Ayai want to implement one of the two solutions I've listed above (the addition to Vagrant was to create a base box with all those dependencies installed, as well as any configuration changes, upload this base box somewhere, create a script that automatically installed vagrant for you and grabbed the box, started it up, and turned it on, giving you only a single file to run that would give you an ip address for you to use locally), feel free to. You can also try the "Have our game server serve the web page as well" strategy, which may or may not work (probably will, but will also probably introduce some new oddities into the backend, since we are coupling our services to the point where they are colliding/competing with each other). This also means that anyone who runs our project will have to install scala/sbt on their machines. I'll also leave the other repositories and libraries that I created for the project running and if you run into a bug with any of them, you can leave an issue and I'll take care of it.

[santiontanon]

Hi Tim,

Let me start by saying that I don’t think you were rude. Actually I was surprised that you even bring that up. So, don’t worry, I am not upset, and I didn’t feel anyone was rude at all :) (you will have to try really very hard to make me upset, hehehe :))

Also, I’m sorry that you felt I was being dismissive: guys! I asked you to explain in the whiteboard, and you said “that would not help”. So, how can I consider an idea that is not explained to me? So, I reiterate my question, where I can I see the design of the new approach in the design doc? (which page?) I really want to understand. That’s why I asked :)

Anyway, I appreciate the effort you have put in the project. But you need to understand that I need to run the project after you leave. See me as your “client”. I don’t care which solution you guys find for the networking part, but I need something that will easily run in my machine, my future machines, and on the machines of my students. That is a requirement of the project. Whether vagrant or ansible is the problem does not matter. The problem is that each dependency makes the project more brittle. What happened in my office is just an example that illustrates my point.

That being said I don’t have anything against the virtual machine approach, if you tell me that it will run now and in 5 years from now, then go ahead with it. But if it depends on projects that might not be maintained by then, then please reconsider...

Long story short: go ahead with whichever option you want, but you need to ensure I can run it, now and a few years from now.

On Mar 19, 2014, at 8:50 PM, Timothy Hahn notifications@github.com wrote:

1) I've done some more research into why it Vagrant didn'tlwork, and it is due to homebrew and macports both being installed. The reason I think this isn't a "typical" user's case and that in that scenario, homebrew does warn very loudly that something is wrong, and we can't expect it to work on a system that doesn't want to install things.

2) I also know a way to remove every single external dependency (including scala), so you don't even need homebrew/ansible or anything like that. Unfortunately, it still uses Vagrant, since it will essentially provide a virtual machine acting as a local server with everything set up and configured for you. However, nothing else will need to be installed.

3) The design reasons for having nginx/changing some routes to make it look like it's all unified under a single domain is due to Cross-origin resource sharing. http://en.wikipedia.org/wiki/Cross-origin_resource_sharing#How_CORS_works

This is a browser/HTTP limitation. This is unique to how web games work, since we are using HTTP/websockets to communicate between the browser and the server (and this is why there is a difference between this and the normal server/client game setupl. As you can see on that wikipedia article, there is a work around, but it is not only not "best practice" as I mentioned yesterday, it forces you to drop several features, such as HTTP Authentication (which we use), SSL (which we would need to provide secure logins) and cookies being sent between the client and the server (while this is time consuming, you can get around it by manually sending a command to set a cookie in both directions).

The cross domain issue comes from the fact that the game server and the server that serves up the static html content (we had python serving this purpose before) are running on two different ports. This is essentially two different domains, and the browser will treat the two different ports as two different domains.

This design has been in our design document and is essentially just a limitation of how web services have worked since HTTP was first drafted. Nothing in the next version of HTTP will address these issues.

The other solution that I have come up with (other than having an improved Vagrant setup) is to have our game server also serve as our web server, and it will provide the static content. The issue with this, however is that:

1) Our game server is not well equipped for serving static content. While it can, it will come at a high performance cost, since it is nowhere near as good as nginx at serving static content. I didn't think it would be a good idea for a person simply looking at our web page to cause noticeable slowdowns in the game itself.

2) The frontend code would have to be moved into backend. The backend would need to be able to have a copy of the frontend at all times to serve it. Putting a git repo inside another git repo is extremely difficult and time consuming (http://codingkilledthecat.wordpress.com/2012/04/28/why-your-company-shouldnt-use-git-submodules/), so that wouldn't work. Simply putting our frontend's code into our backend (not as a repo) would work better, but we then have to consider how to "work around" each other, since the distinction between client work and server work has blurred greatly. While this is beneficial for the end result, it creates extra work when committing code, and you already know this team has had lots of difficulties already in committing code to the right place.

The difficulties in creating a web based MMO were raised since the very first day our group was formed, and I raised them with the entire student group at the time (outside of the class room, after our group first got together). While there have been other web based MMOs, they either use many more extra dependencies than we do (Everquest) or are not intended to be performant or extensible (BrowserQuest). Many of the things that I know my team members would have complained about having to use I have done my best to cut back on, however, this final requirement I had no easy way to overcome without either creating our own browser or using additional plugins/software (such as Flash).

I would also like to apologize about being snippy and rude to you. While I am the person on the team that wanted to do this specific MMO project the least, I am also the person who has put in the most amount of code contribution (just by a bit if you count the commits on my personal GitHub page that are used in our project and by a lot if you count the commits on extra repositories that are not used in our project but have been used to teach or prototype ideas for our group, including Scala, AI, Akka, QuadTrees, ECS, etc.), the most amount of monetary contribution (server time and I have to travel a farther distance than the rest of our group), and I have been working on this system's design nearly every single day since we started. This is simply because I want this project to succeed and I want it to be something we can be proud of.

Additionally, I was extremely defensive about my choices, as on our student team, those who contribute the least are my most vocal criticizers, and while they sometimes do have a point, it is infuriating to listen to a few fellow "teammates" complain about every change I make, and not only spread dissenting comments against me amongst others when they think I am not listening, but to voice their displeasure at my work when the amount of work they put in is sometimes not even reaching the bare minimum.

There is a vast difference between those of my teammates who have challenged my designs in order to help me find holes and improve on the design and those who have challenged my designs simply because they don't care enough to attempt to understand it.

For the most part, however, I've been keeping my complaints to myself, but I've already been hospitalized due to stress related issues since our project started, and voicing my complaints in an infrequent, but sarcastic manner has been more stress relieving than anything I've found so far, since keeping them in have been the source of my stress to begin with.

When we approached you yesterday, my impression of you was that you were dismissive and uninterested in the "why" of the changes to the server. I was also upset since you kept emphasizing you wanted as few external dependency, and the solution I wanted to offer was both fewer dependencies in number and fewer things to actually do. The issue we had with Vagrant yesterday was not due to Vagrant, but instead Ansible and the way homebrew and macports were conflicting, and it was a very unlikely discrepancy that could have been easily fixed if I had a few more minutes to research the subject. I understand that you may not have been intending to sound dismissive, but every time I tried to offer a clear technical answer as to why the changes were made, your only response was either "I'm not convinced" or "But this is more complicated", and in neither case elaborating on your specific issue with it.

Immediately after that, at our Salvage meeting, the group nominated me to start presenting the project (without letting me know beforehand) and followed up with groans of displeasure when I initially hesitated (as I wasn't sure where to start), causing Salvage to move onto the next group. While the entire group is not at fault, the fact that I'm being actively antagonized by the group is getting grating.

At any rate, I've talked with our group leader and he's decided that I should take a break from this, which I think you'll understand I will accept readily.

If any of you other members of Ayai want to implement one of the two solutions I've listed above (the addition to Vagrant was to create a base box with all those dependencies installed, as well as any configuration changes, upload this base box somewhere, create a script that automatically installed vagrant for you and grabbed the box, started it up, and turned it on, giving you only a single file to run that would give you an ip address for you to use locally), feel free to. You can also try the "Have our game server serve the web page as well" strategy, which may or may not work (probably will, but will also probably introduce some new oddities into the backend, since we are coupling our services to the point where they are colliding/competing with each other). This also means that anyone who runs our project will have to install scala/sbt on their machines. I'll also leave the other repositories and libraries that I created for the project running and if you run into a bug with any of them, you can leave an issue and I'll take care of it.

— Reply to this email directly or view it on GitHub.