[DRAFT] Simplify services

[Almamu]

This is mostly a test to see If we can simplify the way the EVEmu server handles requests from the client, there's still a ton of work left to do for it to be useful:

• [x] Build a POC of how it'll work
• [x] Support optional parameters
• [x] Write the new service manager
• [x] Write the new bound services
• [x] Port old service code to the new system
• [x] Remove most of the xmlpktgen packets
• [x] Check session data based on the access level of the service
• [x] Build the machoNet cache from the services list
• [x] Support sparse rowsets
• [x] Add cleanup code for bound services
• [x] Specialize bound services based on the amount of clients it'll serve
• [x] Cleanup all the references to the old PyServiceMgr
• [ ] Anything else?

It'd be good to pay special attention to the trading, market functionality and space when testing this. They've been so far the worst offenders in terms of code change...

[Almamu]

Benefits should start being a little bit more obvious now that I've started rewritting services in the new model. Any input on these changes is appreciated.

[Almamu]

I don't think we'd be able to remove the xmlpktgen yet. The main issue is that right now the Python types are a hassle to deal with. Abstracting the method parameters get's rid of a big part of that "dealing with python types" but for xmlpktgen to go we need to improve how things like lists, tuples, dictionaries and primitive types work so we can easily parse data without the need of auto-generated code. Ideally, (and If life doesn't get in the way), this is part of a two step process:

First we get rid of the biggest hurdle that is creating new services and/or methods and make it easier to work with, which is what I'm doing right now. The idea is to get rid of the biggest offender of the data parsing which is what's using xmlpktgen the most.

Second we improve how the python types are handled. I have some features in mind, that have proven to be very useful in EVESharp, that would go a long way in working with them. It would also help getting rid of the reference counting used currently, which is a pain in the ass. This is a list of different things I have in mind that I have yet to translate into proper designs:

• Implement something similar to IEnumerable in C# so we can defer data parsing to when It's going to be used, specially that in the lists or dictionaries also saving CPU cycles and memory
• Build two heaps for Python data:
  • One for static data that won't be removed often
  • One for temporal data that will be removed after each client call is completed. This second heap would prevent DoS requests with too much data AND memory leaks from the data we've processed in the request.
• Improvements to data dump so It's more readable and easy to understand
• Support foreach loops in list/dictionary types
• Generics for most of the type casting
• Getting rid of the assert calls in the type system and replace them with exceptions
• Simplify how numbers are handled so we get rid of all those PyInt/PyLong/PyFloat and consolidate them into one type
• Simplify how strings are handled so PyWString and PyString are consolidated into one

And I'm pretty sure that I'll come up with more things that need improvement once I start to actually look into this, as these are just some of the pain points that I remember since the last time I did some actual work on EVEmu.

[Almamu]

@jdhirst The PR progresses at a good pace. I think that we can start testing the changes to ensure that nothing breaks. There's still some work left to be done, but It's mostly cleanup and some side features that do not affect how the services are called.

[Almamu]

@jdhirst Soooooooo... for the most part this PR is ready to be tested. There's small things here and there that might need adjustement and improvements. but we've been living with some of these shortcomings for the most part of the project, so I don't think that leaving some TODO's here and there is dangerous (especially when we're talking about improvements over the baseline)... I do plan on looking at those at some point, but having more of the eve features supported would make it easy to see what we really need.

[jdhirst]

With the checklist complete for this, I agree we are ready for testing :slightly_smiling_face:. We're not going to be able to get rid of everything that makes things evemu-y, but this goes a long way to clean things up!

Time to merge to staging and and hopefully nothing explodes!