Universal Back-bone system

[KevinMackenzie]

The WPI lib is great, but it isn't super flexible for external control, so I propose a backbone that has a set of commands which take parameters to activate systems make class instances. The application layer calls these commands via an intermediate layer in java, while external sources send network tables requests. It would also be easy to use command line options to do the same things. this universal backbone means custom-made clients for controlling the robot from the driver station can use a consistent API

[KevinMackenzie]

the only way I can imagine this will work as flexibly as I propose it is to do it kind of like OpenGL does it. It is still object oriented, but instead of seeing the classes, a list of member variables, a instance, reference, etc., it just uses a slew of commands which create objects internally which can be modified by other functions. In this model, the entirety of each part of the library is fully open, but only to the extent of the API calls which expose it. Would this be too "C-Style"? Would it be better to still internally use member functions and various levels of access for each class and just expose the parts which are user-accessible?

This is a tough question to answer; I think to make this as flexible as possible, it will use a RESTful framework. It fits the bill quite well actually. There are a bunch of different operations made; HTTP is used to communicate; J-SON is used to send commands and parameters as well as return values. The answer to that question should not affect how the user interacts with the library, only how it performs the operations.

Will local-host network communications be too slow for fast-paced operation? I do not think so. Human's reaction times are about 200ms at a minimum, not to mention the joystick operations among other things are already being sent over wireless without any noticeable lag, which if anything is higher than local-host connections.

OK. I kind of danced around that earlier question. There are a few options there.

I could make every method which modifies an object a member function of the object and then find the operation on that object to call based on the submitted request. This means any new features emphasizes an objects ability to do different things rather than the ability for outside forces to change how the object is used.

Objects could be completely open to all API methods, but not externally. Now objects can be modified by any function which takes it as a parameter and must be responsible to not put it in a conflicting state. This means more bugs can happen, but is the most flexible.

One could also combine the two. Any type of action where the object it itself "doing" something will be a member function, and any action where the object is having something done to it is a static function. Where is the line drawn though? Maybe a rule of thumb is to have objects only specify what is allowed to be done to them and ensure the integrity of their own state and anything involving calculations or aggregate actions are non-member functions.

Does it matter? The last one mentioned has the distinct advantage of not requiring each command to ensure the integrity of the objects they're modifying's state. That is centralized to each class. I cannot think of any other reason either is better or worse.

[KevinMackenzie]

The application consuming the library will still use the REST API, but should it use local-host HTTP requests, or should the application just call the REST API directly from the library? I thought for the HTTP requests that the client will also be waiting for REST calls from the server for event callbacks registered with the server, but how would this work if HTTP is not being used by the consuming application? Are there any REST architectural guidelines which give advice about this?

[KevinMackenzie]

I think some of my first assumptions were wrong. When the REST API is being used locally, should it follow the same idea of 'global function being passed an object ID/Name' like OpenGL, or should it be more representative of the internal class structure. To keep consistent API usage, I think even the java usage within the consuming application should use the same API format as the HTTP version, just not use HTTP requests to save performance.

I also figured out how to do callbacks. Have the client-side just call a simple PopCallbackQueue command which will return an array of event information which the application is responsible for parsing and handling. The server should never send a request to the client. The same system can work in the consuming application.

Maybe there can be bindings for various languages which can abstract the HTTP interface away and expose the API as the "consuming application" sees it for ease of use.

[KevinMackenzie]

With the idea of PopCallbackQueue how would the server figure out which client to give the callback to? From my rudimentary knowledge of REST, it goes against the principle of not relying on previous interactions with the client to store a separate callback queue for each client, but is there any other way to do it?

[KevinMackenzie]

The REST API should return all of the information about an object in one GET request to cut back http overhead. Making a request with only the member variables to change in the JSON will be how to efficiently PUT new member variables. Unchanged variables will be omitted from the JSON submission.

The URL of the submission should represent the 'noun' and the different http verbs will do different things and will definitely serve multiple purposes depending on the noun. The creation of subsystems will still be supported over REST, therefore subsystems will be top-level nouns and sub-subsystems, or subsystem components will be within the subsystem noun (i.e. subsystem foo, subsystem component bar: foo/bar)

[KevinMackenzie]

I was thinking about using C# as the language for this whole deal. There appears to already be a c# implementation of the WPILib, so maybe we should work off of that? It looks like the programmers coming up are doing java, so I do not know if doing this in c# is the best idea. I like the language features of c# better, but for most of the functional code there are equivalents. Its similar enough to java.

https://github.com/robotdotnet/WPILib/wiki

[KevinMackenzie]

C# also has some really easy tools for creating webservers, so that might be the best option when using REST. I also have some more thought about the how the local source will handle the REST API. Again, the latency of local-host HTTP calls still needs to be tested for significance, but the consuming application could short circuit the system by directly calling the HTTP request-handling methods with some wrapper functions.

[nicklausd]

I'll chime in with some input for the first time in a while, didn't show up to help/mentor at all this season so I feel I owe a little bit.

C# is a good choice for this. It's similar to Java as you say, so any team members with experience in Java can make an easy transition to C#. C# is also just a way nicer language than Java in general as Microsoft has been putting a ton of development into making it a great modern language whereas Oracle has stalled on that front making Java lack many nice features from C#. I think the biggest thing to consider when switching though is the stability of the C# bindings of WPILib. Since it appears to be a community implementation, you would need to consider the risk of it incorrectly or not completely implementing parts of WPILib (though it does seem to have everything but NIVision implemented, which is good).

I've also worked with C# for the past forever and am heavily biased, as it is tied with C++ for my favorite programming language. But the upside of that is I've worked with C# for the past forever and know the language very, very well, so you'd have an adult (legally speaking, at least; I'm a child at heart still and miss the days of being a student on the team) who can provide help when necessary.

(also I haven't read most of the rest of your comments in this thread yet so I'll get around to that and see if I can be any help regarding the more general purpose of this thread, but I saw talk about C# and wanted to give my thoughts on that)

[KevinMackenzie]

After some thought I have realized the previous approach is partially wrong. Instead of setting this whole thing up like a webserver as a base, the webserver should only be a view of the model. Another view I would like is a bash-like command line view.

The actual model should be loaded from some sort of file which represents the structure of all of the different subsystems and the locations of scripts within a .zip file that execute on different events. Additionally custom member functions for these classes can be added with scripts. Lua would be a great language for that. Now all of the subsystem programming can be done visually with custom logic being in script files. This should be transparent to the underlying WPIlib functions with additional ones added as development continues.

Using scripts should be the last resort for the code. Simple things which need custom logic, such as PID output/input ratios, offsets will be built into the non-scripted configuration.

Custom classes which do not fit in as subsystems of the robot will be dynamically loaded from a c# assembly at runtime and NLua will have access to these classes in the script files.

Commands will be slightly different than subsystems, for they will be entirely be Lua scripts for each command/command group. They are so simple they do not really need to be abstracted down to a GUI like the subsystems should. Command groups will be just as simple too. The actual mapping of these commands to joystick might be done in an XML/JSON file which is loaded at runtime. The other option is to abandon the driver station's joystick messages and switch to using the REST api view to send these events and have the button mapping be on the driver station (the downside of which is needing to keep all driver station computers updated). This will support command groups, commands with constant data parameters, and commands with parameters which are data members from the subsystem graph. A syntax for accessing this graph from within the XML/JSON file will need to be determined.

All of this must be supported by a development GUI which also supports deploying to the robot.

[KevinMackenzie]

This is great and all for the developemnt, but to get good diagnostics, methods must be inserted into the WPI lib for getting ALL of the information about everything so the REST api GET calls return accurate and verbose information. There should be a variety of verbosities, and all of this information should be logged to some log file which can be easily viewed by the driver station computer. Is there a way to add custom data to the existing robot log (which can be already be viewed in the DS_LogViewer or something like that? If so might as well use existing code, if not just make a custom log file which excludes some of what is already in the existing log.

How will all of this data be stored; the roborio does not have very much flash memory. A flash drive could be used, and by using a USB hub on the roborio, that is feasable.

[Slabity]

A popular method for solving this sort of problem is by using a publisher-subscriber pattern among your systems. It's used in robotics and diagnostics systems like ROS with great success.

[KevinMackenzie]

I need to do some more research for memory usage of c# webservices on low memory systems, because idle the barebones web service uses 100MB of RAM, but it might be less conservative on a machine with a lot of memory. I think we have 100-150 MB to work with after OS is loaded. P.S. loving new VS2015 features for c#

[KevinMackenzie]

It looks like all of the new features I am looking at adding the robot application can actually just be added on top of the existing WPILib. This means the robot program will consume the c# library and act as a layer on top of it for consuming the visual programming model files, script files, etc. along with creating the REST service. All of the new things added are composites of existing items and do not need special access from inside the WPILib.

[KevinMackenzie]

PERMANENTLY transferring this project to "RobotBuilder.NET"

(Note: this may turn into a fork of the robotdotnet/WPILib if additional hooks for diagnostics are needed)