hiromon0125 commented 4 months ago

Python Overhaul (Creation of Stable and Solid Foundation)

\ “Any fool can write code that a computer can understand. Good programmers write code that humans can understand.” Martin Fowler

“You might not think that programmers are artists, but programming is an extremely creative profession. It's logic-based creativity.” John Romero .

[!WARNING] Read the goal of this PR, mentioned below, before looking at the code.

Feature

Complete python environment

[!NOTE] This is just the environment NOT the robot component classes.

What is in the environment/Why is it taking so long to implement?

When implementing robot software we can identify the code into several groups. Originally we separated by IO, controller, behavior, and anything higher, but the new environment organizes that code differently. Electrical component state/action, component structure, and test/main procedure script. This separation makes more sense for several reasons. The relevant electrical components are defined together. The Irrelevant components are separated into different folders and decoupled. The organization focuses on making hardware components and models be grouped. We also achieved this organization by the loosest coupling possible in the Python world while still being able to make sure that nothing that should not be coupled gets coupled during runtime.

How is this done?

The first thing I wanted to change was the component structure. By that, I mean how the electrical components(pins, potentiometer, motor, latch, etc.) relate to each other. The reason why I wanted to make this environment in the first place is that the hardware changes often for Catbot, and the way it changes is usually either pins got swapped, components got replaced, or an entire group of components got taken out. These changes are not specific to how the component behavior is defined but rather what the components have and don't have. This was solved with a config file. The reason it's not a Python file is that once the code is running, it is not likely to change for Catbot during runtime and it separates the implementation file from the component hierarchy. Therefore having it as just one config file will make it easier to track how the components relate to each other, and in a scenario where we want to just test one component, we can just give the software a small config file and run any test code with it. However, for this to work well, we needed everything to be defined by this single config file. This requires a universal syntax for how things are coded for each component so that the parser parsing the config file knows all of the components that exist and the parser can create the instance of the specific component referenced in the config file. This was the biggest challenge for this environment, and I extracted all of the internal implementations for how this linking of each component to the parser is done such that the component definition is not littered with the parser linking code that is not relevant to the component.

What is state management/store?

State management is one of the key factors that allows us to define each component structure with maximum flexibility. The state management comes from the idea of having a central location for all of the instances of the component held with an identifier(a string for Catbot). The benefit of state management shines in terms of how the components will be coupled in the regular component classes. When we defined components previously, we directly threw the instance of the smaller object into a bigger/parent object meaning the two components are coupled together, but what if you wanted to reverse that relation? What if from the smaller component, you wanted to get the bigger object and do operations with it? You have to create an event handler that was created from the larger object and pass it into the smaller component so that the event handler gets executed during the event. This is okay, but that requires us to code that reference during parsing. What if we could define that reference within the config file? That's exactly what state management allows us to do, we can throw instances of the objects within each other without having to go back to a previous instance of another object. What if we needed to keep the small object in the parser so that the small object can be shared among multiple large objects? The purpose of state management is that it lightens the burden on the parser and allows us to make a highly complex structure with minimal coupling and coding.

What is this syntax?

The syntax borrows its techniques from Functional programming and Procedural programming. Since we have state management, we could have just asked the state management for a certain component and done action with that component. However, this will significantly reduce the point of the state management. For example, when we have a function in a store that returns any component, we have to be confident that the component exists in the store, and on top of that the function exists within that component that we grabbed from the store. We could potentially do this solution and define the type of the returned object to be the correct component class, but that requires us to write variable = store.get("foo") as Foo and that as Foo syntax allows us to browse through the component class for the function we need, but that has multiple issues. We now have an object living outside of the store in variable making the store useless. We could also mistakenly call a function in another component "foo" that was not a Foo class and instead was a Bar class, but if they both have a function with the same name the call will pass and not throw an error. That is a safety risk. Not only that we have to define that variable as Foo class or we will lose the type hinting. For every component we interact with, we will need to define a new variable. This is a stupid amount of work for a script that runs two legs in parallel. What if instead we import the function we want to use, and in that function, the grabbing of the state is already handled? This is what the syntax allows. Let's say we have a method for spinning a motor:

@motor_action
def spin_motor(motor_state: Motor, speed: int) -> None:
   # call some motor call

The decorator handles the grabbing of state and thus we just need to import this method in a location we want to use, and then we can pass the component identifier in the spot of the motor_state. Therefore, the main script syntax looks something like this.

# main test file
from "store/path" import setup
from "motor/module/path" import * as Motor_actions
from time import sleep

setup() # parses the config file and sets up the store

while (true):
   Motor_actions.spin_motor("motor 1", 10)
   Motor_actions.spin_motor("motor 2", 10)
   sleep(10)
   Motor_actions.spin_motor("motor 2", 0)
   Motor_actions.spin_motor("motor 3", 30)
   Motor_actions.spin_motor("motor 1", 0)
   Motor_actions.spin_motor("motor 3", 0)

We do not have to care where the object lives, and the test script file is now readable. I can now confidently give this script code to other engineers and not have to explain what the code does. It also gives me the confidence to pass this software to a non-software engineer and let them play with it without having to tell them how the software works. The software explains by itself.

This is just the start of what this syntax brings and its benefits continue:

imports build the parser and non-imported components are not setup or instantiated
components that share the same library class can be configured in one location as a generic component
the functions with lower components can be called without needing to understand the structure in the test file
central store can be printed to check for correct parsing
unit testing can be done within the same module

All of this software had to be written from scratch and carefully balanced the syntaxes such that it was a minimum effort with minimal code written for any robotics project.

But WHY do all of this when you could have just coded Catbot regularly?

I believe in thinking beyond the code that works: a code that can be used easily, be read easily, and be enjoyed during scripting. Making a program that works is easy, but making a program that does more than function is the challenge I like solving, and this PR is the result of such a challenge. Plus why not explore everything that a language can offer?

nrsprs commented 4 months ago

This is a very nice change and great application of Python for robotics programming. The syntax is very similar to the kinematic control libraries that I use at work, which is probably the easiest part of teaching non-project users about the tech stack. Using this, will you be writing docs for the store?

hiromon0125 commented 4 months ago

I don't see anything egregious here but I also don't know enough about functional Python to be able to comment on that code.

I am a bit concerned by this comment as the functional bit of this PR is only that it relied on decorators and the way it was written, which, to be fair, leaned a little too far to functional as well, and thus there is going to be another successor PR rolling back and a few bugs fixes. However, this shouldn't prevent you from looking further into the code base as there is nothing conceptually difficult. I recommend looking at the basic functionality of device.py and understanding what each part of the functions is doing and how it restricts the component files. Also, a friendly reminder that just because you are not an expert on something should not prevent you from gaining a deep understanding of it and opening up more conversations. Also note when this PR does get merged, anyone working on the Python code will be heavily relying on the syntax decisions I have made for all programs written in this repository.

thatnoobles commented 4 months ago

@hiromon0125 I did look into the codebase. As I said in my initial review, I understood what you were doing conceptually and it looked fine to me. I was more so referring to the specific syntax choices since I'm not familiar with this programming style, but I'll research more into it as we start more actively working on the behavioral level.

RIT-MDRC / Catbot