lib file auto-generation script

[dstroy0]

I'm going to start work on one in python, since I don't know much python this'll be fun. It'll be aimed at user commands.

I'm looking at using tkinter to build the gui, again I don't know how to yet so this may change.

Require the user to copy/paste local/local include the library, add one line to their setup and loop functions. It should generate/edit: InputHandler/src/config/config.h - e ih_cli/ - g ih_cli/cli_setup.h - g ih_cli/cli_commandparameters.h - g ih_cli/cli_functions.h - g ih_cli/cli_functions.cpp - g

[dstroy0]

I think it should work something like: user copies library into their sketch directory user runs this tool and sets their options tool generates a directory inside of the sketch directory "ih_cli" by default and populates it with files user adds one line to their setup() and one line to their loop()

..anything I've missed..

cli just works... and we celebrate?

I think this will be pretty straightforward, I also want to give users the option of saving their options, so probably a json dump to txt in the tool directory. That'll make it easy for users to share configs.

[2bndy5]

I should point out that python std libs already has a very flexible CLI-based lib named argparse.

If your aim is to create a python wrapper, then it would help to first make this lib compatible with C++ std libs -- I'm guessing Linux isn't the only platform you're aiming for in a python wrapper.

If you're goal is to create a auto-gen options tool, then that seems like a worthy beginner project. It would involve the os lib and the argparse std libs. I like python 😉

[dstroy0]

Yes, auto-gen the files from the options set in the tool! I'm using visualtk to start learning how tkinter works, I thought it was a good jumping in point too.

[2bndy5]

CMake also has a mechanism that can be used to generate files based on template files. Most people just use it to inject the current version of a lib into a header (or modify Doxygen's Doxyfile in the same manner).

[dstroy0]

I'm thinking that this will fit into two tabs in the app, the first with radio buttons/check boxes/input fields for process setup and the second tab will be for generating CommandParameters command trees, I think that a dropdown tree-list view here would be most appropriate. Then we can have a menu bar with just file, generate, and about. File can be open/save/save as, it'll just be a json dump with the script/lib version and options/commands set by the user. Generate should generate the cli files the directory above InputHandler inside of the user specified directory name or "ih_cli" by default. About should credit us and link to the git and docs. Thoughts?

[2bndy5]

It sounds like you want a user project generator. PlatformIO is also written in python and may be useful in the generation process; the pio project init (CLI command) may be very useful. you might be able to use parts of the pio pkg API for a more granular control (like updating existing projects), but I've never needed to use PIO like that.

As for the GUI, it sounds like you have a definitive idea 👍🏼.

[dstroy0]

Thanks for the help!

[dstroy0]

Python is neat, I don't know nearly enough about environments or how to set them up that slowed me down a lot. I've had a couple problems trying to use the latest python version with some of the tools. It seems like I've got my learning environment set up now though.

[2bndy5]

Python is neat

Python is a great high level language, but it isn't as fast as C++ (not sure if any other language can beat that).

I don't know nearly enough about environments or how to set them up ...

Using virtual envs is a good habit to harden! I only recently had to break my bad habit of not using them. Python comes with venv which is a builtin module/pkg/lib that can be used to create a virtual envs.

python -m venv .venv

invokes the python interpreter in module mode (-m), and uses the venv module to create a virtual env called .env. You'll notice a new folder named ".env" in your current folder; you can specify an absolute or relative path when naming the virtual env.

To activate the new v-env ("venv" for short), it depends on your OS.

1. Linux uses

   source .env/bin/activate

2. Windows uses

   ./.env/Scripts/activate

   However, you may have to adjust a permission in Windows to allow executing a PowerShell (PS) script (namely the activate script).

---

PlatformIO's installation prefers a fresh venv to isolate the build mechanics in vscode. I'm not sure if pip installing pio is the preferred method anymore (as it may require some non-python dependencies - like compilers/toolchains).

I've had a couple problems trying to use the latest python version

I have seen more compatibilty problems with python 3.10.x than any other iteration. This is mainly because there was a lot of deprecated API (in favor of using newer drop-in-compatible substitutes) that became due for removal.

It is very common to use

import sys
if sys.version_info < (3, 10):
    # do stuff for python 3.9 (& prior)
else:
    # do stuff for python 3.10

or you can use try ... except blocks, but that isn't as obvious as using sys.version_info.

[dstroy0]

Awesome, I've got a couple weeks of learning ahead of me! :)

[dstroy0]

The header spacing isn't right for the table, I'll probably do 30/70. it will look like this on the command tree tab as well but with a tree widget and a Generate button (which is also available in File and as a kb shortcut). I'm just going to use the same icon as the lib. I appreciate all your help and input. I've been playing with the script getting used to different functions and making buttons do things. Now I'm going through the UI and making object names so I can do that stuff programmatically after designing the ui file and using uic to load it in the script, then hooking the objects and doing the business with them.

[dstroy0]

I don't like the whitespace under the code preview, and I think the button group should be centered below the table

[dstroy0]

Code preview should be an expanding list, where the list items are file names and it should display the same things no matter which tab you are in

[2bndy5]

Interesting. I've never used QT Design.

[dstroy0]

It's making fiddly things like placement super easy.

[dstroy0]

So if you've got thoughts or ideas just let me know. We can add/remove menu items, whatever. It's easy with this thing, I'm still figuring it out but it's pretty intuitive.

[2bndy5]

My only real GUI experience has been with kivy (python) or flutter (dart).

[dstroy0]

Awesome, check out the ui script now. I haven't hooked into any of the objects yet but it's super easy. The .ui file is just an xml generated by qt designer

[dstroy0]

It's using uic to load right from the ui file

[2bndy5]

I'm not saying you should try anything different (if you want to, then that's fine too). It's probably best to stick with what you're familiar with. Different GUI builders aren't compatible with each other.

Are you working from a local branch? Is there stuff committed that I can dry run? Probably best to keep the UI app in a separate folder at repo root.

I can look into making a CI workflow that will build the UI app... But I need to get my hands dirty first.

[dstroy0]

Nope, everything you need is cli_gen_tool; the .pyqt folder at the repo root is generated by an extension for vscode and it just makes objects for me. The modules you need to run main.py in cli_gen_tool are PyQt5 and PyQt5-tools.

[dstroy0]

PyQt5-tools gets you Qt designer as well, so you can open the .ui file in designer. I'm open to learning anything, I just started using this and watched a few videos and it seems easy enough. You know way more than me; this is my first ui attempt.

[dstroy0]

Ok, sorry the .qt_for_python is generated by a vscode extension, it uses pyqt uic and pyside6 generates all of the objects based off of the names you pick in qt designer so it saves you some time, you can copy/paste things from there to the main.py in cli_gen_tool to add functionality to buttons etc. If you look in junkyard.py I was manually setting things up in there just to learn about some of the stuff pyqt can do.

[2bndy5]

ok I'm having a peek right now... I have a few questions:

1. Any objection to adding a venv folder to the repo's gitignore? This way I can use a dedicated venv for the repo, and VSCode will/should automatically detect and ask to use the venv in the repo.
2. Any objection to restructuring the _cli_gentool folder into a python package? This can be used for adding requirements.txt and separating out parts of the app into different python scripts (aka "modules").
3. What VSCode ext are you using? I don't see any official QT exts in the VSCode marketplace.
4. I'm still confused about the difference between main.py and cli_tool_gen.py. Is one script for CLI only and the other for GUI only?

[dstroy0]

1. Structure away, you know way more about that than me.
2. Same as number one, I'll learn from you how it should be done.
3. https://marketplace.visualstudio.com/items?itemName=seanwu.vscode-qt-for-python pip install PySide6 this is what generates main.py in .qt_for_python and THAT main.py has all the object hooks for the widgets in cli_gen_tool/cli_gen_tool.py but it can also launch the ui standalone as well there's just a bunch more modules that way.

[dstroy0]

You can get the .qt_for_python dir output by using the compile form option in vscode and selecting the main.ui file as the target.

[dstroy0]

4. main.py is generated by qt_for_python from main.ui

cli_gen_tool.py is a uic wrapper that loads main.ui and we can copy some things from main.py into cli_gen_tool.py to save time and learn how to set options correctly.

Whew I think I got it right that time, this is complicated and I don't understand it fully, sorry.

[2bndy5]

I think using VSCode is better than using QT Creator IDE (not as intuitive as VSCode - for python at least). I've installed the VSCode ext and it generated way more than just main.py.

I'm not sure why anything needs to be generated. I'm learning as much as you are when it comes to using QT for a GUI. It looks like PyQTx is a aftermarket lib that consolidates all the PySidex pkg. IDK the purpose of having a "go-between" lib when PySidex would be installed. Is there a inconvenience to implementing both a py file and a ui file?

• Is there a reason you're using PyQT5 instead of PyQT6?

---

PS - You may have already thought about this, but beginner projects generally don't involve a GUI. However, I'm willing to learn with you...

[dstroy0]

I'm just using qt designer to make the ui file, not the qt creator ide, we can use pyqt6 no problem.

I'd like to get the code preview to be a collapsible list of files that will be edited or created based on the options set.

Making the gui seems simple enough... Generating the code shouldn't be too hard either.

[2bndy5]

I would suggest starting with a CLI only module. Then you can create a GUI that calls functions from the CLI module.

I'll keep reading up on using QT (its very popular). I think I understand now, this tutorial states that the ui file needs to be translated to python code. This approach is a little different from other GUI libs that I've used.

PyQTx isn't advantageous to this app. I think we can simplify things here by not using PyQTx and just stick with PySidex.

I've installed QT v6.3 as that was the latest release. So, I'm using PySide6 for my learning purposes (as that is also suggested in the QT tutorials).

[2bndy5]

Found this example of a settings editor, but it doesn't use a ui file.

[dstroy0]

I wouldn't know where to start with building a cli only version. That example is really nice, it's the way I would like to display the options and the code preview. I'm just having fun using the qt designer and learning python!

[dstroy0]

Found this example of a settings editor, but it doesn't use a ui file.

Pyside generates its code from ui files, then we go in and add functionality to the objects programmatically, is how I understand how the process works so far.

[dstroy0]

I added the venv to the gitignore. Here's the packages I installed to dev the script, if I need PyQt6 I'll grab it too.

(.venv) PS C:\Users\Douglas\Documents\Arduino\libraries\InputHandler> pip list
Package            Version
------------------ -------
pip                22.1.2
PySide6            6.3.0
PySide6-Addons     6.3.0
PySide6-Essentials 6.3.0
setuptools         62.3.3
shiboken6          6.3.0

[dstroy0]

I pushed a barebones script that loads the ui file so we can reposition things in it or add things and not break the rest of the script

[2bndy5]

Here's the packages I installed to dev the script

Pretty sure that installing PySide6 will install all others (even setuptools because it's a CPython C-Extension).

I'd recommend that you start a requirements.txt file and put PySide6 in it. If you find your app using other pip-installable pkgs, then you can add them to that txt file (one pkg per line). This means that all a new dev would have to do is

python -m venv .venv
# activate .venv
python -m pip install -r cli_gen_tool/requirements.txt

[dstroy0]

ok, requirements.txt generated from pip freeze > requirements.txt

[dstroy0]

I also pushed another script, this one is pure python generated from the ui file with qt for python. We can embed our classes inside of that. The thing that sucks is that any time the ui file is changed, the mainwindow class gets regenerated so for ui updates our classes would need to be added to the generated mainwindow class again.

[dstroy0]

I don't think this will change once it's made though.

[2bndy5]

ok, requirements.txt generated from pip freeze > requirements.txt

Using pip freeze is a bad habit because it includes all dependencies that you shouldn't need to manage. For instance, you pip freeze PySide6 which requires something like setuptools^=40.0, but then PySide6 changes their dependencies to require setuptools>=45. Managing dependencies of dependencies shouldn't hinder you dev cycle. BTW, the version numbers in my example are wrong, but you get the point.

[dstroy0]

Thanks for the warning! I'm learning a lot, this is fun!

[dstroy0]

I connected the button/action objects to methods in MainWindow class in cli_gen_tool.py. Now I'm figuring out how to implement the cool settings view you found.

[dstroy0]

the about/about and about/help menu items are functional edit: I mean the help menu is functional :)

[2bndy5]

Sounds exciting

[dstroy0]

It's pretty cool! The open file dialog is working.

[dstroy0]

I truly have no idea how to structure python, should I break out functions that are longer than 20 or so lines from the class?

[dstroy0]

So far everything has been a static function so the methods are short and sweet.

[2bndy5]

20 lines is acceptable in my experience. It gets hard to follow algorithms when the functions are longer than 100 lines or have other functions defined within a function.

There's no explicitly bad practice against defining functions within functions like so:

def func(arg1: int) -> bool:
    """doc string goes here."""

    some_local_var = True

    def local_func() -> bool:
        return some_local_var

    if isinstance(arg1, int):  # proper type checking
        return local_func()
    return False

I've really only seen this done when a JS dev tries their hand at python, and it feels just as hacky as it does in JS. Notice I've added type hints in the function definitions; this is a good habit to start doing (but might get complicated if abusing python's dynamic typing). Type hinting does not get used at runtime (its only meant for type checking tools and readabillity).

[dstroy0]

I'm very happy you're mentoring me through this. It feels much less intimidating. Pretty excited that I got as much done today as I did because I had to read the docs a bunch.

[dstroy0]

Is there a better way to get the os name besides https://docs.python.org/3.9/library/os.html#os.uname ?

import os

uname_tuple = os.uname()
os_name = uname_tuple[0] # sysname

I haven't ran this yet I've just been thinking about how to make the about hyperlink work cross platform with if elif inside of the about method.