binary build/testing

[dstroy0]

I think it might be the __init__.py in /tools/cli_gen_tool_src/ so I'm building it right now without that and seeing if it works. This is with pyinstaller --onefile on wsl 2 Ubuntu 20.04LTS.

[dstroy0]

Sorry, I'm getting this on trying to launch:

dstroy0@Doug:/mnt/c/Users/Douglas/Documents/Arduino/libraries/InputHandler/tools/bin/latest$ ./cli_gen_tool
Traceback (most recent call last):
  File "cli_gen_tool_src/cli_gen_tool.py", line 32, in <module>
ModuleNotFoundError: No module named 'modules'
[27808] Failed to execute script 'cli_gen_tool' due to unhandled exception!

[dstroy0]

It's trying to launch I just need to get the display working...

[2bndy5]

From the pyinstaller docs landing page:

PyInstaller is tested against Windows, MacOS X, and Linux. However, it is not a cross-compiler; to make a Windows app you run PyInstaller on Windows, and to make a Linux app you run it on Linux, etc.

This may apply to the QT python pkgs (which are c-extensions). Meaning, installing pyside in windows and running pyinstaller will produce an app only compatible with windows. You may need the CI to build an app with pyinstaller for each platform. It would be a good idea to test run the built executable in the CI for validation.

Making sure it is compatible with desktop env is a hard thing to achieve in CI though.

[2bndy5]

See also pyinstaller's supported modules. It claims "partial" support for pyside.

[dstroy0]

Yeah, I'm trying to get the exe to launch in linux and work my way from there.

[2bndy5]

It might help to use relative imports in the src (would require a __init__.py in cli_gen_tool_src though). For example:

-from modules.uic.preferencesDialog import Ui_Preferences
+from .modules.uic.preferencesDialog import Ui_Preferences

[dstroy0]

On windows I need to add /system32/downlevel/ to PATH before running pyinstaller to build. It's where one of the missing .dll are.

[dstroy0]

The modulenotfound error was being caused by the init file, it messes with pyinstallers module search or something. Half of my problems the other day were related to my Linux ineptitude. Which is better, having actions user push commits or uploading artifacts to the branch?

[dstroy0]

I also need to change the workflow push action so it only happens once, at the end of the workflow.

[2bndy5]

I still think uploading artifacts is the way to go because I have a personal problem (more like a security concern) with letting someone else's code push binary files to a repo I own/maintain.

[dstroy0]

I changed the workflow quite a bit. I think this is a good way to do it, I need to archive the binary builds to reduce their filesize though.

So far, it's just pathing problems with the binaries, which can be fixed once the script is complete.

[dstroy0]

Also on the windows runner it's having problems finding the .dll in /downlevel/, it's in both system32 and syswow64 so I tried adding both paths, but I don't know if it's in \hostedtoolcache\windows or just \windows\.

[2bndy5]

As a last resort, you could use

    - if: runner.os == 'Windows'
      run: dir folder_name

I tend to use this when I find myself about to bang my head against the wall. Just remember to remove the debugging output before merging into main.

[dstroy0]

I'm working on zipping the binaries right now

[dstroy0]

As it turns out, that particular .dll that it is asking for on the windows runner is no longer packaged with windows so I had to grab it and I put it in /tools/resources.

[dstroy0]

It's building and zipping them now, the linux binary is about 80mb so it should stay under the 100mb limit.

[2bndy5]

As it turns out, that particular .dll that it is asking for on the windows runner is no longer packaged with windows so I had to grab it and I put it in /tools/resources.

I think that might have to do with the github runner using the server edition of windows?? Very strange.

[2bndy5]

I don't think that release assets are constrained to a size, but they are constrained by quantity (I doubt you'll ever hit the quantity max). So, you wouldn't have to worry about the size when publishing releases.

[dstroy0]

I made a new class to specialize the init of the script, that's where the path awareness will be happening. I added most of what is needed already.

[dstroy0]

I was trying for a while to figure out the pyinstaller command syntax for the add binary flag and I think I did, but that didn’t fix the warning. I’ll try adding it to PATH different ways again I guess. maybe copying it to one of the existing paths?

[2bndy5]

I've never used pyinstaller before. All my libs that ship with an executable script all define the executable's entrypoint in the setup.py or pyproject.toml. I also haven't ever had to contend with external DLLs either.

[dstroy0]

The tool script itself runs fine on linux, windows and macos, building the binaries is a fun challenge.

[dstroy0]

I think I've got through most of the pathing, everything is converted to native separators and the script uses absolute paths for everything. The init class asks for the lib dir if it's not inside of it, like if someone moved a binary outside of the lib.

[dstroy0]

I'm going to move the tool save location out of the lib folder to the users' home directory in os.environ. The binaries are being built successfully on the binaries branch, a workflow pulls the tools+docs/img dirs from main to binaries, then builds and zips them there. It'll work the same for release, but the dir will be named release/ and won't remove prior versions like the 'latest' workflow does. I'm going to require a passing release bin build for the three supported os before the release workflow will pass. Also, pyinstaller has a version flag which needs to be set and I should append that same version number to the zipped filename.

[dstroy0]

Just need to do a little more testing with .development removed from the lib root. Any json storage inside of the lib folder has been eliminated except the testing cli json. I'm going to make a branch called tests or testing and pull the lib into that to run the testing workflows to get rid of the generated clis in the lib.

[dstroy0]

binaries are building consistently, I will reopen this issue if they stop