Provide a CLI to be run qmake

[davidzwa]

As was already discussed, we'd possibly like to enhance qmake in a most non-intrusive way with very little maintanance. The reason is for the required FIREBASE_SDK_CPP environment variable which needs to be picked up by the deployment target at hand (Windows, Linux, iOS, Linux Android).

I'd like to think out how we would approach such a thing:

• qmake .pri with more qmake feedback: see it as a pit-stop before we launch the dreaded make and deploy
• cli script: I find powershell core quite nice and this is cross-platform these days (Windows, mac, Linux). Python 3.7+ would be my second option, also very cross-platform.

The CLI would have the following requirements:

• It's optional at all times, only helps, validates, etc
• It's smart and versioned
• Easy to download
• It knows what Qt the user runs and is allowed to bail out on 'sorry, this is too old/new'
• It has 3 modes: qmake/deploy-prep, dry-run analysis and prep/download files like firebase SDK
• It provides consistent results (each run should be the same) and should not tamper with the project using QtFirebase
• It provides correct output (+ has a verbose/non-verbose mode or logs)
• It's as fast as qmake (doesnt slow it down except for the first run)
• It can download Firebase SDK C++
• It only helps analysis and androiddeployqt

Don't worry about the work/effort: I will take care of it. It's more a question of if you'd be interested in it and if the requirements align with what you think of it. Only then I will even start thinking about making such a plugin qmake tool.

[larpon]

A bulletproof CLI tool would be wonderful, no doubt.

I'm quite tempted to just let you do whatever you want - it just needs to work on Linux, macOS and Windows :slightly_smiling_face:

I have zero knowledge about powershell... How would I run it on my Linux machine?

[larpon]

... actually having the tool as a separate project might be even better - that way support tickets for the tool don't end up here - or at least I have a place to send people to, if you catch my drift. We can just make it an official "qtfirebase-setup-tool"?

[davidzwa]

Let's first work out the details and then decide on this repo or another. It would still be nice if we are both co-owner to the tool's repo in that case. The tool should help your troubles with understanding issues, so to be able to adjust it is in your benefit.

Powershell for Windows, Linux and Mac are:

• Core variants (v6 and 7) -> cross-platform

Powershell for Windows

• .NET variant (5) -> not cross-platform, but code-compatible
• Core variants (v6 and 7) -> cross-platform

Powershell on Windows is directly installable as setup, side-to-side with another version of pwsh on Path. Powershell on Mac is installable with Brew. Powershell on Linux is first-tier supported for these distro's:

• Ubuntu 16.04
• Ubuntu 18.04
• Debian 8
• Debian 9
• Debian 10
• Alpine 3.9 and 3.10
• CentOS 7
• Red Hat Enterprise Linux (RHEL) 7
• Fedora 28
• Fedora 29
• Fedora 30
• openSUSE 42.3
• openSUSE Leap 15

And a lot more by community (so basically everything).

The way I would install in on Ubuntu is with snapd as a snap package as I had issues with direct source install.

I do think powershell is a great scripting tool, but as I said: so is Python (which can do more). What has your preference?

[larpon]

Hmm... I see. :thinking: I think we'll have to go with python here.

My impression about what users find the most "hard"/confusing is currently:

1. All the places you need to set the sdk/ndk paths (ideally it would follow the value of an entry from the users .pro or some kind of external small key/value file ... but ... you know ... qmake :man_shrugging: ... ).
2. Some kind of assurance to which/each part of the setup "looks ok" (some kind of validation of at least SDK and NDK paths)
3. Setup of messaging (on iOS there's a lot of web interface fiddling which we can't do much about?)
4. "Exotic" configs when not using all firebase modules at once - what to include/exclude in .pro vs. build.gradle Info.plist etc.
5. Probably much more I never see in the issues :)

I guess the tool could cover a lot of the above. It's also important that the tool can be re-run (here I'm thinking it has to be able to change SDK/NDK paths continuously).

I have no problem with introducing code/format conventions to help the tool (I.e QTFIREBASE_SDK_PATH shoudl always have this or that format in the .pro etc.) - given it's impossible to work around otherwise :slightly_smiling_face:

[davidzwa]

Nice, so what I'd like to start with is the qmake python interaction. How to run the tool and validate the existence of the right python. We dont want users to be forced to install python, but I would suggest 'if you want help, run the tool' approach. Or to put it differently: 'if you report an issue, include the tools logs'.

Second up for design is the cli itself. My preference is to design: the arguments with argparse and a config for static arguments. Argparse can merge them easily. Thoughts?

[larpon]

@davidzwa - that sound wonderful! It'd be nice with some more info in the issues for sure.

the arguments with argparse and a config for static arguments. <...>

Sounds good to me - no need to have 2 different input formats to the tool at this point.

Thanks for your input and efforts here - much obliged!

[davidzwa]

The efforts are mutual, and I cant stand to see something so worthy to fall over because of external changes. This tool is what you have been missing: isolation of problems.

The next TLDR will cover:

• Qmake/python interaction
• Dependencies (avoiding cry-baby issues: isolating package dependencies, placing responsibility to user, not us.)
• Configuration file description

Qmake/python interaction

Ok so about the qmake interaction, we have to make sure either the tool is blazing fast or that it is run only when needed (bail on a valid/non-dirty state). I hope to focus on the first, but I wont be afraid to fall back to the second. About the CLI from qmake, I'm thinking about this basic call: exit_code = $$system($$pwd + '/qtfirebase_cli.py') This should listen for the returned error code, and continue 'just fine if exit_code == 0' otherwise the qmake should error out and quit with a reference to where the analysis log can be found. I mean, you ran the tool for a reason: save time, get analysis before make/deploy.

Dependencies:

• Will not install dependencies, but we will provide both a Pipfile and requirements.txt file for people to get started. If they do not install dependencies, we cannot guarantee a downloaded and extracted Firebase SDK. This is not our responsibility: we just provide ease of setup.
• Will use Pipfile (+Pipfile) on the repository as primary focus, as this is a self-healing solution for dependencies.
• Will import default logging, as we depend on filtered logging. Default in python.
• Will import default argparse, as we need a correct CLI interaction.
• Will import installed requests only when configuration says 'download firebase SDK|Android-SDK|iOS-SDK to be non-falsy'

  "Could not find requests and you've configured QtFirebaseCLI to download firebase for you, which was found to be missing. Please install dependencies from the Pipfile using Pipenv or requirements.txt using pip. Skipping SDK download. Expecting build failure!"

• Will import installed py7zr for 7zip only when configuration says 'download firebase SDK|Android-SDK|iOS-SDK to be non-falsy'
• We will not support Python 2 (https://www.python.org/doc/sunset-python-2/)
• We will have to declare non-support for 3.7 in half a year: this tool is meant for serious development, and not outdated systems. (so 3.7/3.8/3.9 is our focus for now, after 2020 only 3.8/3.9)

Configuration file description

As per config I'm thinking of at least the following requirements:

• PYTHON_PATH unset/set to valid path with python binary (I want people to be able to freely choose a python version, as well as use their default python on the command line)
• All other commands are also available per cli arguments in the format of

  [-a action] [-l log_level] [-t target] [-build_args ... any future SDK/NDK/ROOT arguments passed into the CLI as keyvalue or JSON]

  [QtFirebaseAnalysisConfig]
  PYTHON_PATH="optional_python_3.7+_path_here,advise_leave_unset"
  log_level=debug|3|information|2|warning|1|silent|0
  target=ios|android|other
  logs="optional_existingornonexisting_relativeorabsolute_path_to_analsysis_logs"
  action=analysis|sync

Please read it through, I know it's quite in-depth, so you can skip the stuff non-essential to QtFirebase <-> deployment as long as you validate the following design choice:

Users of QtFirebase accept that they need to setup their system properly concerning Python, if they don't, this tool cannot provide them the full ease of setup. If they cannot install requests/py7zr, we provide fall-back to downloading Firebase to a location specified by default or specified manually.

[larpon]

@davidzwa it looks really good - engage! 😀

[davidzwa]

Will start reporting once started, nice task with a bit of change

[larpon]

Looking forward to see it! I've been wanting something like this for the project in ages.

[davidzwa]

Running python from QMake:

Project MESSAGE: INFO: 2020-08-03 23:53:12,646 [run_cli.py:6]: qtfirebase cli quit sucessfully
Project MESSAGE: INFO: 2020-08-03 23:53:12,733 [run_cli.py:6]: qtfirebase cli quit sucessfully

Running python from QMake with error:

Project MESSAGE: CRITICAL: 2020-08-03 23:54:09,571 [run_cli.py:7]: qtfirebase cli had a big ouch
Project MESSAGE: CRITICAL: 2020-08-03 23:54:09,658 [run_cli.py:7]: qtfirebase cli had a big ouch

W.I.P. please read the TODO.md to discuss what's next after I'm done with the tasks for the upcoming week/ 2 weeks.

https://github.com/davidzwa/QtFirebaseCLI

Hope you like my 2 hours of slamming a keyboard really hard

[larpon]

@davidzwa looks good at a glance - nice job! I'm a bit busy these days but will get back ASAP. Thanks for doing this!

[davidzwa]

No worries, will continue as planned next time I find free time.