q version 3.1.0-beta - automatic immutable caching and direct queries on sqlite databases

[harelba]

Also includes new packaging mechanism using github actions and pyoxidizer.

Fingers Crossed :)

[SMillerDev]

Instead of hardcoding the formulae here, why not just use https://github.com/marketplace/actions/bump-homebrew-formula to create a PR? Or run brew bump-formula-pr

[harelba]

Instead of hardcoding the formulae here, why not just use https://github.com/marketplace/actions/bump-homebrew-formula to create a PR? Or run brew bump-formula-pr

Hi, thanks a lot for the suggestion.

The reason for hardcoding it is because I've completely changed the formula in this version (new and better packaging flow), so I preferred this way in order to make sure that the entire flow is working well.

I'm considering two options:

1. Issue a manual PR to brew for this change after everything is ironed-out, and then change the github action workflow to just use bump-formula-pr as you've mentioned.
2. Issue a manual PR from the github workflow every time

The advantage of option 2 is to leave the entire logic related to q's building/packaging in the same repository, which would allow easier evolution over time.

I'll make the final decision after the complete flow is working well, as it's part of a major change in q itself.

[SMillerDev]

The advantage of option 2 is to leave the entire logic related to q's building/packaging in the same repository, which would allow easier evolution over time.

You're not the one responsible for packaging in Homebrew though, Homebrew is. So there will be changes in the formula based on Homebrew and you'll have to port them back every time. Azure CLI is doing that, and it's an absolute pain to deal with.

Option 1 is really the only way to work together with the brew maintainers on packaging it.

[harelba]

I think I understand what you're saying - They update the formula themselves at some point right (with bottle information etc.). If they do, then indeed that would be a pain to manage.

The main disadvantage in option 1 is that any changes/additions i do to the build process in q require porting them to homebrew manually, and it creates an implicit coupling between homebrew and q build flow.

I guess I can control my own changes better than controlling homebrew's changes, so you're probably right.

Btw, building the formula and the bottle in q's workflow has one advantage - No manual steps at all in validating that the mac version fully works. I actually run the entire test suite of q itself on the resulting q executable (in the path) after installing it from the actual bottle.

I'll finish with productization of the entire flow, and then probably issue an initial PR to homebrew. After that, i'll just change the workflow so it uses the bump-pr subcommand.

[SMillerDev]

The main disadvantage in option 1 is that any changes/additions i do to the build process in q require porting them to homebrew manually, and it creates an implicit coupling between homebrew and q build flow.

There already is, Homebrew generally uses the upstream build commands but if upstream does something that doesn't fit the Homebrew model we will change it.

Btw, building the formula and the bottle in q's workflow has one advantage - No manual steps at all in validating that the mac version fully works.

Homebrew will always create it's own bottles though, and we have a test block in the formula to have CI validate if it works. On the Homebrew side nothing is manual.

https://docs.brew.sh/How-to-Create-and-Maintain-a-Tap#official-vendor-taps might be an interesting read regarding this since it touches on some of the wishes I think you're expressing here.

[harelba]

There already is, Homebrew generally uses the upstream build commands but if upstream does something that doesn't fit the Homebrew model we will change it.

Could you describe what you mean by that? Is there a best-practice way to make sure that the actual "def install" and "test" section would reside in my repository, while the rest of the homebrew formula will reside in homebrew itself? Idealy, I would like to manage the formula in the same repository as the code, as they're tightly coupled in terms of logic.

Taps are a good option, but in my view they are a bit more cumbersome for usage, and also convey a lesser sense of trust from the users. Also, if i'm not missing something, then it still means that there is a need to manage the same logic in two different repositories.

[SMillerDev]

Could you describe what you mean by that? Is there a best-practice way to make sure that the actual "def install" and "test" section would reside in my repository, while the rest of the homebrew formula will reside in homebrew itself? Idealy, I would like to manage the formula in the same repository as the code, as they're tightly coupled in terms of logic.

This will never happen for a Homebrew managed formula. You are the developer of the code, Homebrew packages it. You can help package it, but you get the same control over it as every other member of the community. Your software being open source allows it to package it however we see fit, the only option to have more control then that is making the source closed and creating a tap.

[harelba]

his will never happen for a Homebrew managed formula. You are the developer of the code, Homebrew packages it.

Thanks for the reply. I fully understand this distinction, and brew is a very good package management system. However, brew formulas contain build and test code, so effectively it's not only responsible for the packaging, but also for the other parts of the workflow.

I understand the need for that, but it means duplicate logic, which has downsides.

Anyway, I'm obviously planning to align with brew's concepts and guidelines