Installing PHIVE PHAR with Composer

[filips123]

Currently, there isn't any easy way to use PHIVE with Composer. Although there are some Composer plugins (like phar-io/composer-plugin and MacFJA/composer-phar-bin), they all have some limitations. It seems that in general any Composer plugin will have similar limitations (can't run PHIVE on first composer install because plugin is not yet installed). This makes things like #167 which I requested few years ago harder. Until those issues are fixed (which might be hard and may even require to change how Composer plugins work or integrate it directly into Composer), I have another idea which is probably easier to implement (but doesn't support all features).

Provide a way to install PHIVE PHAR as normal Composer dependency. This could be done simply by just adding package which would only contain built PHIVE PHAR file as bin file and publish it to Packagist. Other packages would then be able to just require that package and have it installed in vendor/bin/phive. This opens few possibilities how can packages now use PHIVE:

• Users don't need to manually install PHIVE themselves as it is already installed using Composer.
• It should also work on first install because it just uses normal dependencies without any plugins.
• Packages can install PHARs seamlessly while instaling other dependencies by adding post-install-cmd and post-update-cmd script hooks to composer.json.
• Packages can also make that PHARs will be installed into vendor/bin, allowing to call them directly from other scripts.

However, this has some problems in comparison with plugin-based approaches:

• Installation isn't as seamless as it could be.
• You still need to have separate PHIVE config file and can't declare dependencies in composer.json. This can be solved if PHIVE gets support to read PHARs from composer.json and maybe even write lock to composer.lock (if that won't cause problems with Composer).
• It seems PHIVE sometimes causes problems if you want to use existing config file to install dependenciers that specified there or if config file doesn't contain all keys. Part of this can be fixed with adding command which just installs PHARs specified in config file without adding new one (like composer install vs composer require). Update: phive update --prefer-offline already seems to do thhis. Other part can be fixed with adding additional lock file (see my comment in #219).
• There still needs to be some way to automatically trust keys without user inetraction. This could be done if repository's config file would also support specifying trusted keys. This is similar to #158, except that it would be for repository's config file and not global config file.

I think that providing PHAR as Composer package won't be hard to do, so it could be done quickly. Supporting installation of PHARs from config file and additional lock file would probably be a bit harder, but still easier that Composer plugin.

[theseer]

Thanks for providing this issue and the laid out ideas. It's really appreciated.

I'm unfortunately still not convinced about all of this. Let me try to explain:

To me, composer is doing a pretty good job at managing runtime dependencies. As in, dependencies my software has to 3rd party code at runtime. It does a terrible job at managing everything else. But that's okay, because it's not the job of a runtime dependency manager to manage other things, imho.

So that's why I/we started developing phive. And while I'm quite happy that more people seem to see phive as a useful thing, it turned out to become bigger than I expected and already supports more use cases and CI runs than I originally planned or hoped for.

As one can probably tell from the neglect on our very own composer-plugin, we don't really see a use case for it. Maybe the approach was wrong, but people asked for a means to run phive commands via composer. That's what the plugin is/was for. I never used it myself, to be honest.

Given that many people also use composer to install non-runtime dependencies and to ease migrating those from composer.json to phive, we have a command to scan for known phars. This may need to get extended but in general it works.

That's as far as a "connection" between composer and phive exists. And imho, should exist.

There are of course possible cases, where this clear separation might fail: For instance a post-install hook of something installed via composer requiring a CLI application that could otherwise seemingly be managed via phive. I would argue though that that doesn't qualify as an independent tool as it would quite likely require a tight match of installed versions of the CLI and it's runtime counterpart. Thus it's not a supported use case (yet). We're considering to widen the supported types phive can handle but that's a different story and that case won't likely be included anyway.

Provide a way to install PHIVE PHAR as normal Composer dependency. This could be done simply by just adding package which would only contain built PHIVE PHAR file as bin file and publish it to Packagist. Other packages would then be able to just require that package and have it installed in vendor/bin/phive.

While we of course could easily do this, I fail to see how this would help solve any of the problems? Maybe I'm missing something important ;)

This opens few possibilities how can packages now use PHIVE:

* Users don't need to manually install PHIVE themselves as it is already installed using Composer.

I don't understand this assessment at all: Somebody has to declare it as a dependency somewhere. For it to be "magically" there, it as to be a dependency of my dependencies because otherwise I would have to add it myself - making it identical to the plugin installation.

If, on the other hand, if it's a dev-dependency of one of my dependencies - or worse a dependency of my dependencies dependency - I do not want it on my system. It's most likely not a runtime-dependency of my code base, so why would I care? That's exactly why phive doesn't recurse through vendor to find additional things to install - let alone the conflict-hell you'd likely to end up in..

* It should also work on first install because it just uses normal dependencies without any plugins.

Is that still true for composer 2.0? As in, the same limitation as with 1.0?

* Packages can install PHARs seamlessly while instaling other dependencies by adding `post-install-cmd` and `post-update-cmd` script hooks to `composer.json`.

As previously stated, I'm not convinced that using two different means of installing things that obviously are very tightly coupled (for good reason in this case) is a good idea. I'd actually - at least currently ;) - would recommend adding these phars as vendor/bin within the very dependency you try to install.

* Packages can also make that PHARs will be installed into `vendor/bin`, allowing to call them directly from other scripts.

Why would the location of the phar be relavant? Or why would any other path make it impossible to be called?

However, this has some problems in comparison with plugin-based approaches:

* Installation isn't as seamless as it could be.

* You still need to have separate PHIVE config file and can't declare dependencies in `composer.json`. This can be solved if PHIVE gets support to read PHARs from `composer.json` and maybe even write lock to `composer.lock` (if that won't cause problems with Composer).

We have no intention to ever mess with composer.lock. The only thing we might implement may be removal of the entries found from composer.json. Again, we do not plan to intercept calls or otherwise integrate phive into composer.

* It seems PHIVE sometimes causes problems if you want to use existing config file to install dependencies that specified there or if config file doesn't contain all keys.

I'm not understanding this sentence, sorry.

Part of this can be fixed with adding command which just installs PHARs specified in config file without adding new one (like composer install vs composer require). Update: phive update --prefer-offline already seems to do thhis. Other part can be fixed with adding additional lock file (see my comment in #219).

If you just say phive install it does exactly that. It installs everything as is found in phive.xml / .phive/phars.xml in their respective installed="x.y" version. If you say update --prefer-offline it will update to whatever latest version is on your system in case it's later than the one currently installed. That's not the same. There is, imho, no issue to be fixed.

* There still needs to be some way to automatically trust keys without user inetraction.

That's what --trust-gpg-keys is for?

This could be done if repository's config file would also support specifying trusted keys. This is similar to #158, except that it would be for repository's config file and not global config file.

Auto-Trusting keys is not so simple as the discussion in #158 already showed. We might provide a switch though that would allow to auto-accept keys mentioned by id in a special config file. Not sure how to best tackle this yet.

The main idea is to avoid lookups on keyservers if we can store the public key in our repositories.xml.

I think that providing PHAR as Composer package won't be hard to do, so it could be done quickly.

Sure.

Supporting installation of PHARs from config file

That already is supported since day one?

and additional lock file would probably be a bit harder, but still easier that Composer plugin.

We don't need nor do I want a separate lock file. I don't see any benefits, only downsides.

[filips123]

I don't understand this assessment at all: Somebody has to declare it as a dependency somewhere. For it to be "magically" there, it as to be a dependency of my dependencies because otherwise I would have to add it myself - making it identical to the plugin installation.

Yes, package creator needs to declare it as a dependency (in dev dependencies), like for plugin. But problem with plugins (at least on Composer 1.0, I don't know for 2.0) is that they don't run on first install. Installing phive as normal dependency and using post-install-cmd hook will make sure it also runs on first install.

However, users/developers who want to develop that package won't have to separately install phive.

For example, package uses PHPUnit for tests which is required as dev dependency using Composer. Developers then need to run:

0. Install Composer.
1. composer install to install all runtime and dev dependencies.
2. ./vendor/bin/phpunit or composer test (if PHPUnit is defined as Composer script) to run tests.

If package uses phive to install PHPUnit, developers will need to run:

0. Install Composer and phive.
1. composer install to install all runtime dependencies and non-PHAR dev dependencies.
2. phive install to install PHARs like PHPUnit.
3. ./tools/phpunit (or ./vendor/bin/phpunit if specified in config file) or composer test (if PHPUnit is defined as Composer script) to run tests.

If package uses phive to install PHPUnit, but with combination of phive PHAR as dev depenency and post-install-cmd script, developers will need to run:

0. Install Composer.
1. composer install to install all runtime dependencies phive PHAR and non-PHAR dev dependencies. Then, post-install-cmd script will run and install all PHARs like PHPUnit.
2. ./tools/phpunit (or ./vendor/bin/phpunit if specified in config file) or composer test (if PHPUnit is defined as Composer script) to run tests.

If users want to install non-dev dependencies, it will work normally and won't install any PHARs, because phive (declared as dev dependency) won't be installed, and post-install-cmd script can have check (using COMPOSER_DEV_MODE env var) to not run when --no-dev is specified.

Is that still true for composer 2.0? As in, the same limitation as with 1.0?

I don't know because both plugins are still for 1.0.

Why would the location of the phar be relavant? Or why would any other path make it impossible to be called?

Composer automatically adds vendor/bin to path while running scripts so you can just run phpunit. It is also possible to run programs from other paths but then you need to run ./tools/phpunit.

I'm not understanding this sentence, sorry. That already is supported since day one?

I had broken config file... It works now.

That's what --trust-gpg-keys is for? Auto-Trusting keys is not so simple as the discussion in #158 already showed. We might provide a switch though that would allow to auto-accept keys mentioned by id in a special config file. Not sure how to best tackle this yet.

Yes, but with --trust-gpg-keys you can only specify trusted keys for all tools at the same time. If you have many tools, you will need to specify all of them and manually track which key is for which tool. If trusted keys would be in config file, you could specify trusted key for each PHAR separatly (if you want) and track which key if for which tool.

Additionally, if one of PHARs doesn't have signature, you can already use --force-accept-unsigned, but that will accept unsigned PHARs for all tools. If you would have accept-unsigned in config file separate by tool, you would be able to just accept specific tool unsigned and still force signature for all others.

This doesn't have big effect on users who already have tools installed (and keys imported), but mainly on CI servers and users who don't have tools and phive installed yet.

[MacFJA]

The Composer plugin I wrote was to solve very simple but annoying problem: some of my dev dependencies were in conflict with my no-dev dependencies. (mainly because, most of my dev tool are in stable state, and don't need to by update every time Symfony decide to update all their components)

As they are only dev dependencies, I have no issue to manages them externally with Phive, but my concerne was to remove the maximum of human actions (it's also why I heavily use Makefile) in my installation procedure. The idea was also to be able to just use my plugin globally so it completely disappear and work seamless (and that also remove most of its limitation) on all my existing project (and others projects too).

---

I didn't look at the new Composer 2.0 Plugin API, maybe the issue that was having is not present anymore. Or maybe the issue I have right now is just me not correctly understanding how plugin works.

---

Your approach won't solve the dependencies issue (that I have) as Composer still calculate dev-dependencies (where my plugin hot-swap them completely with the phar and them remove them from the dependency resolution). Your idea can at most replace dependencies after they are installed.

Except if you just have phive as dev dependency and then have phive packages declare elsewhere (in composer extra section? .phive/phars.xml?), but then what the difference with a simple post-install-cmd that install and run Phive? And then, the phar-io/composer-plugin can remove the install part, so the post-install-cmd only contain @composer phive:run install.

Or I'm missing something?