Open hugocool opened 4 days ago
looks duplicate in the #5506 family
i wouldnt say its a duplicate, in my case i would argue that one can expect it should be possible for a package to only be installed for a specific sys_platform. And the docs dont say otherwise, nor does logic dictate that when we specify that when the sys_platform != darwin, that package is still causing resolving issues.
i wouldnt say its a duplicate
it still is, though!
Issue Kind
Change in current behaviour
Description:
I have encountered an issue with Poetry's handling of platform-specific dependencies, particularly when using environment markers to resolve conflicts between incompatible packages. The specific scenario involves using
vllm
on Linux andllama-cpp-python
on macOS. The issue arises becausetorch
version2.3.0
is required on Linux, but it is not installable on macOS, wheretorch
version2.2.0
is needed instead. Additionally,vllm
is not installable on macOS and its compatible version withtorch
2.2.0
is incompatible withpydantic
2.0.0
.Background:
pydantic
version^2.0.0
(required on both macOS and Linux).torch
version2.3.0
for Linux and version2.2.0
for macOS (due to lack of wheels for2.3.0
on macOS and issues with lower versions).vllm
version0.5.0.post1
for Linux (optional and incompatible with macOS).Problem:
Despite using environment markers to specify platform-specific dependencies, Poetry fails to resolve dependencies when there are conflicts across platforms. For example, specifying
vllm
withmarkers = "sys_platform == 'linux'"
should ideally exclude it from macOS dependency resolution. However, Poetry still attempts to resolvevllm
for macOS, leading to version conflicts withtorch
and making the entire dependency resolution process fail.Detailed Steps and Attempts:
Initial Configuration:
Attempted Solution with Groups:
Result:
Another Attempt with Markers:
Result:
Expected Behavior:
When using environment markers such as
sys_platform == 'linux'
, Poetry should completely excludevllm
from the resolution process on macOS. This means:vllm
should not influence or cause conflicts in the dependency resolution on macOS.Proposed Solution:
Introduce a mechanism in Poetry to truly exclude dependencies based on platform markers during the resolution process. This could be an enhancement to the existing marker handling to ensure that platform-specific optional dependencies do not interfere with the resolution on incompatible platforms.
Impact
This issue impacts users who manage multi-platform projects with platform-specific dependencies. It is particularly crucial for projects that rely on different packages on macOS and Linux due to compatibility issues. The inability to exclude certain dependencies based on the platform during resolution can lead to unresolved conflicts, hindering development and deployment processes.
Addressing this issue would allow for more precise and flexible dependency management, ensuring that platform-specific constraints are respected. This improvement would enhance the user experience by preventing unnecessary conflicts, enabling smoother workflows, and ensuring that developers can use the appropriate packages for each platform without manual intervention or complex workarounds.
Workarounds
Workarounds
1. Using Alternative Dependency Configurations:
One workaround is to adjust the dependency configurations as follows:
This configuration resolves to a very early version of
vllm
that barely specified any constraints. While this may work, it is not ideal as it relies on a very old version ofvllm
.2. Using a Poetry Plugin:
A better workaround involves creating a Poetry plugin that conditionally excludes a specific package from the dependency resolution process based on the platform. Below is an example of such a plugin:
This plugin modifies the dependency pool during resolution to exclude
vllm
on macOS, preventing any related conflicts. This approach ensures that platform-specific dependencies are respected, improving the overall dependency management experience in multi-platform projects.