Elasticsearch 8 package has incompatible composer dependencies with elasticsearch 7

damienwebdev commented 1 year ago

Preconditions and environment

Magento version: 2.4-develop

Looks like the dependencies determined for the meta package contain both magento/module-elasticsearch-7 and magento/module-elasticsearch-8

But those have incompatible base package dependencies

Steps to reproduce

Check the composer.json of the packages in the 2.4-develop branch.

Try to composer install these two packages in the same project.

Expected result

Successful install.

Actual result

Additional information

No response

Release note

No response

Triage and priority

[ ] Severity: S0 - Affects critical data or functionality and leaves users without workaround.
[X] Severity: S1 - Affects critical data or functionality and forces users to employ a workaround.
[ ] Severity: S2 - Affects non-critical data or functionality and forces users to employ a workaround.
[ ] Severity: S3 - Affects non-critical data or functionality and does not force users to employ a workaround.
[ ] Severity: S4 - Affects aesthetics, professional look and feel, “quality” or “usability”.

m2-assistant[bot] commented 1 year ago

Hi @damienwebdev. Thank you for your report. To speed up processing of this issue, make sure that you provided the following information:

Summary of the issue
Information on your environment
Steps to reproduce
Expected and actual results

Make sure that the issue is reproducible on the vanilla Magento instance following Steps to reproduce. To deploy vanilla Magento instance on our environment, Add a comment to the issue:

@magento give me 2.4-develop instance - upcoming 2.4.x release

For more details, review the Magento Contributor Assistant documentation.

Add a comment to assign the issue: @magento I am working on this

To learn more about issue processing workflow, refer to the Code Contributions.

Join Magento Community Engineering Slack and ask your questions in #github channel.

:warning: According to the Magento Contribution requirements, all issues must go through the Community Contributions Triage process. Community Contributions Triage is a public meeting.

:clock10: You can find the schedule on the Magento Community Calendar page.

:telephone_receiver: The triage of issues happens in the queue order. If you want to speed up the delivery of your contribution, join the Community Contributions Triage session to discuss the appropriate ticket.

:pencil2: Feel free to post questions/proposals/feedback related to the Community Contributions Triage process to the corresponding Slack Channel

m2-assistant[bot] commented 1 year ago

Hi @engcom-Bravo. Thank you for working on this issue. In order to make sure that issue has enough information and ready for development, please read and check the following instruction: :point_down:

[ ] 1. Verify that issue has all the required information. (Preconditions, Steps to reproduce, Expected result, Actual result).
Details
If the issue has a valid description, the label Issue: Format is valid will be added to the issue automatically. Please, edit issue description if needed, until label Issue: Format is valid appears.
[ ] 2. Verify that issue has a meaningful description and provides enough information to reproduce the issue. If the report is valid, add Issue: Clear Description label to the issue by yourself.
[ ] 3. Add Component: XXXXX label(s) to the ticket, indicating the components it may be related to.
[ ] 4. Verify that the issue is reproducible on 2.4-develop branch
Details
- Add the comment @magento give me 2.4-develop instance to deploy test instance on Magento infrastructure.
- If the issue is reproducible on 2.4-develop branch, please, add the label Reproduced on 2.4.x.
- If the issue is not reproducible, add your comment that issue is not reproducible and close the issue and stop verification process here!
[ ] 5. Add label Issue: Confirmed once verification is complete.
[ ] 6. Make sure that automatic system confirms that report has been added to the backlog.

hostep commented 1 year ago

Is probably caused by:

Which won't give a problem when cloning from github, but only when installing Magento through composer. Someone within Adobe will probably (hopefully) notice when they prepare the first beta for 2.4.6 ...

engcom-Lima commented 1 year ago

@magento give me 2.4.5-p1 instance

magento-deployment-service[bot] commented 1 year ago

Hi @engcom-Lima. Thank you for your request. I'm working on Magento instance for you.

m2-assistant[bot] commented 1 year ago

Hi @engcom-Dash. Thank you for working on this issue. In order to make sure that issue has enough information and ready for development, please read and check the following instruction: :point_down:

1. Verify that issue has all the required information. (Preconditions, Steps to reproduce, Expected result, Actual result).
  Details
  If the issue has a valid description, the label Issue: Format is valid will be added to the issue automatically. Please, edit issue description if needed, until label Issue: Format is valid appears.
1. Verify that issue has a meaningful description and provides enough information to reproduce the issue. If the report is valid, add Issue: Clear Description label to the issue by yourself.
1. Add Component: XXXXX label(s) to the ticket, indicating the components it may be related to.
1. Verify that the issue is reproducible on 2.4-develop branch
  Details
  - Add the comment @magento give me 2.4-develop instance to deploy test instance on Magento infrastructure.
  - If the issue is reproducible on 2.4-develop branch, please, add the label Reproduced on 2.4.x.
  - If the issue is not reproducible, add your comment that issue is not reproducible and close the issue and stop verification process here!

Join Magento Community Engineering Slack and ask your questions in #github channel.

magento-deployment-service[bot] commented 1 year ago

Hi @engcom-Lima, here is your Magento Instance: https://bd4ac8ba4c661e5d529e59fc147e44b7.instances.magento-community.engineering Admin access: https://bd4ac8ba4c661e5d529e59fc147e44b7.instances.magento-community.engineering/admin_9d1d Login: c613c987 Password: 43d96cfb6f36

engcom-Lima commented 1 year ago

@magento give me 2.4.5-p1 instance with edition b2b

magento-deployment-service[bot] commented 1 year ago

Hi @engcom-Lima. Thank you for your request. I'm working on Magento instance for you.

magento-deployment-service[bot] commented 1 year ago

Hi @engcom-Lima, unfortunately there is no ability to deploy Magento instance at the moment. Please try again later.

engcom-Lima commented 1 year ago

@magento give me 2.4.5-p1 instance with edition b2b

magento-deployment-service[bot] commented 1 year ago

Hi @engcom-Lima. Thank you for your request. I'm working on Magento instance for you.

magento-deployment-service[bot] commented 1 year ago

Hi @engcom-Lima, unfortunately there is no ability to deploy Magento instance at the moment. Please try again later.

engcom-Dash commented 1 year ago

Hi @damienwebdev ,

We are trying to reproduce the issue in 2.4-develop branch...But we are not able to reproduce the issue. We installed two packages of Elasticserach successfully.Then we tried composer install command,Its installed successfully without any issue. Magento Version 2.4-develop PHP 8.1 Screenshots:

Let us know if we missing anything!!!

Thanks!!!

damienwebdev commented 1 year ago

@engcom-Dash https://github.com/magento/magento2/blob/2.4-develop/app/code/Magento/Elasticsearch7/composer.json and https://github.com/magento/magento2/blob/2.4-develop/app/code/Magento/Elasticsearch8/composer.json are the files at issue.

https://github.com/mage-os/generate-mirror-repo-js/actions/runs/3873321241/jobs/6603290130

You can only install one of them, how are we supposed to choose which?

damienwebdev commented 1 year ago

@maghamed for visibility, otherwise when this goes to release, there will be issues.

engcom-Dash commented 1 year ago

Hi @damienwebdev ,

We are trying to reproduce the issue in 2.4-develop branch and its reproducible.

Magento version 2.4-develop PHP 8.1

Steps: 1.install magento instance 2.Install Elastic serach 7 and 8 3.Run the composer install command.

Screenshots:

We are getting Actual result as per the description...Hence we are confirming the issue.

Let us know if we are missing anything!!!

Thanks!!!

damienwebdev commented 1 year ago

@engcom-Dash perfect. However this ends up decided, since only one version of the ES package is installable, this will create a problem for merchants. There should likely be an announcement about this backwards incompatibility because otherwise many will be caught at upgrade time for v2.4.6+ without knowing about this change.

hostep commented 1 year ago

The first bèta of Magento 2.4.6 is supposed to get released this evening. So I'll try to see tomorrow how it works and if they did something funky maybe in the packaging stage to get this resolved or not.

hostep commented 1 year ago

@damienwebdev: So, using Magento 2.4.6-beta1, I see the following:

The module magento/module-elasticsearch contains the constraint "elasticsearch/elasticsearch": "~7.17.0"
The module magento/module-elasticsearch-7 contains the constraint "elasticsearch/elasticsearch": "^7.17"
The module magento/module-elasticsearch-8 does not contain any constraint for elasticsearch/elasticsearch
The metapackage magento/product-community-edition contains the constraint "elasticsearch/elasticsearch": "^7.17||^8.5"

So it sounds like some shenanigans is happening in the packaging phase where they removed the constraint from magento/module-elasticsearch-8 for now, until they figure out how to make this work.

Maybe @tpogrebniak can elaborate on what's going on and what the plans are for the final release?

github-jira-sync-bot commented 1 year ago

:white_check_mark: Jira issue https://jira.corp.adobe.com/browse/AC-7697 is successfully created for this GitHub issue.

m2-assistant[bot] commented 1 year ago

:white_check_mark: Confirmed by @engcom-Hotel. Thank you for verifying the issue.
Issue Available: @engcom-Hotel, You will be automatically unassigned. Contributors/Maintainers can claim this issue to continue. To reclaim and continue work, reassign the ticket to yourself.

hostep commented 1 year ago

Small update for the people interested, this is the current status in Magento 2.4.6-beta8:

The module magento/module-elasticsearch contains the constraint "elasticsearch/elasticsearch": "~7.17.0 || ~8.5.0"
The module magento/module-elasticsearch-7 contains the constraint "elasticsearch/elasticsearch": "^7.17"
There is no longer a module magento/module-elasticsearch-8 getting installed by default
The metapackage magento/product-community-edition contains the constraint "elasticsearch/elasticsearch": "~7.17.0 || ~8.5.0"
The module magento/module-elasticsearch-8 is still available to be installed and contains the constraint "elasticsearch/elasticsearch": "~8.5.0" and (this is important) also a replace section: "magento/module-elasticsearch-7": "*"

So, after some quick testing, you can switch from ES7 to ES8 by:

$ composer require --no-update magento/module-elasticsearch-8 100.4.0-beta8
./composer.json has been updated

$ composer update magento/module-elasticsearch-7 magento/module-elasticsearch-8 elasticsearch/elasticsearch
Loading composer repositories with package information
Updating dependencies
Lock file operations: 5 installs, 1 update, 1 removal
  - Removing magento/module-elasticsearch-7 (100.4.6-beta8)
  - Locking elastic/transport (v8.6.0)
  - Upgrading elasticsearch/elasticsearch (v7.17.1 => v8.5.3)
  - Locking magento/module-elasticsearch-8 (100.4.0-beta8)
  - Locking php-http/discovery (1.15.2)
  - Locking php-http/httplug (2.3.0)
  - Locking php-http/promise (1.1.0)
Writing lock file
...

This seems like an ok solution to me.

These changes haven't made it to github yet as far as I can see, so the Mage-OS people will probably still run into issues when trying to generate nightly builds.

VyacheslavCh commented 1 year ago

hello @damienwebdev @hostep have you found solution to this? We have installed manento 2.4.6 and elastcisearch 8.4 on server; the composer file however requires elasticsearch 7.17.1. I have tried your solution above for composer, but here is the message I receive: Root composer.json requires magento/module-elasticsearch-8, it could not be found in any version, there may be a typo in the package name.

Please let me know. Thank you!

magenx commented 1 year ago

you wish... https://experienceleague.adobe.com/docs/commerce-operations/release/packages/magento-open-source.html?lang=en

damienwebdev commented 1 year ago

The docs for v2.4.6 are wrong. You can only use 7.17. 8.4 won't be available until v2.4.7.

hostep commented 1 year ago

@VyacheslavCh: does the following link help: https://experienceleague.adobe.com/docs/commerce-operations/upgrade-guide/prepare/prerequisites.html#upgrade-elasticsearch ? (there is a typo however, it should be composer require ... instead of composer update ..., and apparently it still installs a beta version, ... talking to @sidolov about it on Slack as we speak, it looks like Adobe forgot to release a stable version of this package ...)

@damienwebdev: as far as I understood it, it should work with ES 8, it just requires some manual work to replace the ES 7 composer package with the ES 8 one.

VyacheslavCh commented 1 year ago

@hostep Im getting this error message, is there any specific command I can use? also is there a stable package? thank you!!

composer require magento/module-elasticsearch-8 --update-with-all-dependencies Info from https://repo.packagist.org: #StandWithUkraine

[InvalidArgumentException]
Could not find a matching version of package magento/module-elasticsearch-8. Check the package spelling, your version constraint a
nd that the package is available in a stability which matches your minimum-stability (stable).

hostep commented 1 year ago

@VyacheslavCh: I just got confirmation on Slack (and tested it myself) that it now works:

$ composer require magento/module-elasticsearch-8 --update-with-all-dependencies
Info from https://repo.packagist.org/: #StandWithUkraine
./composer.json has been updated
Running composer update magento/module-elasticsearch-8 --with-all-dependencies
Loading composer repositories with package information
Updating dependencies
Lock file operations: 4 installs, 1 update, 1 removal
  - Removing magento/module-elasticsearch-7 (100.4.6)
  - Locking elastic/transport (v8.6.0)
  - Upgrading elasticsearch/elasticsearch (v7.17.1 => v8.5.3)
  - Locking magento/module-elasticsearch-8 (100.4.0)
  - Locking php-http/httplug (2.3.0)
  - Locking php-http/promise (1.1.0)
...

VyacheslavCh commented 1 year ago

@hostep thank you so much!

Chris25602 commented 1 year ago

I am also having this issue. My install path was

git clone https://github.com/magento/magento2.git
composer install
php magento setup:install -v --db-name mage --search-engine elasticsearch8 --elasticsearch-host localhost --elasticsearch-port 9200 --elasticsearch-enable-auth 1 --elasticsearch-username elastic --elasticsearch-password ###### --elasticsearch-index-prefix elastic.magento2

I realize I need the magento/module-elasticsearch-8 I just have no idea how to get it.

hostep commented 1 year ago

@Chris25602: I already mentioned the instructions a couple of comments earlier, you can find them here.

Chris25602 commented 1 year ago

thanks I just stepped down to es7 as it seems like the package has been removed.

mrrobotisback commented 8 months ago

Only with me or there is a problem with Magento\Elasticsearch8\SearchAdapter\Mapper, that use mapper from Magento\Elasticsearch\Elasticsearch5\SearchAdapter\Mapper as Elasticsearch5Mapper; ???

engcom-Hotel commented 1 month ago

@magento give me 2.4.7-p2 instance

magento / magento2