amazon-braket / amazon-braket-pennylane-plugin-python

A plugin for allowing Xanadu PennyLane to use Amazon Braket devices
https://amazon-braket-pennylane-plugin-python.readthedocs.io/
Apache License 2.0
43 stars 37 forks source link

Add PL integration test to CI #254

Open astralcai opened 4 months ago

astralcai commented 4 months ago

The current CI only contains the unit tests, but not integration tests with PennyLane. This leads to PRs passing CI but breaking compatibility with PL getting merged.

Here's the integration tests run on the PL plugin test matrix: https://github.com/PennyLaneAI/plugin-test-matrix/blob/master/.github/workflows/braket-latest-latest.yml

math411 commented 4 months ago

Hi @astralcai, taking a look at this test matrix, it installs both the latest PL and latest PL-plugin. I would like to avoid having the build process for PRs being aware of the latest version of dependency. Is the purpose here to have better notifications on the Braket side for incoming breaking changes?

Additionally, it seems we should evaluate the TF and torch versions being used as these are looking to be upgraded:

  TF_VERSION: 2.12.0
  TORCH_VERSION: 2.0.0+cpu

https://github.com/PennyLaneAI/plugin-test-matrix/blob/master/.github/workflows/braket-latest-latest.yml#L19

This is something I can look at cutting an issue for.

astralcai commented 4 months ago

In principle, for the plugins, we want to run CI with the latest version of PennyLane. We only verify that the latest plugin still works with stable PennyLane once a week in the test matrix.

Is the purpose here to have better notifications on the Braket side for incoming breaking changes?

This is exactly the reason. In this release we found out that the latest braket plugin did not work with the soon-to-be-released PennyLane a week before the release, which is something that should've been caught way earlier.

math411 commented 3 months ago

Sorry about the delay, I am getting back to this. I am getting out https://github.com/amazon-braket/amazon-braket-pennylane-plugin-python/pull/264. This will fail until the obs are fixed up.