search

glossier / solidus_retail

Solidus Extension to Support Retail Operations
BSD 3-Clause "New" or "Revised" License
8 stars 4 forks source link

Reliance on Existing State in Shopify #5

Closed DanielWright closed 8 years ago

DanielWright commented 8 years ago

The test suite relies on the Shopify shop being in a specific state. This is exceedingly brittle, even as a "private" integration. If the state of the shop changes, or we use a different shop, the tests break. Some ideas:

  1. Use WebMock or Fakeweb to generate the desired Shopify API responses. The upside is we have a great level of control over the API responses, they will be very fast, and they eliminate any reliance on Shopify's state. The downside is the tests will not protect the extension from changes to the API behaviour.
  2. Use the Shopify API to create any objects needed for acceptance testing. The upside is we exercise the full breadth of the extension, we eliminate reliance on existing state in Shopify, and we're protected from changes to the API behaviour. We can make this solution performant by recording the interactions with VCR. The downside is considerable overhead.

Thoughts?

bryanmtl commented 8 years ago

I would vote for option #2 at this point.

thisiscab commented 8 years ago

@DanielWright

2 was built up from the beginning. The spec suites creates the object in Shopify via the API in order for the test to run and then it destroy those objects to make sure that they are not any test left-overs. Did you experience otherwise?

Changing the Shopify shop or if the "state" of the shop changes all the tests should still be green.

DanielWright commented 8 years ago

Hm. When I ran the suite without the "correct" shop credentials, my spec suite failed.

thisiscab commented 8 years ago

I have removed VCR due to multiple issues I was hitting, those issues are detailed in their respective commit messages, I needed some help in order to figure out the best way to solve those issues.

But yes, if no cassettes were saved, we are required to have a valid shop in order to correctly create the items on Shopify else I am unsure how we'll be able to retrieve them (if we put VCR aside).

Once the VCR problems are resolved then we'll be good to go.

DanielWright commented 8 years ago

The issue is this line, which cites a specific line-item ID in the Shopify store. If I use any other store, that line-item won't exist.

thisiscab commented 8 years ago

I'll have it double-checked then! :)

thisiscab commented 8 years ago

@DanielWright Actually the line-item ID specified is not relevant, it will correctly assign the variant to the product without creating it. It acts like "stub" if you want.

It doesn't look for that line-item anywhere with how the Shopify API is made. I am unsure what the problem you were hitting was, but I can't seem to reproduce it.

thisiscab commented 8 years ago

This issue can now be closed.