Trying to create a provider for Shopify. All seems to go well (?) until making a request. Then Shopify says api_key is missing

[adrianocr]

Please bear with me on this long-winded essay I'm about to write because I'm not sure I know how to create this issue without it...

So Shopify uses the Authorization Code Grant flow. Their entire oauth documentation is located here: https://help.shopify.com/en/api/getting-started/authentication/oauth. Basically:

1. The merchant makes a request to install the app.
2. The app redirects to Shopify (https://{shop_name}.myshopify.com) to load the OAuth grant screen and requests the required scopes.
3. Shopify displays a prompt to receive authorization and prompts the merchant to login if required.
4. The merchant consents to the scopes and is redirected to the redirect_uri.
5. The app makes an access token request to Shopify (https://{shop_name}.myshopify.com/admin/oauth/access_token) including the client_id, client_secret, and code.
6. Shopify returns the access token and requested scopes.

So I created a basic app.py with the following contents based on https://github.com/singingwolfboy/flask-dance-github/blob/master/github.py:

from flask import Flask, jsonify, request, redirect, url_for
from contrib.shopify import make_shopify_blueprint, shopify
import uuid

app = Flask(__name__)
app.config.from_pyfile('config.cfg')
app.secret_key = '1pretend2this3is4a5secret6key'
shop_name = app.config['SHOP_NAME']
nonce = uuid.uuid4().hex

shopify_bp = make_shopify_blueprint(shop_name=shop_name, nonce=nonce, grant_options='per-user')
app.register_blueprint(shopify_bp, url_prefix="/login")

@app.route('/')
def index():
    if not shopify.authorized:
        return redirect(url_for("shopify.login"))
    resp = shopify.get("/admin/api/2019-10/products.json")
    # assert resp.ok
    return jsonify(resp)

if __name__ == "__main__":
    app.run(ssl_context=('localhost.pem', 'localhost-key.pem'), debug=True, host="localhost")

Then I duplicated flask_dance/contrib/github.py into myproject_base_dir/contrib/shopify.py:

from flask_dance.consumer import OAuth2ConsumerBlueprint
from functools import partial
from flask.globals import LocalProxy, _lookup_app_object

try:
    from flask import _app_ctx_stack as stack
except ImportError:
    from flask import _request_ctx_stack as stack

__maintainer__ = "adrianocr <me@MYDOMAIN.com>"

def make_shopify_blueprint(
        client_id=None,
        client_secret=None,
        scope=None,
        redirect_url=None,
        redirect_to=None,
        login_url=None,
        authorized_url=None,
        session_class=None,
        storage=None,
        shop_name=None,
        nonce=None,
        grant_options=None
):

    shopify_bp = OAuth2ConsumerBlueprint(
        "shopify",
        __name__,
        client_id=client_id,
        client_secret=client_secret,
        scope=scope,
        base_url=f"https://{shop_name}.myshopify.com",
        authorization_url=f"https://{shop_name}.myshopify.com/admin/oauth/authorize",
        token_url=f"https://{shop_name}.myshopify.com/admin/oauth/access_token",
        redirect_url=redirect_url,
        redirect_to=redirect_to,
        login_url=login_url,
        authorized_url=authorized_url,
        session_class=session_class,
        storage=storage,
    )

    shopify_bp.from_config["client_id"] = "SHOPIFY_OAUTH_CLIENT_ID"
    shopify_bp.from_config["client_secret"] = "SHOPIFY_OAUTH_CLIENT_SECRET"
    shopify_bp.from_config["scope"] = "write_products,read_products,read_script_tags,write_script_tags"
    shopify_bp.from_config["grant_options"] = "per-user"

    @shopify_bp.before_app_request
    def set_applocal_session():
        ctx = stack.top
        ctx.shopify_oauth = shopify_bp.session

    return shopify_bp

shopify = LocalProxy(partial(_lookup_app_object, "shopify_oauth"))

In Shopify the app is set up like this:

When I fire this up and go to to https://apps.MYDOMAIN.com shopify redirects to:

https://{store_name}.myshopify.com/admin/oauth/request_grant?client_id=49cbab21a048fea4f7a8ca9d38891e61&redirect_uri=https%3A%2F%2Fapps.MYDOMAIN.com%2Flogin%2Fshopify%2Fauthorized&scope=write_products%2Cread_products%2Cread_script_tags%2Cwrite_script_tags&state=fNUFy4IHD7rzRASaijGpfJymq8OFqq

And if I accept the installation at the url above, I'm then redirected to:

https://apps.MYDOMAIN.com/login/shopify/authorized?code=b3c2e7465335efd3c71339066e6fbca5&hmac=837dead7f446d07bc1f6c234c77c2e23c3d6fdb945562181c823513c9cf66468&shop={shop_name}.myshopify.com&state=fNUFy4IHD7rzRASaijGpfJymq8OFqq&timestamp=1579452037

and that page displays (via flask's debug screen): oauthlib.oauth2.rfc6749.errors.InvalidClientIdError: (invalid_request) Could not find Shopify API application with api_key

Here is the full traceback from flask debug:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 2463, in __call__
    return self.wsgi_app(environ, start_response)
  File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 2449, in wsgi_app
    response = self.handle_exception(e)
  File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1866, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/local/lib/python3.7/site-packages/flask/_compat.py", line 39, in reraise
    raise value
  File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 2446, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1951, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1820, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/local/lib/python3.7/site-packages/flask/_compat.py", line 39, in reraise
    raise value
  File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1949, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1935, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/usr/local/lib/python3.7/site-packages/flask_dance/consumer/oauth2.py", line 260, in authorized
    **self.token_url_params
  File "/usr/local/lib/python3.7/site-packages/requests_oauthlib/oauth2_session.py", line 360, in fetch_token
    self._client.parse_request_body_response(r.text, scope=self.scope)
  File "/usr/local/lib/python3.7/site-packages/oauthlib/oauth2/rfc6749/clients/base.py", line 421, in parse_request_body_response
    self.token = parse_token_response(body, scope=scope)
  File "/usr/local/lib/python3.7/site-packages/oauthlib/oauth2/rfc6749/parameters.py", line 431, in parse_token_response
    validate_token_parameters(params)
  File "/usr/local/lib/python3.7/site-packages/oauthlib/oauth2/rfc6749/parameters.py", line 438, in validate_token_parameters
    raise_from_error(params.get('error'), params)
  File "/usr/local/lib/python3.7/site-packages/oauthlib/oauth2/rfc6749/errors.py", line 405, in raise_from_error
    raise cls(**kwargs)
oauthlib.oauth2.rfc6749.errors.InvalidClientIdError: (invalid_request) Could not find Shopify API application with api_key

It seems to me the issue is that flask-dance isn't successfully negotiating the access_token exchange which shopify which is described at https://help.shopify.com/en/api/getting-started/authentication/oauth#step-3-confirm-installation. Additionally to make authenticated requests the request needs to include a X-Shopify-Access-Token: {access_token} header which I'm not sure flask-dance does, correct?

I've been banging my head against this for 3-4 days now. I've read the full flask-dance documentation (top to bottom) and the requests documentation (top to bottom) and I can't figure it out. I'm afraid I'm just not experienced enough to understand what's happening 😞

_P.s. I understand the above URL examples I included have clientids and secrets in them but it doesn't matter, I've already revoked the api credentials in the app.

[daenney]

Flask-Dance wouldn't include an X-Shopify-Access-Token header no, it has not way of knowing you have to do so. This is a bit unfortunate on Shopify's side, the accepted header for this is Authorization, usually of the from Authorization: Bearer <token> which is what it'll do.

It looks like in order to fix this you'll have to submit a compliance fix to https://github.com/requests/requests-oauthlib/tree/master/requests_oauthlib/compliance_fixes to deal with the header.

I'm not quite sure what's up with the other error, but since it errors out on InvalidClientIdError it seems to be that you didn't configure Flask Dance with the client ID from Shopify. Are you sure you copied the API Key into client_id and API Secret into client_secret?

[adrianocr]

@daenney yes, I just verified and then also hardcoded the api key and secret directly into shopify.py instead of pulling it from the config file. Still doesn't work.

I'm guessing the issue is that the proper header isn't in place like Shopify expects so it doesn't find the key and therefore: invalid/missing.

Is there a way to hack / monkey pactch in the key into the header at least for testing purposes? Like a way to tap into the request call and manually include:

headers = {"X-Shopify-Access-Token": access_token}
shopify.get('/products.json', headers=headers)

Or something like that?

Additionally is there a way for me to check if flask-dance has actually successfully negotiated the access_token and access it? Such as shopify_bp.session.access_token or something? If yes I can pull that token and try to manually insert it into the header some way or another.

Edit: additionally I'm not sure if it'd be any help but Shopify has an official Koa middleware for handling oauth that might be helpful in understanding how they handle the ouath process? https://github.com/Shopify/quilt/tree/master/packages/koa-shopify-auth

[daenney]

Assuming it's stored it and didn't wipe it due to the next error, blueprint.token["access_token"] lets you get it, or current_app.blueprints["shopify"].token["access_token"]

[adrianocr]

Nope, that's returning NoneType so I assume it hasn't actually been set/stored.

I guess this if failing earlier in the process than I thought. Any tips on what I should try? I'm genuinely at a loss for where to even start trying to correct this.

[singingwolfboy]

It appears that you're using the shopify_bp.from_config feature incorrectly. As the documentation states, it's used for dynamically loading values from the Flask application config (app.config) into your blueprint. This can be useful if you want to set the client ID and client secret using environment variables, for example.

In your code, you have this:

shopify_bp.from_config["client_id"] = "SHOPIFY_OAUTH_CLIENT_ID"
shopify_bp.from_config["client_secret"] = "SHOPIFY_OAUTH_CLIENT_SECRET"
shopify_bp.from_config["scope"] = "write_products,read_products,read_script_tags,write_script_tags"
shopify_bp.from_config["grant_options"] = "per-user"

The first line says "Look for app.config["SHOPIFY_OAUTH_CLIENT_ID"] and use that as the client_id if it's set", which is reasonable. The third line says "Look for app.config["write_products,read_products,read_script_tags,write_script_tags"], and use that as the scope if it's set", which is probably not what you want.

In your code, you're using app.config.from_pyfile('config.cfg') to populate app.config. That's perfectly fine, but I don't know what's in that config.cfg file. Are you setting these values correctly?

[adrianocr]

@singingwolfboy sorry, my intention there was to just imply (so to speak) that the config.cfg file was fine and not the source of the problem but I should have probably included that. So here is config.cfg:

TESTING = True
DEBUG = True
TEMPLATES_AUTO_RELOAD = True
SHOP_NAME = 'myshopifystorename'
SHOPIFY_OAUTH_CLIENT_ID = '49cbab21a048fea4f7a8ca9d38891e61'
SHOPIFY_OAUTH_CLIENT_SECRET = 'f18e1ed6801f3203053908815c4bf656'
SCOPE = 'write_products,read_products,read_script_tags,write_script_tags'

And you're completely right about the scope entry, but that was just for testing purposes. Initially I had it as shopify_bp.from_config["scope"] = "SCOPE" like you'd expect.

[avianion]

Did you end up fixing this @adrianocr

Full working code would be appreciated.