GDPR Webhooks Authenticity

[Muhammad-Daniyal4]

Issue summary

Hello everyone, I have created an app using Shopify CLI - PHP template. My app got rejected three times because of GDPR webhooks endpoints.

I have verifies the hmac using shopify documentation in web/app/Lib/Handlers/Gdpr directory and set the endpoints for it in partners shopify.

First Attempt Error: App must verify the authenticity of the request from Shopify. Expected HTTP 401 (Unauthorized), but got HTTP 405 from https://my-app-domain/webhooks/shop/redact. Your app's HTTPS webhooks endpoints must validate the HMAC digest of each request, and return an HTTP 401 (Unauthorized) response code when rejecting a request that has an invalid digest.

Again, I have created routes for them in web.php and implement the same hmac verification on controller and set the endpoints for them in partners Shopify but the error says:

Second Attempt Error App must verify the authenticity of the request from Shopify. Expected HTTP 401 (Unauthorized), but got HTTP 419 from https://my-app-domain/shop/redact. Your app's HTTPS webhook endpoints must validate the HMAC digest of each request, and return an HTTP 401 (Unauthorized) response code when rejecting a request that has an invalid digest.

Anyone who can guide me how to verify them and set the endpoints for them.

[rafaelstz]

If your webhook route is like this below, try to check your Route:fallback

Route::post('/api/webhooks', function (Request $request) {
    try {
        $topic = $request->header(HttpHeaders::X_SHOPIFY_TOPIC, '');

        $response = Registry::process($request->header(), $request->getContent());
        if (!$response->isSuccess()) {
            Log::error("Failed to process '$topic' webhook: {$response->getErrorMessage()}");

            return response()->json(['message' => "Failed to process '$topic' webhook"], 500);
        }
    } catch (InvalidWebhookException $e) {
        Log::error("Got invalid webhook request for topic '$topic': {$e->getMessage()}");

        return response()->json(['message' => "Got invalid webhook request for topic '$topic'"], 401);
    } catch (Exception $e) {
        Log::error("Got an exception when handling '$topic' webhook: {$e->getMessage()}");

        return response()->json(['message' => "Got an exception when handling '$topic' webhook"], 500);
    }
});

I recommend to test it sending a non-authorized request to your webhook route before submitting again.

[Muhammad-Daniyal4]

Some how the code is sending 200 response and validating the HMAC. But, it is sending the 200 response as well if HMAC is not verified. Any idea ?

Here is my below logic:

Route::post('/customers/data_request', function () { define('CLIENT_SECRET', env('SHOPIFY_API_SECRET')); function verify_webhook($data, $hmac_header) { $calculated_hmac = base64_encode(hash_hmac('sha256', $data, CLIENT_SECRET, true)); return hash_equals($calculated_hmac, $hmac_header); }

$hmac_header = $_SERVER['HTTP_X_SHOPIFY_HMAC_SHA256'];
info($hmac_header);
$data = file_get_contents('php://input');
$verified = verify_webhook($data, $hmac_header);
error_log('Webhook verified: ' . var_export($verified, true));
if ($verified) {
    info('true');
    # Process webhook payload
    # ...
    http_response_code(200);
} else {
    info('false');
    http_response_code(401);
}

});

[rafaelstz]

I don't recommend creating all the HMAC validation as you're doing; you could make it, but it's already done in the template you're using.

You can restore that part as it comes from the template, then add your logic to data cleanup inside this file: https://github.com/Shopify/shopify-app-template-php/blob/main/web/app/Lib/Handlers/Gdpr/CustomersRedact.php#L20

[Muhammad-Daniyal4]

Thanks for the response. But, when I have submitted the app. It is rejected because it was not verifying the HMAC.

[rafaelstz]

It's weird, so try adding Shopify\Utils class, and check the HMAC using the same simple way the middleware shopify.auth is doing it here: https://github.com/Shopify/shopify-app-template-php/blob/main/web/app/Http/Middleware/EnsureShopifySession.php#L95C33-L95C33

[NomanHameed]

Hello everyone. I'm first time trying to submit my shopify application using shopify-app-template-php. But i have no idea about GDPR Webhooks Authenticity. Anyone who can guide how to verify and set the endpoints for verification from shopify.

I have some question about this...

1. shopify-app-template-php already configure for GDPR Webhooks?
2. GDPR Webhooks routes are already exist in web.php or not?
3. If not already present in web.php then what logic is require for these 3 GDPR webhooks routes.

Thanks

[Muhammad-Daniyal4]

Hello Noman, Just create endpoints in api.php for the GDPR webhooks and defined them in Shopify partners account:

For example: Customer data request endpoint: https:/your.domain.com/api/customers/data_request Customer data erasure endpoint: https:/your.domain.com/api/customers/redact Shop data erasure endpoint: https:/your.domain.com/api/shop/redact

In api.php you have to verify the webhook request. If it is verified send 200 response otherwise send 401 unauthorize response.

Example:

`// Customer Data Erasure Webhook Route::post('/customers/redact', function () { define('CLIENT_SECRET', env('SHOPIFY_API_SECRET')); function verify_webhook($data, $hmac_header) { $calculated_hmac = base64_encode(hash_hmac('sha256', $data, CLIENT_SECRET, true)); return hash_equals($calculated_hmac, $hmac_header); } $hmac_header = $_SERVER['HTTP_X_SHOPIFY_HMAC_SHA256'];

$data = file_get_contents('php://input');
$verified = verify_webhook($data, $hmac_header);
error_log('Webhook verified: ' . var_export($verified, true));
if ($verified) {
    # Process webhook payload
    # ...
    http_response_code(200);
} else {
    response()->json('', 401)->send();
}

});`

Do the same for the other two as well and then you are good to go :+1:

[NomanHameed]

Thanks for response @Muhammad-Daniyal4. Actually shopify-app-template-php already contain handlers

1. app/Http/Lib/Handlers/Gdpr/CustomersDataRequest
2. app/Http/Lib/Handlers/Gdpr/CustomersRedact
3. app/Http/Lib/Handlers/Gdpr/ShopRedact

That's why I'm little bit confused why we need to create logic in new route. Anyone please guide me shopify-app-template-php already configured form GDPR web-hooks completely or we need to create routes in web.php or api/php files.

[Muhammad-Daniyal4]

@NomanHameed, I know the logic is implemented but when I submitted the app I got rejection from Shopify. So, I have created apis for them to handle with the above logic and I got approval. I will suggest you to go with the same logic :100:

You need to create routes in api.php and define them in shopify partners account as I mentioned above.

[rafaelstz]

With the template, you have the URL /api/webhooks by default which works with the GDPR handlers. It doesn't mean you don't need to set them up in your Shopify App panel. It needs to be assigned to the three GDPR endpoints in your Shopify App panel.

[NomanHameed]

Thanks for response @rafaelstz I agreed that template already contain /api/webhooks and by default works with three GDPR handlers also we need to add these three routes in Shopify App panel . My Question is should we still define routes

For example:
Customer data request endpoint: https:/your.domain.com/api/customers/data_request
Customer data erasure endpoint: https:/your.domain.com/api/customers/redact
Shop data erasure endpoint: https:/your.domain.com/api/shop/redact 

and process payload in handlers or anything else we have to do?

[rafaelstz]

You don't need to create the extra routes. I published some public apps without it. You can use the /api/webhooks to detect the type of request (customer data request, customer data erase, and shop data erasure) and take an action based on that type, even if this treatment is not required to publish your app, you need to do it to follow GDPR regulations.

[NomanHameed]

I appreciate your response @rafaelstz I have one last doubt related to this question hope you also guide me with this /api/webhooks now I'm using this webhook route but my question is what endpoints/routes we have to provide in our shopify app setup configuration to execute this successfully.

If you need any details related to this question I'll provide please feel free to ask.

As per @Muhammad-Daniyal4 guide I've created these static routes for my application but now I've removed these routes just as per your guide

Please correct these routes using api/webhooks

Customer data request endpoint: https:/your.domain.com/api/customers/data_request
Customer data erasure endpoint: https:/your.domain.com/api/customers/redact
Shop data erasure endpoint: https:/your.domain.com/api/shop/redact

Thanks Again

[NomanHameed]

@rafaelstz i hope you are doing well.

Kindly guide me about GDPR routes you are using to submit app for approval in Shopify app setup using this template.

Thanks.

[rafaelstz]

Hi @NomanHameed You can use the below:

Customer data request endpoint: https:/your.domain.com/api/webhooks
Customer data erasure endpoint: https:/your.domain.com/api/webhooks
Shop data erasure endpoint: https:/your.domain.com/api/webhooks

[helidonashabani]

Hi @rafaelstz,

as the template actually has, the handlers with returns are void. How will be verified and response back whenever Shopify will call these webhooks? Because in their documentation it said that we need to respond with status 200 if the HMAC is verified.

Looking forward to your help!

[helidonashabani]

Based on my observation: verification of HMAC is handled by Registry::process in web.php. So what I'm thinking is that the only part that we are missing is to return the status 200 when $response->isSuccess() return response()->json(['message' => "Successfully processed '$topic' webhook"], 200);

[rafaelstz]

Hey @helidonashabani it will return 200, if an error occur than it returns 500 here: https://github.com/Shopify/shopify-app-template-php/blob/main/web/routes/web.php#L136 You can add your success message if you want as well.

[DN-PERSONAL-REPO]

Hi @rafaelstz I have a custom app deployed on fly.io currenlty working on adding order creation webhook to it so that if an order is created i could perform some third party integration based on the data of the order. I guess this is the function that verifies HMAC can you please let me know what would be Context::API_SECRET_KEY or where we have to set it up.