Payment URL contains HTML-encoded characters when subscription is a renewal order

Describe the bug

We are having a problem with orders not loading in the WooCommerce iOS app for some stores due to orders' payment_url having unexpected HTML-encoded characters: https://github.com/woocommerce/woocommerce-ios/issues/9683

This is very similar to the issue resolved in https://github.com/woocommerce/woocommerce-subscriptions/pull/4522 but persists with WooCommerce Subscriptions 5.1.1. As far as I can tell, the problem is with orders with a payment URL containing the subscription_renewal=true query arg.

I'm seeing this issue on a test store hosted on WordPress.com. For example, the REST API returns this payment_url for an order: https://superlativecentaur.wpcomstaging.com/checkout/order-pay/573/?pay_for_order=true&key=wc_order_9tpoAz6J9axmo&subscription_renewal=true

A non-renewal order has the expected payment URL, for example: https://superlativecentaur.wpcomstaging.com/checkout/order-pay/603/?pay_for_order=true&key=wc_order_oZA37wFLmMOE1

As far as I can tell, this looks like it's coming from the escaping when the payment URL is created for renewal orders here:

https://github.com/Automattic/woocommerce-subscriptions-core/blob/a8c653e9a797e2def251bc2af33a1bbbbb1ca00f/includes/class-wcs-cart-renewal.php#L607

To Reproduce

Ensure you have at least one subscription renewal order on your store.
Make a request for that order from the REST API
Check the payment_url field in the API response.
Notice the & characters are encoded to &.

You can also see how this affects the WooCommerce iOS app by opening the app, logging in to a store with a subscription renewal order, and selecting the Orders tab. You will see an error message about a failure to load the orders on the store.

Product impact

This is currently breaking the Orders tab in the WooCommerce iOS app for some merchants.

Additional context

User reports were gathered here: https://github.com/woocommerce/woocommerce-ios/issues/5512 Followup issue with more detail: https://github.com/woocommerce/woocommerce-ios/issues/9683

Status report from my affected test store:

``` ` ### WordPress Environment ### WordPress address (URL): https://superlativecentaur.wpcomstaging.com Site address (URL): https://superlativecentaur.wpcomstaging.com WC Version: 7.6.1 REST API Version: ✔ 7.6.1 WC Blocks Version: ✔ 9.8.5 Action Scheduler Version: ✔ 3.5.4 Log Directory Writable: ✔ WP Version: 6.2 WP Multisite: – WP Memory Limit: 512 MB WP Debug Mode: ✔ WP Cron: ✔ Language: en_US External object cache: ✔ ### Server Environment ### Server Info: nginx PHP Version: 8.0.28 PHP Post Max Size: 2 GB PHP Time Limit: 1200 PHP Max Input Vars: 6144 cURL Version: 7.86.0 OpenSSL/1.1.1n SUHOSIN Installed: – MySQL Version: 10.4.26-MariaDB-log Max Upload Size: 2 GB Default Timezone is UTC: ✔ fsockopen/cURL: ✔ SoapClient: ✔ DOMDocument: ✔ GZip: ✔ Multibyte String: ✔ Remote Post: ✔ Remote Get: ✔ ### Database ### WC Database Version: 7.6.1 WC Database Prefix: wp_ Total Database Size: 36.83MB Database Data Size: 22.24MB Database Index Size: 14.59MB wp_woocommerce_sessions: Data: 0.44MB + Index: 0.02MB + Engine InnoDB wp_woocommerce_api_keys: Data: 0.02MB + Index: 0.03MB + Engine InnoDB wp_woocommerce_attribute_taxonomies: Data: 0.02MB + Index: 0.02MB + Engine InnoDB wp_woocommerce_downloadable_product_permissions: Data: 0.02MB + Index: 0.06MB + Engine InnoDB wp_woocommerce_order_items: Data: 0.05MB + Index: 0.02MB + Engine InnoDB wp_woocommerce_order_itemmeta: Data: 0.33MB + Index: 0.25MB + Engine InnoDB wp_woocommerce_tax_rates: Data: 0.02MB + Index: 0.06MB + Engine InnoDB wp_woocommerce_tax_rate_locations: Data: 0.02MB + Index: 0.03MB + Engine InnoDB wp_woocommerce_shipping_zones: Data: 0.02MB + Index: 0.00MB + Engine InnoDB wp_woocommerce_shipping_zone_locations: Data: 0.02MB + Index: 0.03MB + Engine InnoDB wp_woocommerce_shipping_zone_methods: Data: 0.02MB + Index: 0.00MB + Engine InnoDB wp_woocommerce_payment_tokens: Data: 0.02MB + Index: 0.02MB + Engine InnoDB wp_woocommerce_payment_tokenmeta: Data: 0.02MB + Index: 0.03MB + Engine InnoDB wp_woocommerce_log: Data: 0.02MB + Index: 0.02MB + Engine InnoDB wp_actionscheduler_actions: Data: 9.02MB + Index: 6.50MB + Engine InnoDB wp_actionscheduler_claims: Data: 0.02MB + Index: 0.02MB + Engine InnoDB wp_actionscheduler_groups: Data: 0.02MB + Index: 0.02MB + Engine InnoDB wp_actionscheduler_logs: Data: 3.02MB + Index: 3.06MB + Engine InnoDB wp_automatewoo_abandoned_carts: Data: 0.02MB + Index: 0.08MB + Engine InnoDB wp_automatewoo_customers: Data: 0.02MB + Index: 0.09MB + Engine InnoDB wp_automatewoo_customer_meta: Data: 0.02MB + Index: 0.03MB + Engine InnoDB wp_automatewoo_events: Data: 0.02MB + Index: 0.05MB + Engine InnoDB wp_automatewoo_guests: Data: 0.02MB + Index: 0.06MB + Engine InnoDB wp_automatewoo_guest_meta: Data: 0.02MB + Index: 0.03MB + Engine InnoDB wp_automatewoo_logs: Data: 0.02MB + Index: 0.05MB + Engine InnoDB wp_automatewoo_log_meta: Data: 0.02MB + Index: 0.03MB + Engine InnoDB wp_automatewoo_queue: Data: 0.02MB + Index: 0.05MB + Engine InnoDB wp_automatewoo_queue_meta: Data: 0.02MB + Index: 0.03MB + Engine InnoDB wp_automatewoo_referrals: Data: 0.02MB + Index: 0.08MB + Engine InnoDB wp_automatewoo_referral_advocate_keys: Data: 0.02MB + Index: 0.02MB + Engine InnoDB wp_automatewoo_referral_invites: Data: 0.02MB + Index: 0.05MB + Engine InnoDB wp_commentmeta: Data: 0.02MB + Index: 0.03MB + Engine InnoDB wp_comments: Data: 0.06MB + Index: 0.09MB + Engine InnoDB wp_gla_attribute_mapping_rules: Data: 0.02MB + Index: 0.00MB + Engine InnoDB wp_gla_budget_recommendations: Data: 0.22MB + Index: 0.14MB + Engine InnoDB wp_gla_merchant_issues: Data: 0.06MB + Index: 0.00MB + Engine InnoDB wp_gla_shipping_rates: Data: 0.02MB + Index: 0.03MB + Engine InnoDB wp_gla_shipping_times: Data: 0.02MB + Index: 0.02MB + Engine InnoDB wp_links: Data: 0.02MB + Index: 0.02MB + Engine InnoDB wp_mailpoet_automations: Data: 0.02MB + Index: 0.00MB + Engine InnoDB wp_mailpoet_automation_runs: Data: 0.02MB + Index: 0.02MB + Engine InnoDB wp_mailpoet_automation_run_logs: Data: 0.02MB + Index: 0.02MB + Engine InnoDB wp_mailpoet_automation_triggers: Data: 0.02MB + Index: 0.00MB + Engine InnoDB wp_mailpoet_automation_versions: Data: 0.02MB + Index: 0.02MB + Engine InnoDB wp_mailpoet_custom_fields: Data: 0.02MB + Index: 0.02MB + Engine InnoDB wp_mailpoet_dynamic_segment_filters: Data: 0.02MB + Index: 0.02MB + Engine InnoDB wp_mailpoet_feature_flags: Data: 0.02MB + Index: 0.02MB + Engine InnoDB wp_mailpoet_forms: Data: 0.02MB + Index: 0.00MB + Engine InnoDB wp_mailpoet_log: Data: 0.02MB + Index: 0.00MB + Engine InnoDB wp_mailpoet_migrations: Data: 0.02MB + Index: 0.02MB + Engine InnoDB wp_mailpoet_newsletters: Data: 0.08MB + Index: 0.03MB + Engine InnoDB wp_mailpoet_newsletter_links: Data: 0.02MB + Index: 0.05MB + Engine InnoDB wp_mailpoet_newsletter_option: Data: 0.02MB + Index: 0.02MB + Engine InnoDB wp_mailpoet_newsletter_option_fields: Data: 0.02MB + Index: 0.02MB + Engine InnoDB wp_mailpoet_newsletter_posts: Data: 0.02MB + Index: 0.02MB + Engine InnoDB wp_mailpoet_newsletter_segment: Data: 0.02MB + Index: 0.02MB + Engine InnoDB wp_mailpoet_newsletter_templates: Data: 2.52MB + Index: 0.00MB + Engine InnoDB wp_mailpoet_scheduled_tasks: Data: 0.02MB + Index: 0.03MB + Engine InnoDB wp_mailpoet_scheduled_task_subscribers: Data: 0.02MB + Index: 0.02MB + Engine InnoDB wp_mailpoet_segments: Data: 0.02MB + Index: 0.03MB + Engine InnoDB wp_mailpoet_sending_queues: Data: 0.06MB + Index: 0.03MB + Engine InnoDB wp_mailpoet_settings: Data: 0.02MB + Index: 0.02MB + Engine InnoDB wp_mailpoet_statistics_bounces: Data: 0.02MB + Index: 0.00MB + Engine InnoDB wp_mailpoet_statistics_clicks: Data: 0.02MB + Index: 0.05MB + Engine InnoDB wp_mailpoet_statistics_forms: Data: 0.02MB + Index: 0.02MB + Engine InnoDB wp_mailpoet_statistics_newsletters: Data: 0.02MB + Index: 0.03MB + Engine InnoDB wp_mailpoet_statistics_opens: Data: 0.02MB + Index: 0.08MB + Engine InnoDB wp_mailpoet_statistics_unsubscribes: Data: 0.02MB + Index: 0.05MB + Engine InnoDB wp_mailpoet_statistics_woocommerce_purchases: Data: 0.02MB + Index: 0.06MB + Engine InnoDB wp_mailpoet_stats_notifications: Data: 0.02MB + Index: 0.03MB + Engine InnoDB wp_mailpoet_subscribers: Data: 0.02MB + Index: 0.13MB + Engine InnoDB wp_mailpoet_subscriber_custom_field: Data: 0.02MB + Index: 0.02MB + Engine InnoDB wp_mailpoet_subscriber_ips: Data: 0.02MB + Index: 0.02MB + Engine InnoDB wp_mailpoet_subscriber_segment: Data: 0.02MB + Index: 0.03MB + Engine InnoDB wp_mailpoet_subscriber_tag: Data: 0.02MB + Index: 0.03MB + Engine InnoDB wp_mailpoet_tags: Data: 0.02MB + Index: 0.02MB + Engine InnoDB wp_mailpoet_user_agents: Data: 0.02MB + Index: 0.02MB + Engine InnoDB wp_mailpoet_user_flags: Data: 0.02MB + Index: 0.02MB + Engine InnoDB wp_options: Data: 2.08MB + Index: 0.08MB + Engine InnoDB wp_postmeta: Data: 1.52MB + Index: 0.73MB + Engine InnoDB wp_posts: Data: 0.39MB + Index: 0.06MB + Engine InnoDB wp_taxjar_record_queue: Data: 0.02MB + Index: 0.02MB + Engine InnoDB wp_termmeta: Data: 0.02MB + Index: 0.03MB + Engine InnoDB wp_terms: Data: 0.02MB + Index: 0.03MB + Engine InnoDB wp_term_relationships: Data: 0.02MB + Index: 0.02MB + Engine InnoDB wp_term_taxonomy: Data: 0.02MB + Index: 0.03MB + Engine InnoDB wp_usermeta: Data: 0.09MB + Index: 0.06MB + Engine InnoDB wp_users: Data: 0.02MB + Index: 0.05MB + Engine InnoDB wp_wc_admin_notes: Data: 0.08MB + Index: 0.00MB + Engine InnoDB wp_wc_admin_note_actions: Data: 0.05MB + Index: 0.02MB + Engine InnoDB wp_wc_category_lookup: Data: 0.02MB + Index: 0.00MB + Engine InnoDB wp_wc_customer_lookup: Data: 0.02MB + Index: 0.03MB + Engine InnoDB wp_wc_download_log: Data: 0.02MB + Index: 0.03MB + Engine InnoDB wp_wc_order_bundle_lookup: Data: 0.02MB + Index: 0.09MB + Engine InnoDB wp_wc_order_composite_lookup: Data: 0.02MB + Index: 0.09MB + Engine InnoDB wp_wc_order_coupon_lookup: Data: 0.02MB + Index: 0.03MB + Engine InnoDB wp_wc_order_product_lookup: Data: 0.05MB + Index: 0.06MB + Engine InnoDB wp_wc_order_stats: Data: 0.02MB + Index: 0.05MB + Engine InnoDB wp_wc_order_tax_lookup: Data: 0.02MB + Index: 0.03MB + Engine InnoDB wp_wc_points_rewards_user_points: Data: 0.02MB + Index: 0.03MB + Engine InnoDB wp_wc_points_rewards_user_points_log: Data: 0.02MB + Index: 0.05MB + Engine InnoDB wp_wc_product_attributes_lookup: Data: 0.02MB + Index: 0.02MB + Engine InnoDB wp_wc_product_download_directories: Data: 0.02MB + Index: 0.02MB + Engine InnoDB wp_wc_product_meta_lookup: Data: 0.02MB + Index: 0.09MB + Engine InnoDB wp_wc_rate_limits: Data: 0.02MB + Index: 0.02MB + Engine InnoDB wp_wc_reserved_stock: Data: 0.02MB + Index: 0.00MB + Engine InnoDB wp_wc_tax_rate_classes: Data: 0.02MB + Index: 0.02MB + Engine InnoDB wp_wc_webhooks: Data: 0.02MB + Index: 0.02MB + Engine InnoDB wp_woocommerce_bis_activity: Data: 0.02MB + Index: 0.05MB + Engine InnoDB wp_woocommerce_bis_notifications: Data: 0.02MB + Index: 0.09MB + Engine InnoDB wp_woocommerce_bis_notificationsmeta: Data: 0.02MB + Index: 0.03MB + Engine InnoDB wp_woocommerce_bundled_itemmeta: Data: 0.06MB + Index: 0.06MB + Engine InnoDB wp_woocommerce_bundled_items: Data: 0.02MB + Index: 0.03MB + Engine InnoDB wp_woocommerce_gc_activity: Data: 0.02MB + Index: 0.06MB + Engine InnoDB wp_woocommerce_gc_cards: Data: 0.02MB + Index: 0.06MB + Engine InnoDB wp_woocommerce_gc_cardsmeta: Data: 0.02MB + Index: 0.03MB + Engine InnoDB wp_woocommerce_prl_deploymentmeta: Data: 0.02MB + Index: 0.03MB + Engine InnoDB wp_woocommerce_prl_deployments: Data: 0.02MB + Index: 0.06MB + Engine InnoDB wp_woocommerce_prl_frequencies: Data: 0.02MB + Index: 0.02MB + Engine InnoDB wp_woocommerce_prl_tracking_conversions: Data: 0.02MB + Index: 0.05MB + Engine InnoDB wp_woocommerce_shipping_table_rates: Data: 0.02MB + Index: 0.00MB + Engine InnoDB wp_woocommerce_subscription_downloads: Data: 0.02MB + Index: 0.00MB + Engine InnoDB ### Post Type Counts ### attachment: 39 aw_workflow: 4 customize_changeset: 1 global_product_addon: 1 jetpack_migration: 2 jp_img_sitemap: 6 jp_sitemap: 6 jp_sitemap_master: 6 mailpoet_page: 1 page: 9 post: 2 prl_engine: 1 product: 54 product_variation: 44 revision: 14 shop_coupon: 1 shop_order: 148 shop_order_refund: 6 shop_subscription: 24 wpcode: 4 wp_global_styles: 2 wp_navigation: 1 ### Security ### Secure connection (HTTPS): ✔ Hide errors from visitors: ✔ ### Active Plugins (30) ### WCS staging: by mb – 1.0 Akismet Anti-Spam: by Automattic – 5.1 AutomateWoo: by WooCommerce – 5.7.3 Facebook for WooCommerce: by Facebook – 3.0.23 WordPress.com Editing Toolkit: by Automattic – 3.64806 Google Listings and Ads: by WooCommerce – 2.4.5 Gutenberg: by Gutenberg Team – 15.7.1 WPCode Lite: by WPCode – 2.0.11 Jetpack: by Automattic – 12.2-a.5 Layout Grid: by Automattic – 1.8.2 Page Optimize: by Automattic – 0.5.3 Pinterest for WooCommerce: by WooCommerce – 1.2.21 TikTok: by TikTok – 1.0.16 WooCommerce Back In Stock Notifications: by WooCommerce – 1.4.3 WooCommerce Brands: by WooCommerce – 1.6.49 WooCommerce Composite Products: by WooCommerce – 8.7.6 WooCommerce Gift Cards: by WooCommerce – 1.15.4 WooCommerce Google Analytics Integration: by WooCommerce – 1.8.1 WooCommerce Min/Max Quantities: by WooCommerce – 4.0.7 WooCommerce Payments: by Automattic – 5.8.1 WooCommerce Product Add-ons: by WooCommerce – 6.2.0 WooCommerce Product Bundles: by WooCommerce – 6.18.4 WooCommerce Product Recommendations: by WooCommerce – 2.3.0 WooCommerce Shipping & Tax: by WooCommerce – 2.2.4 WooCommerce Shipment Tracking: by WooCommerce – 2.3.1 WooCommerce UPS Shipping: by WooCommerce – 3.4.2 WooCommerce USPS Shipping: by WooCommerce – 4.6.3 WooCommerce Subscriptions: by WooCommerce – 5.1.1 WooCommerce Table Rate Shipping: by WooCommerce – 3.1.0 (update to version 3.1.1 is available) WooCommerce: by Automattic – 7.6.1 ### Inactive Plugins (18) ### AutomateWoo - Birthdays Add-on: by WooCommerce – 1.3.17 (update to version 1.3.20 is available) AutomateWoo - Refer A Friend Add-on: by WooCommerce – 2.6.18 Avalara AvaTax: by Avalara – 2.3.0 Classic Editor: by WordPress Contributors – 1.6.3 Crowdsignal Forms: by Automattic – 1.7.0 Crowdsignal Polls & Ratings: by Automattic Inc. – 3.0.10 MailPoet: by MailPoet – 4.15.0 TaxJar - Sales Tax Automation for WooCommerce: by TaxJar – 4.1.5 WooCommerce All Products For Subscriptions: by WooCommerce – 4.0.6 WooCommerce Australia Post Shipping: by WooCommerce – 2.5.1 WooCommerce Canada Post Shipping: by WooCommerce – 2.8.0 WooCommerce Conditional Shipping and Payments: by WooCommerce – 1.14.5 WooCommerce EU VAT Number: by WooCommerce – 2.8.3 WooCommerce FedEx Shipping: by WooCommerce – 3.7.2 WooCommerce Points and Rewards: by WooCommerce – 1.7.29 (update to version 1.7.32 is available) WooCommerce Royal Mail: by WooCommerce – 2.8.0 WooCommerce Subscription Downloads: by WooCommerce – 1.4.0 WooCommerce Subscriptions Gifting: by WooCommerce – 2.7.0 ### Dropin Plugins (2) ### advanced-cache.php: advanced-cache.php object-cache.php: Memcached ### Must Use Plugins (1) ### WP.com Site Helper: by – ### Settings ### API Enabled: – Force SSL: – Currency: USD ($) Currency Position: left Thousand Separator: , Decimal Separator: . Number of Decimals: 2 Taxonomies: Product Types: bundle (bundle) composite (composite) external (external) grouped (grouped) simple (simple) subscription (subscription) variable (variable) variable subscription (variable-subscription) Taxonomies: Product Visibility: exclude-from-catalog (exclude-from-catalog) exclude-from-search (exclude-from-search) featured (featured) outofstock (outofstock) rated-1 (rated-1) rated-2 (rated-2) rated-3 (rated-3) rated-4 (rated-4) rated-5 (rated-5) Connected to WooCommerce.com: ✔ Enforce Approved Product Download Directories: ✔ Order datastore: WC_Order_Data_Store_CPT ### WC Pages ### Shop base: #8 - /shop/ Cart: #9 - /cart/ Checkout: #10 - /checkout/ My account: #11 - /my-account/ Terms and conditions: ❌ Page not set ### Theme ### Name: Tazza Version: 1.0.0 Author URL: https://automattic.com/ Child Theme: ❌ – If you are modifying WooCommerce on a parent theme that you did not build personally we recommend using a child theme. See: How to create a child theme WooCommerce Support: ✔ ### Templates ### Overrides: – ### Subscriptions ### WCS_DEBUG: ✔ No Subscriptions Mode: ✔ Live Subscriptions Live URL: https://superlativecentaur.wpcomstaging.com Subscriptions-core Library Version: 5.7.0 Subscription Statuses: wc-active: 18 wc-expired: 3 wc-on-hold: 3 WooCommerce Account Connected: ✔ Yes Active Product Key: ✔ Yes Report Cache Enabled: ✔ Yes Cache Update Failures: ✔ 0 failure ### Store Setup ### Country / State: United States (US) — California ### Subscriptions by Payment Gateway ### WooCommerce Payments: wc-active: 18 wc-expired: 3 wc-on-hold: 3 ### Payment Gateway Support ### WooCommerce Payments: products refunds multiple_subscriptions subscription_cancellation subscription_payment_method_change_admin subscription_payment_method_change_customer subscription_payment_method_change subscription_reactivation subscription_suspension subscriptions subscription_amount_changes subscription_date_changes tokenization add_payment_method Direct bank transfer: products Cash on delivery: products ### Back In Stock ### Database Version: 1.4.3 Loopback Test: ✔ ### Composite Products ### Database Version: 8.7.6 Loopback Test: ✔ Template Overrides: – ### Gift Cards ### Database Version: 1.15.4 Loopback Test: ✔ Task Queueing Test: ✔ ### Product Bundles ### Database Version: 6.18.4 Loopback Test: ✔ Template Overrides: – ### Admin ### Enabled Features: activity-panels analytics coupons customer-effort-score-tracks import-products-task experimental-fashion-sample-products shipping-smart-defaults shipping-setting-tour homescreen marketing multichannel-marketing mobile-app-banner onboarding onboarding-tasks remote-inbox-notifications remote-free-extensions payment-gateway-suggestions shipping-label-banner subscriptions store-alerts transient-notices woo-mobile-welcome wc-pay-promotion wc-pay-welcome-page Disabled Features: block-editor-feature-enabled minified-js navigation new-product-management-experience product-variation-management settings Daily Cron: ✔ Next scheduled: 2023-05-10 10:24:56 +01:00 Options: ✔ Notes: 79 Onboarding: skipped ### WooCommerce Payments ### Version: 5.8.1 Connected to WPCOM: Yes Blog ID: 214253715 Account ID: acct_1MRY6LFmpgWy4Lrl ### Action Scheduler ### Canceled: 3 Oldest: 2023-04-19 22:11:44 +0100 Newest: 2023-05-09 22:37:33 +0100 Complete: 10,863 Oldest: 2023-04-09 11:10:57 +0100 Newest: 2023-05-10 11:08:40 +0100 Failed: 6,163 Oldest: 2023-02-02 17:22:39 +0000 Newest: 2023-05-10 11:08:40 +0100 Pending: 32 Oldest: 2023-05-10 11:10:15 +0100 Newest: 2024-05-23 22:36:40 +0100 ### Product Recommendations ### Database Version: 2.3.0 Loopback Test: ✔ Task Queueing Test: ✔ Page Cache Test: Cache in use ### Status report information ### Generated at: 2023-05-10 11:10:08 +01:00 ` ```

Automattic / woocommerce-subscriptions-core