ekwoka
commented
8 months ago Here it looks like theres timeouts that are causing it to be slow, that you're losing connection. Is your connection ever unstable?
Closed dzschille closed 7 months ago
ekwoka
commented
8 months ago Here it looks like theres timeouts that are causing it to be slow, that you're losing connection. Is your connection ever unstable?
dzschille
commented
8 months ago Is your connection ever unstable?
Not that i am aware of. I just disabled my LAN connection to the cable modem and used wifi to connect to my mobile and downloaded the theme again in an empty directory:
┃ DEBUG get templates/page.json
┃ DEBUG GET https://shpfy-cli-test.myshopify.com/admin/api/unstable/themes/133896143043/assets.json?asset%5Bkey%5D=templates%2Fpage.json with request_id: 45e0be65-235a-4db9-ab71-dc8533f51816 98%
┃ DEBUG get sections/header-group.json
┃ DEBUG GET https://shpfy-cli-test.myshopify.com/admin/api/unstable/themes/133896143043/assets.json?asset%5Bkey%5D=sections%2Fheader-group.json with request_id: 4f65c78f-b859-4f28-88a8-95557e8dce74
┃ DEBUG get sections/footer-group.json
┃ DEBUG GET https://shpfy-cli-test.myshopify.com/admin/api/unstable/themes/133896143043/assets.json?asset%5Bkey%5D=sections%2Ffooter-group.json with request_id: df963d45-9f34-43f7-88c7-90c60f60c583
┃ DEBUG timeout in GET https://shpfy-cli-test.myshopify.com/admin/api/unstable/themes/133896143043/assets.json?asset%5Bkey%5D=snippets%2Flanguage-localization.liquid 99%
┃ DEBUG GET https://shpfy-cli-test.myshopify.com/admin/api/unstable/themes/133896143043/assets.json?asset%5Bkey%5D=snippets%2Flanguage-localization.liquid with request_id: 50a909f1-51c3-4bb1-bac5-0bd9b7393913
┃ 100%
┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ (351.09s) ━━
✓ Theme pulled successfully.
2024-02-16T09:46:45.049Z:
Running system process:
· Command: /home/linuxbrew/.linuxbrew/opt/ruby/bin/ruby -v
· Working directory: /home/david/repos/shopify-cli-v3-test
2024-02-16T09:46:45.618Z: Request to https://monorail-edge.shopifysvc.com/v1/produce completed in 554 ms
With response headers:
- x-request-id: 763a23bd-1b16-4f77-a36f-27bef05ea057
2024-02-16T09:46:45.619Z: Analytics event sent: {
"command": "theme pull",
"time_start": 1708076446339,
"time_end": 1708076805040,
"total_time": 358701,
"success": true,
"cli_version": "3.56.0",
"ruby_version": "3.3.0",
"node_version": "21.6.1",
"is_employee": false,
"uname": "linux amd64",
"env_ci": false,
"env_plugin_installed_any_custom": false,
"env_plugin_installed_shopify": "[\"@shopify/cli\",\"@shopify/plugin-did-you-mean\",\"@shopify/theme\"]",
"env_shell": "zsh",
"env_device_id": "1019fba5751f42ca3e0240a5b3b2c92a06c0fc5c",
"env_cloud": "localhost",
"env_package_manager": "unknown",
"cmd_all_timing_network_ms": 0,
"cmd_all_timing_prompts_ms": 0,
"cmd_all_launcher": "unknown",
"cmd_all_topic": "theme",
"cmd_all_plugin": "@shopify/theme",
"cmd_all_force": false,
"cmd_all_timing_active_ms": 358701,
"cmd_all_exit": "ok",
"args": "--store=shpfy-cli-test.myshopify.com --verbose",
"env_plugin_installed_all": "[\"@shopify/cli\",\"@shopify/plugin-did-you-mean\",\"@shopify/theme\"]",
"metadata": "{\"extraPublic\":{},\"extraSensitive\":{}}"
}
2024-02-16T09:46:45.619Z: Completed command theme pull
~/repos/shopify-cli-v3-test took 6m 1s at 10:46:46 It took just half of the time but still the same timeout messages. Than i switched back to cable and tried it again, needs double the time:
┃ DEBUG get sections/footer-group.json
┃ DEBUG GET https://shpfy-cli-test.myshopify.com/admin/api/unstable/themes/133896143043/assets.json?asset%5Bkey%5D=sections%2Ffooter-group.json with request_id: 061ead66-362b-42fb-8d41-90f9144aecfa
┃ DEBUG timeout in GET https://shpfy-cli-test.myshopify.com/admin/api/unstable/themes/133896143043/assets.json?asset%5Bkey%5D=templates%2Flist-collections.json 99%
┃ DEBUG GET https://shpfy-cli-test.myshopify.com/admin/api/unstable/themes/133896143043/assets.json?asset%5Bkey%5D=templates%2Flist-collections.json with request_id: 0f023877-50e2-450d-b9ae-d880f35bdf54
┃ 100%
┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ (736.46s) ━━
✓ Theme pulled successfully.
2024-02-16T10:00:57.043Z:
Running system process:
· Command: /home/linuxbrew/.linuxbrew/opt/ruby/bin/ruby -v
· Working directory: /home/david/repos/shopify-cli-v3-test
2024-02-16T10:00:57.255Z: Request to https://monorail-edge.shopifysvc.com/v1/produce completed in 196 ms
With response headers:
- x-request-id: 567b6c97-9f76-4c45-af16-b671ff485edc
2024-02-16T10:00:57.256Z: Analytics event sent: {
"command": "theme pull",
"time_start": 1708076917868,
"time_end": 1708077657033,
"total_time": 739165,
"success": true,
"cli_version": "3.56.0",
"ruby_version": "3.3.0",
"node_version": "21.6.1",
"is_employee": false,
"uname": "linux amd64",
"env_ci": false,
"env_plugin_installed_any_custom": false,
"env_plugin_installed_shopify": "[\"@shopify/cli\",\"@shopify/plugin-did-you-mean\",\"@shopify/theme\"]",
"env_shell": "zsh",
"env_device_id": "1019fba5751f42ca3e0240a5b3b2c92a06c0fc5c",
"env_cloud": "localhost",
"env_package_manager": "unknown",
"cmd_all_timing_network_ms": 0,
"cmd_all_timing_prompts_ms": 0,
"cmd_all_launcher": "unknown",
"cmd_all_topic": "theme",
"cmd_all_plugin": "@shopify/theme",
"cmd_all_force": false,
"cmd_all_timing_active_ms": 739165,
"cmd_all_exit": "ok",
"args": "--store=shpfy-cli-test.myshopify.com --verbose",
"env_plugin_installed_all": "[\"@shopify/cli\",\"@shopify/plugin-did-you-mean\",\"@shopify/theme\"]",
"metadata": "{\"extraPublic\":{},\"extraSensitive\":{}}"
}
2024-02-16T10:00:57.256Z: Completed command theme pull
~/repos/shopify-cli-v3-test took 12m 21s at 11:00:58 However: both are to slow. Here i uploaded the theme to a small vServer:
> scp -r ./* ssh@myserver.com:/www/htdocs/test/
localization-form.js 100% 7694 197.7KB/s 00:00
section-main-page.css 100% 305 8.1KB/s 00:00
show-more.js 100% 1208 32.3KB/s 00:00
component-discounts.css 100% 562 14.6KB/s 00:00
component-show-more.css 100% 162 4.3KB/s 00:00
template-giftcard.css 100% 6820 150.2KB/s 00:00
...
...
password.json 100% 1108 29.2KB/s 00:00
page.contact.json 100% 419 11.2KB/s 00:00
~/repos/shopify-cli-v3-test took 49s at 11:04:13Less than a minute. Download slightly faster:
> scp -r ssh@myserver.com:/www/htdocs/test/ .
...
took 46s at 11:12:16If i look for the way the the packets take through the internet to the shop i can see that Cloudflare is dropping packets:
> mtr shpfy-cli-test.myshopify.com --report -c 10
Start: 2024-02-16T11:22:21+0100
HOST: david-pc Loss% Snt Last Avg Best Wrst StDev
1.|-- fritz.box 0.0% 10 0.6 0.9 0.6 1.1 0.2
2.|-- ipxyz.static.kabel-d 0.0% 10 9.3 12.5 9.3 18.4 3.1
3.|-- ipxyz.static.kabel-d 0.0% 10 12.2 11.9 9.2 17.0 2.6
4.|-- 145.254.3.196 0.0% 10 29.4 14.1 10.2 29.4 5.6
5.|-- 145.254.2.179 0.0% 10 17.0 22.2 16.3 37.7 7.7
6.|-- 162.158.84.8 20.0% 10 20.2 29.4 19.8 80.8 21.1
7.|-- 162.158.92.2 0.0% 10 31.0 26.3 16.1 72.4 17.0
8.|-- shops.myshopify.com 0.0% 10 18.4 17.9 15.0 23.3 2.4
~/repos/shopify-cli-v3-test took 15s at 11:22:36
> mtr shopify.com --report -c 10
Start: 2024-02-16T11:23:21+0100
HOST: david-pc Loss% Snt Last Avg Best Wrst StDev
1.|-- fritz.box 0.0% 10 0.8 0.8 0.6 1.4 0.2
2.|-- ipxyz.static.kabel-d 0.0% 10 11.1 11.5 10.1 14.3 1.3
3.|-- ipxyz.static.kabel-d 0.0% 10 13.5 11.9 9.0 16.4 2.5
4.|-- 145.254.3.196 0.0% 10 12.7 12.0 9.4 19.0 2.9
5.|-- 145.254.2.179 0.0% 10 17.6 17.1 15.9 19.3 1.0
6.|-- 162.158.84.8 30.0% 10 21.2 22.7 16.8 36.8 6.7
7.|-- 172.71.244.3 0.0% 10 20.2 23.0 16.5 33.2 6.4
8.|-- checkout.shopify.com 0.0% 10 19.0 17.4 15.6 21.9 2.0
~/repos/shopify-cli-v3-test took 15s at 11:23:36
Is it possible that shopify has a rate limit or something similar set up at Cloudflare which slows the connection down?
github-actions[bot]
commented
7 months ago This issue seems inactive. If it's still relevant, please add a comment saying so. Otherwise, take no action. → If there's no activity within a week, then a bot will automatically close this. Thanks for helping to improve Shopify's dev tooling and experience.
P.S. You can learn more about why we stale issues here.
dzschille
commented
7 months ago The issue seems a lot better now: the same shopify theme pull --store=shpfy-cli-test.myshopify.com --verbose now takes only 2:30 minutes. And i see now in the verbose output that the slowdown is indeed because of an API limit (Near API call limit, waiting 2 seconds):
...
┃ DEBUG **Near API call limit, waiting 2 seconds**
┃ DEBUG GET https://shpfy-cli-test.myshopify.com/admin/api/unstable/themes/133896143043/assets.json?asset%5Bkey%5D=templates%2Fproduct.json with request_id: fbed5c88-dcfd-4ea1-ab05-1b4600dc4d60-1712573097
┃ DEBUG get templates/blog.json 97%
┃ 97% DEBUG get templates/index.json
┃ DEBUG GET https://shpfy-cli-test.myshopify.com/admin/api/unstable/themes/133896143043/assets.json?asset%5Bkey%5D=templates%2Fblog.json with request_id: d2ae8eaf-bdf6-4b43-b2d0-f154e26b676c-1712573099
┃ DEBUG get templates/list-collections.json
┃ DEBUG GET https://shpfy-cli-test.myshopify.com/admin/api/unstable/themes/133896143043/assets.json?asset%5Bkey%5D=templates%2Findex.json with request_id: cdee77c0-5aab-4fba-9298-fc8c81fb9269-1712573099
┃ DEBUG get templates/page.json
┃ DEBUG GET https://shpfy-cli-test.myshopify.com/admin/api/unstable/themes/133896143043/assets.json?asset%5Bkey%5D=templates%2Flist-collections.json with request_id: e515474d-9eef-4c0d-9534-1d6394ba0765-1712573100
┃ DEBUG get sections/header-group.json
┃ DEBUG GET https://shpfy-cli-test.myshopify.com/admin/api/unstable/themes/133896143043/assets.json?asset%5Bkey%5D=templates%2Fpage.json with request_id: f051baab-b2e8-4e58-94ec-3959c0aab6d7-1712573100
┃ DEBUG get sections/footer-group.json
┃ DEBUG GET https://shpfy-cli-test.myshopify.com/admin/api/unstable/themes/133896143043/assets.json?asset%5Bkey%5D=sections%2Fheader-group.json with request_id: e48ad20e-059e-453e-80af-bde7d91990c0-1712573100
┃ DEBUG **Near API call limit, waiting 2 seconds**
┃ DEBUG GET https://shpfy-cli-test.myshopify.com/admin/api/unstable/themes/133896143043/assets.json?asset%5Bkey%5D=sections%2Ffooter-group.json with request_id: 2109acdf-3439-4b12-95c2-811b03b9390a-1712573100
Please confirm that you have:
In which of these areas are you experiencing a problem?
Theme
Expected behavior
If i run
theme pullfor the first time to download a theme in an empty directory it should download all theme files in a few seconds.Actual behavior
If i run
theme pullfor the first time to download a theme in an empty directory it needs minutes (often more than 10) to download all theme files.Also the cmd_all_timing_active_ms value is around half the real time it needed to download the theme.
See the verbose output where i downloaded a fresh installed Dawn theme what needed 11 minutes. The size of all theme files are just 5,2 MB, my internet connection is 1Gbit/s.
Verbose output
Reproduction steps
shopify theme pull --store=your-test-store.myshopify.comOperating System
Ubuntu 23.10
Shopify CLI version (check your project's
package.jsonif you're not sure)3.56.0
Shell
zsh
Node version (run
node -vif you're not sure)v21.1.0
What language and version are you using in your application?
No response