theme download gets stuck at 97%

[alexissel]

Hi there,

I recently updated themekit to latest version (1.1.1) and since then it cannot download all the files... I tried with 3 different themes and all of them get stuck around 97-98%

[development] Warning, this is the live theme on XXXXX-THEME. [development] 389|401 [===================================================================>--] 97 %

Any thoughts on this?

Thanks

OS: Mac Themekit: v1.1.1

[vfonic]

Hey @tanema! Thanks for the time you put into this!

I upgraded to --version=v1.1.3-issue801 and ran theme download on the same theme. Unfortunately, it actually didn't finish successfully. :/

I also tried couple of more times and I also setup a new Shopify store, uploaded the theme there and ran into the same issue.

I've got couple of such errors:

[development] error downloading assets/home_slide1.png: received a malformed response from shopify, this usually indicates a problem with your connection
[development] error downloading snippets/product-footer.liquid: received a malformed response from shopify, this usually indicates a problem with your connection

And then the very last line was:

finished command with errors

Another issue with this last line is that the program finished with exit code 0 (which means successful). So if this was on a CI, the build would seem like it's passing!

Repro steps:

1. Export the theme zip file
2. Extract and add all files to new git repo and commit the changes
3. Upload the theme to a new store by uploading exported zip file or by using themekit 1.0.3 (if I'm not mistaken: upload the theme w/o checksums)
4. Run theme update --version=v1.1.3-issue801
5. Delete some files just to have them in the git status: rm assets/* snippets/* sections/* layouts/* (better do this through file explorer)
6. Run theme download

Expected:

git status -s displays no output (no changes)

Actual:

git status -s shows the list of previously deleted files that were not downloaded successfully

[tanema]

@vfonic by not finishing successfully do you mean finished with errors or hanging still?

Another issue with this last line is that the program finished with exit code 0 (which means successful)

There is an open PR to fix this.

received a malformed response from shopify,

These are not related.

[vfonic]

@tanema

@vfonic by not finishing successfully do you mean finished with errors or hanging still?

It finishes with the above errors, but finishes. Maybe the errors are unrelated, but some of the files were not downloaded, perhaps after failing several retries.

[tanema]

It finishes with the above errors, but finishes. Maybe the errors are unrelated, but some of the files were not downloaded, perhaps after failing several retries.

Thank you so much, you are absolutely right and this is very good news. I can address your other problems in separate issues.

[vfonic]

I agree with you. If you think that #839 PR fixes this issue good enough (my go skills are pretty bad) and that these other issues could be addressed in the following PRs, I think it's the best way forward, fixing one thing at a time.

Thank you very much, Tim!

[tanema]

Since we will not be merging anything around Black Friday/Cyber Monday, I have created a prerelease with the additions for this issue included in it. You can install the prerelease by running theme update --version=v1.1.3-pre, please share any feedback that you have on this.

[absorpheus]

@tanema thank you for your efforts in resolving this, much appreciated. I will test and give feedback soon 👍

[absorpheus]

@tanema I upgraded from ThemeKit v1.1.1 by running theme update --version=v1.1.3-pre and can confirm that it successfully downloads the theme from the store without any errors.

I also tested other parts of the API (watching for file changes when developing, downloading a single file, listing themes and deploying) and everything seems to be working as expected.

If anyone wants to test ThemeKit v1.1.3-pre using the node wrapper @shopify/themekit, I forked the project here so it automatically updates the binary to ThemeKit v1.1.3-pre after install.

This could be useful for testing a pre release of the binary and testing CI/CD workflows where your project is dependent on using the node-wrapper instead of using the ThemeKit binary itself.

OS: MacOS Mojave 10.14.5 ThemeKit: v1.1.3-pre

[alexissel]

I've run theme download for 4 different themes and this is what I've got:

Test 1: [development] 583|583 [======================================================] 100 % [development] 583 files, Downloaded: 546, No Change: 37

Test 2: [development] 542|542 [======================================================] 100 % [development] 542 files, Downloaded: 539, No Change: 3, Errored: 1 [development] Errors encountered: [development] error downloading config/settings_data.json: request failed after 5 retries finished command with errors

Test 3: [development] 543|543 [======================================================] 100 % [development] 543 files, Downloaded: 540, No Change: 3, Errored: 1 [development] Errors encountered: [development] error downloading config/settings_data.json: request failed after 5 retries finished command with errors

Test 4: [development] 255|255 [======================================================] 100 % [development] 255 files, Downloaded: 237, No Change: 18, Errored: 1 [development] Errors encountered: [development] error downloading config/settings_data.json: request failed after 5 retries finished command with errors

OS: MacOS Big Sur 11.0.1 ThemeKit: v1.1.3-pre

[alexissel]

Here's another test:

[development] 353|353 [======================================================] 100 % [development] 353 files, Downloaded: 332, No Change: 21, Errored: 4 [development] Errors encountered: [development] error downloading assets/about_us_parallax.jpg: could not read response body This may mean that the request was not able to finish successfully Http Response Status: 200 Request ID: 218ec5d6-25fd-497e-b6a7-a45773de010a

[development] error downloading assets/mailing_list_bg.jpg: could not read response body This may mean that the request was not able to finish successfully Http Response Status: 200 Request ID: c326a64d-17e2-4c9d-9ae5-a9025f06dc4f

[development] error downloading assets/banner_on_top_right.jpg: could not read response body This may mean that the request was not able to finish successfully Http Response Status: 200 Request ID: c3908fd5-d29d-4053-aa57-19f7633c12d5

[development] error downloading assets/block_collection_img.jpg: could not read response body This may mean that the request was not able to finish successfully Http Response Status: 200 Request ID: 189b0b86-83a3-4c5c-b1f2-6e6d94e8fc4e finished command with errors

[tanema]

@alexissel Thanks so much! The new error output is really helpful, though not that helpful because this tells me that the request succeeded but themekit could not read it for some reason. The request IDs showed me that the requests completed successfully on Shopify's side as well. This is helpful for #631 as it at least narrows down where the issues are stemming from. Thanks again for all the detail that you provided!

Edit: By any chance, are those files really large?

[vfonic]

@tanema at least in my case I had issues with large files failing to download. Looking at the file names (images, settings_data.json), I'd assume these are large as well.

BTW Thank you @tanema! I've been using themekit v1.1.3-pre and it works much better! I didn't know I could do theme deploy to add checksums that were missing. My theme deploys are much faster now! Thanks again!

[alexissel]

@tanema I started my own investigation about the config/settings_data.json to find out why it doesn't download the file... So, I opened the file with Shopify's code editor and saw these 3 errors. All 3 images have encrypted URLs.

[tanema]

v1.1.3 has been released with the fix in it. Please reopen this issue if the problem persists