fails while fetching ecommerce components - forEach on null

amsanket22 commented 2 years ago

Writing vue with_image_grid.vue ‼️ TypeError: Cannot read properties of null (reading 'forEach')

kiliman commented 2 years ago

Are you on the latest version? I just tried it myself and it appears to be working.

ALaDST commented 2 years ago

I'm getting the same error as @amsanket22.

During: Processing /components/ecommerce/components/product-overviews After: Writing vue with_image_grid.vue -> (Component got be written to output directory) (it's also the first of the five components that got processed, like in the screenshot of @amsanket22. Not like your screenshot where with_image_grid is the fourth component processed) TypeError: Cannot read properties of null (reading 'forEach')

at processComponent (file:///[...]/index.mjs:240:22)
at processComponentPage (file:///[...]/index.mjs:215:11)
at runMicrotasks (<anonymous>)
at processTicksAndRejections (node:internal/process/task_queues:96:5)
at async file:///[...]/index.mjs:431:26

Here are my settings (but I think it's more a simple error in your code):

Operating System: Windows 11
NPM-Version: 8.15.1

I've also disabled everything in the env file -> BUILDINDEX=0, TEMPLATES=0 And choose only -> LANGUAGES=html I've also promoted the process to system32 right-scheme.

Nothing works. Please take a look at this problem, because at the moment your tool is (atleast for me) unuseable in my setting.

My guess: I've looked in the page that doesn't work and at the pages that work. My observation is: The with_image_grid component has no "Get the code →"-Link on its top, like the other pages have. Maybe your crawler isn't aware of that.

kiliman commented 2 years ago

I'll take a look at this. Yes, the crawler is sensitive to the HTML structure, so any deviations could cause issues.

kiliman commented 2 years ago

I just tested this on Windows 11 and was able to download all components for Vue. I checked the TWUI site and the "With image grid" component does have the Vue code. I wonder if the issue you had was perhaps you were downloading during the middle of them deploying an update. Have you tried it multiple times to see if are still experiencing the same problem?

ALaDST commented 2 years ago

Hello kiliman,

sorry for my late answer, I was on vacation.

I have tried it more than multiple times, also now and I'm still getting the same error. For debugging I was looking into the code where the error appears. So the component that should be processed has no "snippets". Don't know why this is the case, but that is in the end the problem. I've then printed the JSON.stringify(component) in this case and got the following result:

{
    "name":"With tiered images",
    "uuid":"3304906f2c7f1cef35c787a7fb586015",
    "isNew":false,
    "downloadable":false,
    "imageCreditsHtml":{

    },
    "requiresJavaScript":true,
    "iframeHtml":"<!doctype html><meta charset=\"utf-8\"><meta name=\"viewport\" content=\"width=device-width, initial-scale=1, shrink-to-fit=no\"><link rel=\"stylesheet\" href=\"/v2-assets/components.css?id=aea5170682ce1cd9dc3d65c38020a970\"><script src=\"/v2-assets/components.js?id=04b6e66ea653b6e4103cba0bd678003d\"></script><script src=\"/js/iframe.js?id=165e0af3d723efd6d5d88c7ff17ca86a\" defer></script><script src=\"/js/alpine.js?id=aec657470ad97cbba5c2659fd719c2de\" defer></script><body class=\"antialiased font-sans bg-gray-200 overflow-hidden\"><div class=\"\" style=\"\">\n  \n  <div class=\"bg-white\">\n    <div class=\"pt-6 pb-16 sm:pb-24\">\n      <nav aria-label=\"Breadcrumb\" class=\"max-w-7xl mx-auto px-4 sm:px-6 lg:px-8\">\n        <ol role=\"list\" class=\"flex items-center space-x-4\">\n          \n            <li>\n              <div class=\"flex items-center\">\n                <a href=\"#\" class=\"mr-4 text-sm font-medium text-gray-900\">\n                  Women\n                </a>\n                <svg viewBox=\"0 0 6 20\" xmlns=\"http://www.w3.org/2000/svg\" aria-hidden=\"true\" class=\"h-5 w-auto text-gray-300\">\n                  <path d=\"M4.878 4.34H3.551L.27 16.532h1.327l3.281-12.19z\" fill=\"currentColor\"></path>\n                </svg>\n              </div>\n            </li>\n          \n            <li>\n              <div class=\"flex items-center\">\n                <a href=\"#\" class=\"mr-4 text-sm font-medium text-gray-900\">\n                  Clothing\n                </a>\n                <svg viewBox=\"0 0 6 20\" xmlns=\"http://www.w3.org/2000/svg\" aria-hidden=\"true\" class=\"h-5 w-auto text-gray-300\">\n                  <path d=\"M4.878 4.34H3.551L.27 16.532h1.327l3.281-12.19z\" fill=\"currentColor\"></path>\n                </svg>\n              </div>\n            </li>\n          \n          <li class=\"text-sm\">\n            <a href=\"#\" aria-current=\"page\" class=\"font-medium text-gray-500 hover:text-gray-600\">\n              Basic Tee\n            </a>\n          </li>\n        </ol>\n      </nav>\n      <div class=\"mt-8 max-w-2xl mx-auto px-4 sm:px-6 lg:max-w-7xl lg:px-8\">\n        <div class=\"lg:grid lg:grid-cols-12 lg:auto-rows-min lg:gap-x-8\">\n          <div class=\"lg:col-start-8 lg:col-span-5\">\n            <div class=\"flex justify-between\">\n              <h1 class=\"text-xl font-medium text-gray-900\">\n                Basic Tee\n              </h1>\n              <p class=\"text-xl font-medium text-gray-900\">\n                $35\n              </p>\n            </div>\n            <!-- Reviews -->\n            <div class=\"mt-4\">\n              <h2 class=\"sr-only\">Reviews</h2>\n              <div class=\"flex items-center\">\n                <p class=\"text-sm text-gray-700\">\n                  3.9\n                  <span class=\"sr-only\"> out of 5 stars</span>\n                </p>\n                <div class=\"ml-1 flex items-center\">\n                  \n                    <svg class=\"text-yellow-400 h-5 w-5 flex-shrink-0\" x-state:on=\"Active\" x-state:off=\"Inactive\" x-state-description=\"Active: &quot;text-yellow-400&quot;, Inactive: &quot;text-gray-200&quot;\" x-description=\"Heroicon name: mini/star\" xmlns=\"http://www.w3.org/2000/svg\" viewBox=\"0 0 20 20\" fill=\"currentColor\" aria-hidden=\"true\">\n  <path fill-rule=\"evenodd\" d=\"M10.868 2.884c-.321-.772-1.415-.772-1.736 0l-1.83 4.401-4.753.381c-.833.067-1.171 1.107-.536 1.651l3.62 3.102-1.106 4.637c-.194.813.691 1.456 1.405 1.02L10 15.591l4.069 2.485c.713.436 1.598-.207 1.404-1.02l-1.106-4.637 3.62-3.102c.635-.544.297-1.584-.536-1.65l-4.752-.382-1.831-4.401z\" clip-rule=\"evenodd\"></path>\n</svg>\n                  \n                    <svg class=\"text-yellow-400 h-5 w-5 flex-shrink-0\" x-state-description=\"undefined: &quot;text-yellow-400&quot;, undefined: &quot;text-gray-200&quot;\" x-description=\"Heroicon name: mini/star\" xmlns=\"http://www.w3.org/2000/svg\" viewBox=\"0 0 20 20\" fill=\"currentColor\" aria-hidden=\"true\">\n  <path fill-rule=\"evenodd\" d=\"M10.868 2.884c-.321-.772-1.415-.772-1.736 0l-1.83 4.401-4.753.381c-.833.067-1.171 1.107-.536 1.651l3.62 3.102-1.106 4.637c-.194.813.691 1.456 1.405 1.02L10 15.591l4.069 2.485c.713.436 1.598-.207 1.404-1.02l-1.106-4.637 3.62-3.102c.635-.544.297-1.584-.536-1.65l-4.752-.382-1.831-4.401z\" clip-rule=\"evenodd\"></path>\n</svg>\n                  \n                    <svg class=\"text-yellow-400 h-5 w-5 flex-shrink-0\" x-state-description=\"undefined: &quot;text-yellow-400&quot;, undefined: &quot;text-gray-200&quot;\" x-description=\"Heroicon name: mini/star\" xmlns=\"http://www.w3.org/2000/svg\" viewBox=\"0 0 20 20\" fill=\"currentColor\" aria-hidden=\"true\">\n  <path fill-rule=\"evenodd\" d=\"M10.868 2.884c-.321-.772-1.415-.772-1.736 0l-1.83 4.401-4.753.381c-.833.067-1.171 1.107-.536 1.651l3.62 3.102-1.106 4.637c-.194.813.691 1.456 1.405 1.02L10 15.591l4.069 2.485c.713.436 1.598-.207 1.404-1.02l-1.106-4.637 3.62-3.102c.635-.544.297-1.584-.536-1.65l-4.752-.382-1.831-4.401z\" clip-rule=\"evenodd\"></path>\n</svg>\n                  \n                    <svg class=\"text-yellow-400 h-5 w-5 flex-shrink-0\" x-state-description=\"undefined: &quot;text-yellow-400&quot;, undefined: &quot;text-gray-200&quot;\" x-description=\"Heroicon name: mini/star\" xmlns=\"http://www.w3.org/2000/svg\" viewBox=\"0 0 20 20\" fill=\"currentColor\" aria-hidden=\"true\">\n  <path fill-rule=\"evenodd\" d=\"M10.868 2.884c-.321-.772-1.415-.772-1.736 0l-1.83 4.401-4.753.381c-.833.067-1.171 1.107-.536 1.651l3.62 3.102-1.106 4.637c-.194.813.691 1.456 1.405 1.02L10 15.591l4.069 2.485c.713.436 1.598-.207 1.404-1.02l-1.106-4.637 3.62-3.102c.635-.544.297-1.584-.536-1.65l-4.752-.382-1.831-4.401z\" clip-rule=\"evenodd\"></path>\n</svg>\n                  \n                    <svg class=\"text-gray-200 h-5 w-5 flex-shrink-0\" x-state-description=\"undefined: &quot;text-yellow-400&quot;, undefined: &quot;text-gray-200&quot;\" x-description=\"Heroicon name: mini/star\" xmlns=\"http://www.w3.org/2000/svg\" viewBox=\"0 0 20 20\" fill=\"currentColor\" aria-hidden=\"true\">\n  <path fill-rule=\"evenodd\" d=\"M10.868 2.884c-.321-.772-1.415-.772-1.736 0l-1.83 4.401-4.753.381c-.833.067-1.171 1.107-.536 1.651l3.62 3.102-1.106 4.637c-.194.813.691 1.456 1.405 1.02L10 15.591l4.069 2.485c.713.436 1.598-.207 1.404-1.02l-1.106-4.637 3.62-3.102c.635-.544.297-1.584-.536-1.65l-4.752-.382-1.831-4.401z\" clip-rule=\"evenodd\"></path>\n</svg>\n                  \n                </div>\n                <div aria-hidden=\"true\" class=\"ml-4 text-sm text-gray-300\">·</div>\n                <div class=\"ml-4 flex\">\n                  <a href=\"#\" class=\"text-sm font-medium text-indigo-600 hover:text-indigo-500\">See all 512 reviews</a>\n                </div>\n              </div>\n            </div>\n          </div>\n\n          <!-- Image gallery -->\n          <div class=\"mt-8 lg:mt-0 lg:col-start-1 lg:col-span-7 lg:row-start-1 lg:row-span-3\">\n            <h2 class=\"sr-only\">Images</h2>\n\n            <div class=\"grid grid-cols-1 lg:grid-cols-2 lg:grid-rows-3 lg:gap-8\">\n              \n                <img src=\"https://tailwindui.com/img/ecommerce-images/product-page-01-featured-product-shot.jpg\" alt=\"Back of women&#039;s Basic Tee in black.\" class=\"lg:col-span-2 lg:row-span-2 rounded-lg\">\n              \n                <img src=\"https://tailwindui.com/img/ecommerce-images/product-page-01-product-shot-01.jpg\" alt=\"Side profile of women&#039;s Basic Tee in black.\" class=\"hidden lg:block rounded-lg\">\n              \n                <img src=\"https://tailwindui.com/img/ecommerce-images/product-page-01-product-shot-02.jpg\" alt=\"Front of women&#039;s Basic Tee in black.\" class=\"hidden lg:block rounded-lg\">\n              \n            </div>\n          </div>\n\n          <div class=\"mt-8 lg:col-span-5\">\n            <form>\n              <!-- Color picker -->\n              <div>\n                <h2 class=\"text-sm font-medium text-gray-900\">Color</h2>\n\n                <fieldset x-data=\"window.Components.radioGroup({ initialCheckedIndex: 0 })\" x-init=\"init()\" class=\"mt-2\">\n                  <legend class=\"sr-only\">\n                    Choose a color\n                  </legend>\n                  <div class=\"flex items-center space-x-3\">\n                    \n                      <label x-radio-group-option=\"\" class=\"-m-0.5 relative p-0.5 rounded-full flex items-center justify-center cursor-pointer focus:outline-none ring-gray-900\" x-description=\"Active and Checked: &quot;ring ring-offset-1&quot;\tNot Active and Checked: &quot;ring-2&quot;\" :class=\"{ 'ring ring-offset-1': (active === 'Black') && (value === 'Black'), 'undefined': !(active === 'Black') || !(value === 'Black'), 'ring-2': !(active === 'Black') && (value === 'Black'), 'undefined': (active === 'Black') || !(value === 'Black') }\">\n<input type=\"radio\" x-model=\"value\" name=\"color-choice\" value=\"Black\" class=\"sr-only\" aria-labelledby=\"color-choice-0-label\">\n                        <span id=\"color-choice-0-label\" class=\"sr-only\">\n                          Black\n                        </span>\n                        <span aria-hidden=\"true\" class=\"h-8 w-8 bg-gray-900 border border-black border-opacity-10 rounded-full\"></span>\n                      </label>\n                    \n                      <label x-radio-group-option=\"\" class=\"-m-0.5 relative p-0.5 rounded-full flex items-center justify-center cursor-pointer focus:outline-none ring-gray-400\" x-description=\"Active and Checked: &quot;ring ring-offset-1&quot;\tNot Active and Checked: &quot;ring-2&quot;\" :class=\"{ 'ring ring-offset-1': (active === 'Heather Grey') && (value === 'Heather Grey'), 'undefined': !(active === 'Heather Grey') || !(value === 'Heather Grey'), 'ring-2': !(active === 'Heather Grey') && (value === 'Heather Grey'), 'undefined': (active === 'Heather Grey') || !(value === 'Heather Grey') }\">\n<input type=\"radio\" x-model=\"value\" name=\"color-choice\" value=\"Heather Grey\" class=\"sr-only\" aria-labelledby=\"color-choice-1-label\">\n                        <span id=\"color-choice-1-label\" class=\"sr-only\">\n                          Heather Grey\n                        </span>\n                        <span aria-hidden=\"true\" class=\"h-8 w-8 bg-gray-400 border border-black border-opacity-10 rounded-full\"></span>\n                      </label>\n                    \n                  </div>\n                </fieldset>\n              </div>\n\n              <!-- Size picker -->\n              <div class=\"mt-8\">\n                <div class=\"flex items-center justify-between\">\n                  <h2 class=\"text-sm font-medium text-gray-900\">Size</h2>\n                  <a href=\"#\" class=\"text-sm font-medium text-indigo-600 hover:text-indigo-500\">See sizing chart</a>\n                </div>\n\n                <fieldset x-data=\"window.Components.radioGroup({ initialCheckedIndex: 2 })\" x-init=\"init()\" class=\"mt-2\">\n                  <legend class=\"sr-only\">\n                    Choose a size\n                  </legend>\n                  <div class=\"grid grid-cols-3 gap-3 sm:grid-cols-6\">\n                    \n                      <label x-radio-group-option=\"\" class=\"border rounded-md py-3 px-3 flex items-center justify-center text-sm font-medium uppercase sm:flex-1 cursor-pointer focus:outline-none\" x-description=\"In Stock: &quot;cursor-pointer&quot;, Out of Stock: &quot;opacity-25 cursor-not-allowed&quot;\tActive: &quot;ring-2 ring-offset-2 ring-indigo-500&quot;\tChecked: &quot;bg-indigo-600 border-transparent text-white hover:bg-indigo-700&quot;, Not Checked: &quot;bg-white border-gray-200 text-gray-900 hover:bg-gray-50&quot;\" :class=\"{ 'ring-2 ring-offset-2 ring-indigo-500': (active === 'XXS'), 'undefined': !(active === 'XXS'), 'bg-indigo-600 border-transparent text-white hover:bg-indigo-700': (value === 'XXS'), 'bg-white border-gray-200 text-gray-900 hover:bg-gray-50': !(value === 'XXS') }\">\n<input type=\"radio\" x-model=\"value\" name=\"size-choice\" value=\"XXS\" class=\"sr-only\" aria-labelledby=\"size-choice-0-label\">\n                        <span id=\"size-choice-0-label\">\n                          XXS\n                        </span>\n                      </label>\n                    \n                      <label x-radio-group-option=\"\" class=\"border rounded-md py-3 px-3 flex items-center justify-center text-sm font-medium uppercase sm:flex-1 cursor-pointer focus:outline-none\" x-description=\"In Stock: &quot;cursor-pointer&quot;, Out of Stock: &quot;opacity-25 cursor-not-allowed&quot;\tActive: &quot;ring-2 ring-offset-2 ring-indigo-500&quot;\tChecked: &quot;bg-indigo-600 border-transparent text-white hover:bg-indigo-700&quot;, Not Checked: &quot;bg-white border-gray-200 text-gray-900 hover:bg-gray-50&quot;\" :class=\"{ 'ring-2 ring-offset-2 ring-indigo-500': (active === 'XS'), 'undefined': !(active === 'XS'), 'bg-indigo-600 border-transparent text-white hover:bg-indigo-700': (value === 'XS'), 'bg-white border-gray-200 text-gray-900 hover:bg-gray-50': !(value === 'XS') }\">\n<input type=\"radio\" x-model=\"value\" name=\"size-choice\" value=\"XS\" class=\"sr-only\" aria-labelledby=\"size-choice-1-label\">\n                        <span id=\"size-choice-1-label\">\n                          XS\n                        </span>\n                      </label>\n                    \n                      <label x-radio-group-option=\"\" class=\"border rounded-md py-3 px-3 flex items-center justify-center text-sm font-medium uppercase sm:flex-1 cursor-pointer focus:outline-none\" x-description=\"In Stock: &quot;cursor-pointer&quot;, Out of Stock: &quot;opacity-25 cursor-not-allowed&quot;\tActive: &quot;ring-2 ring-offset-2 ring-indigo-500&quot;\tChecked: &quot;bg-indigo-600 border-transparent text-white hover:bg-indigo-700&quot;, Not Checked: &quot;bg-white border-gray-200 text-gray-900 hover:bg-gray-50&quot;\" :class=\"{ 'ring-2 ring-offset-2 ring-indigo-500': (active === 'S'), 'undefined': !(active === 'S'), 'bg-indigo-600 border-transparent text-white hover:bg-indigo-700': (value === 'S'), 'bg-white border-gray-200 text-gray-900 hover:bg-gray-50': !(value === 'S') }\">\n<input type=\"radio\" x-model=\"value\" name=\"size-choice\" value=\"S\" class=\"sr-only\" aria-labelledby=\"size-choice-2-label\">\n                        <span id=\"size-choice-2-label\">\n                          S\n                        </span>\n                      </label>\n                    \n                      <label x-radio-group-option=\"\" class=\"border rounded-md py-3 px-3 flex items-center justify-center text-sm font-medium uppercase sm:flex-1 cursor-pointer focus:outline-none\" x-description=\"In Stock: &quot;cursor-pointer&quot;, Out of Stock: &quot;opacity-25 cursor-not-allowed&quot;\tActive: &quot;ring-2 ring-offset-2 ring-indigo-500&quot;\tChecked: &quot;bg-indigo-600 border-transparent text-white hover:bg-indigo-700&quot;, Not Checked: &quot;bg-white border-gray-200 text-gray-900 hover:bg-gray-50&quot;\" :class=\"{ 'ring-2 ring-offset-2 ring-indigo-500': (active === 'M'), 'undefined': !(active === 'M'), 'bg-indigo-600 border-transparent text-white hover:bg-indigo-700': (value === 'M'), 'bg-white border-gray-200 text-gray-900 hover:bg-gray-50': !(value === 'M') }\">\n<input type=\"radio\" x-model=\"value\" name=\"size-choice\" value=\"M\" class=\"sr-only\" aria-labelledby=\"size-choice-3-label\">\n                        <span id=\"size-choice-3-label\">\n                          M\n                        </span>\n                      </label>\n                    \n                      <label x-radio-group-option=\"\" class=\"border rounded-md py-3 px-3 flex items-center justify-center text-sm font-medium uppercase sm:flex-1 cursor-pointer focus:outline-none\" x-description=\"In Stock: &quot;cursor-pointer&quot;, Out of Stock: &quot;opacity-25 cursor-not-allowed&quot;\tActive: &quot;ring-2 ring-offset-2 ring-indigo-500&quot;\tChecked: &quot;bg-indigo-600 border-transparent text-white hover:bg-indigo-700&quot;, Not Checked: &quot;bg-white border-gray-200 text-gray-900 hover:bg-gray-50&quot;\" :class=\"{ 'ring-2 ring-offset-2 ring-indigo-500': (active === 'L'), 'undefined': !(active === 'L'), 'bg-indigo-600 border-transparent text-white hover:bg-indigo-700': (value === 'L'), 'bg-white border-gray-200 text-gray-900 hover:bg-gray-50': !(value === 'L') }\">\n<input type=\"radio\" x-model=\"value\" name=\"size-choice\" value=\"L\" class=\"sr-only\" aria-labelledby=\"size-choice-4-label\">\n                        <span id=\"size-choice-4-label\">\n                          L\n                        </span>\n                      </label>\n                    \n                      <label x-radio-group-option=\"\" class=\"border rounded-md py-3 px-3 flex items-center justify-center text-sm font-medium uppercase sm:flex-1 opacity-25 cursor-not-allowed\" x-description=\"In Stock: &quot;cursor-pointer&quot;, Out of Stock: &quot;opacity-25 cursor-not-allowed&quot;\tActive: &quot;ring-2 ring-offset-2 ring-indigo-500&quot;\tChecked: &quot;bg-indigo-600 border-transparent text-white hover:bg-indigo-700&quot;, Not Checked: &quot;bg-white border-gray-200 text-gray-900 hover:bg-gray-50&quot;\" :class=\"{ 'ring-2 ring-offset-2 ring-indigo-500': (active === 'XL'), 'undefined': !(active === 'XL'), 'bg-indigo-600 border-transparent text-white hover:bg-indigo-700': (value === 'XL'), 'bg-white border-gray-200 text-gray-900 hover:bg-gray-50': !(value === 'XL') }\">\n<input type=\"radio\" x-model=\"value\" name=\"size-choice\" value=\"XL\" disabled class=\"sr-only\" aria-labelledby=\"size-choice-5-label\">\n                        <span id=\"size-choice-5-label\">\n                          XL\n                        </span>\n                      </label>\n                    \n                  </div>\n                </fieldset>\n              </div>\n\n              <button type=\"submit\" class=\"mt-8 w-full bg-indigo-600 border border-transparent rounded-md py-3 px-8 flex items-center justify-center text-base font-medium text-white hover:bg-indigo-700 focus:outline-none focus:ring-2 focus:ring-offset-2 focus:ring-indigo-500\">Add to cart</button>\n            </form>\n\n            <!-- Product details -->\n            <div class=\"mt-10\">\n              <h2 class=\"text-sm font-medium text-gray-900\">Description</h2>\n\n              <div class=\"mt-4 prose prose-sm text-gray-500\">\n                \n        <p>The Basic tee is an honest new take on a classic. The tee uses super soft, pre-shrunk cotton for true comfort and a dependable fit. They are hand cut and sewn locally, with a special dye technique that gives each tee it's own look.</p>\n        <p>Looking to stock your closet? The Basic tee also comes in a 3-pack or 5-pack at a bundle discount.</p>\n      \n              </div>\n            </div>\n\n            <div class=\"mt-8 border-t border-gray-200 pt-8\">\n              <h2 class=\"text-sm font-medium text-gray-900\">Fabric &amp; Care</h2>\n\n              <div class=\"mt-4 prose prose-sm text-gray-500\">\n                <ul role=\"list\">\n                  \n                    <li>Only the best materials</li>\n                  \n                    <li>Ethically and locally made</li>\n                  \n                    <li>Pre-washed and pre-shrunk</li>\n                  \n                    <li>Machine wash cold with similar colors</li>\n                  \n                </ul>\n              </div>\n            </div>\n\n            <!-- Policies -->\n            <section aria-labelledby=\"policies-heading\" class=\"mt-10\">\n              <h2 id=\"policies-heading\" class=\"sr-only\">Our Policies</h2>\n\n              <dl class=\"grid grid-cols-1 gap-6 sm:grid-cols-2 lg:grid-cols-1 xl:grid-cols-2\">\n                \n                  <div class=\"bg-gray-50 border border-gray-200 rounded-lg p-6 text-center\">\n                    <dt>\n                      <svg class=\"mx-auto h-6 w-6 flex-shrink-0 text-gray-400\" x-description=\"Heroicon name: outline/globe-americas\" xmlns=\"http://www.w3.org/2000/svg\" fill=\"none\" viewBox=\"0 0 24 24\" stroke-width=\"1.5\" stroke=\"currentColor\" aria-hidden=\"true\">\n  <path stroke-linecap=\"round\" stroke-linejoin=\"round\" d=\"M6.115 5.19l.319 1.913A6 6 0 008.11 10.36L9.75 12l-.387.775c-.217.433-.132.956.21 1.298l1.348 1.348c.21.21.329.497.329.795v1.089c0 .426.24.815.622 1.006l.153.076c.433.217.956.132 1.298-.21l.723-.723a8.7 8.7 0 002.288-4.042 1.087 1.087 0 00-.358-1.099l-1.33-1.108c-.251-.21-.582-.299-.905-.245l-1.17.195a1.125 1.125 0 01-.98-.314l-.295-.295a1.125 1.125 0 010-1.591l.13-.132a1.125 1.125 0 011.3-.21l.603.302a.809.809 0 001.086-1.086L14.25 7.5l1.256-.837a4.5 4.5 0 001.528-1.732l.146-.292M6.115 5.19A9 9 0 1017.18 4.64M6.115 5.19A8.965 8.965 0 0112 3c1.929 0 3.716.607 5.18 1.64\"></path>\n</svg>\n                      <span class=\"mt-4 text-sm font-medium text-gray-900\">\n                        International delivery\n                      </span>\n                    </dt>\n                    <dd class=\"mt-1 text-sm text-gray-500\">\n                      Get your order in 2 years\n                    </dd>\n                  </div>\n                \n                  <div class=\"bg-gray-50 border border-gray-200 rounded-lg p-6 text-center\">\n                    <dt>\n                      <svg class=\"mx-auto h-6 w-6 flex-shrink-0 text-gray-400\" x-description=\"Heroicon name: outline/currency-dollar\" xmlns=\"http://www.w3.org/2000/svg\" fill=\"none\" viewBox=\"0 0 24 24\" stroke-width=\"1.5\" stroke=\"currentColor\" aria-hidden=\"true\">\n  <path stroke-linecap=\"round\" stroke-linejoin=\"round\" d=\"M12 6v12m-3-2.818l.879.659c1.171.879 3.07.879 4.242 0 1.172-.879 1.172-2.303 0-3.182C13.536 12.219 12.768 12 12 12c-.725 0-1.45-.22-2.003-.659-1.106-.879-1.106-2.303 0-3.182s2.9-.879 4.006 0l.415.33M21 12a9 9 0 11-18 0 9 9 0 0118 0z\"></path>\n</svg>\n                      <span class=\"mt-4 text-sm font-medium text-gray-900\">\n                        Loyalty rewards\n                      </span>\n                    </dt>\n                    <dd class=\"mt-1 text-sm text-gray-500\">\n                      Don&#039;t look at other tees\n                    </dd>\n                  </div>\n                \n              </dl>\n            </section>\n          </div>\n        </div>\n      </div>\n    </div>\n  </div>\n\n</div></body>",
    "snippets":null,
    "largeWidth":1280,
    "largeHeight":1754,
    "smallWidth":375,
    "smallHeight":1866,
    "smallScreenshotUrl":"/img/components/product-overviews.01-with-tiered-images-xs.jpg",
    "largeScreenshotUrl":"/img/components/product-overviews.01-with-tiered-images-xl.jpg"
}

So this is the component with the problem. Maybe you have an idea why this fails. Because it is working for you I could imagine that it probably relates to different account models? Over the env I enter the account login information. Is it possible that my account does not see anything in opposite to yours, which leads to the error?

Otherwise I wouldn't understand why we got different results ...

Ennazk commented 2 years ago

I am getting this error because I don't have e-commerce components in my purchase.

How can I ignore e-commerce componenets?

manveru commented 1 year ago

This worked for me:

diff --git a/index.mjs b/index.mjs
index 34711e9..cd65616 100755
--- a/index.mjs
+++ b/index.mjs
@@ -422,6 +422,7 @@ async function saveTemplates() {
       const link = links[i]
       const url = $(link).attr('href')
       if (!url.startsWith('/components')) continue
+      if (url.startsWith('/components/ecommerce')) continue
       urls.push(url)
     }
     const count = process.env.COUNT || urls.length

kiliman commented 1 year ago

Thanks. I should probably add a setting where you can specify which component sets you have purchased (default to all).

connecteev commented 1 year ago

@kiliman I am having the same issue. I tested this with a fresh build from today. I haven't purchased the ecommerce bundle and the script dies with the same "forEach on null" error.

I also tested @manveru 's fix and it seems to be working. https://github.com/kiliman/tailwindui-crawler/issues/71#issuecomment-1500678576

Any chance you could make the list of component sets an .env setting? Suggestion: call it "components to exclude" so that you dont have to maintain the list of components to include...and a comment showing the list of all available components will be helpful.

diff --git a/index.mjs b/index.mjs
index a0c8726..19f7c8d 100755
--- a/index.mjs
+++ b/index.mjs
@@ -422,6 +422,7 @@ async function saveTemplates() {
       const link = links[i]
       const url = $(link).attr('href')
       if (!url.startsWith('/components')) continue
+if (url.startsWith('/components/ecommerce')) continue
       urls.push(url)
     }
     const count = process.env.COUNT || urls.length

kiliman commented 1 year ago

I added a COMPONENTS option to specify which component packages to download (defaults to all). See v4.1.0

connecteev commented 1 year ago

@kiliman amazing, I verified that thus works well now, thanks for the quick turnaround

kiliman / tailwindui-crawler

fails while fetching ecommerce components - forEach on null #71