not working - Githubissues

SunD004 commented 1 year ago

actually not working

migijc commented 1 year ago

descriptionUrl Error?

VladSam42 commented 1 year ago

yeah =>

/Users/vlad/Projects/test_aes/node_modules/aliexpress-product-scraper/src/aliexpressProductScraper.js:19
  const descriptionUrl = data.descriptionModule.descriptionUrl;
                                                ^

TypeError: Cannot read properties of undefined (reading 'descriptionUrl')
    at AliexpressProductScraper (/Users/vlad/Projects/test_aes/node_modules/aliexpress-product-scraper/src/aliexpressProductScraper.js:19:49)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

Node.js v20.5.1
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

rafaelsg-01 commented 10 months ago

Change the aliexpress-product-scraper/src/aliexpressProductScraper.js file code to this one:

const puppeteer = require('puppeteer');
const cheerio = require('cheerio');

const Variants = require('./variants');
const Feedback = require('./feedback');

async function AliexpressProductScraper(productId, feedbackLimit) {
  const FEEDBACK_LIMIT = feedbackLimit || 10;
  const browser = await puppeteer.launch();
  const page = await browser.newPage();

  /** Scrape the aliexpress product page for details */
  await page.goto(`https://www.aliexpress.com/item/${productId}.html`);
  const aliExpressData = await page.evaluate(() => runParams);

  const data = aliExpressData.data;

  /** Scrape the description page for the product using the description url */
  const descriptionUrl = data.productDescComponent.descriptionUrl;
  await page.goto(descriptionUrl);
  const descriptionPageHtml = await page.content();

  /** Build the AST for the description page html content using cheerio */
  const $ = cheerio.load(descriptionPageHtml);
  const descriptionData = $('body').html();

  /** Fetch the adminAccountId required to fetch the feedbacks */
  const adminAccountId = await page.evaluate(() => adminAccountId);
  await browser.close();

  let feedbackData = [];

  if (data.feedbackComponent.totalValidNum > 0) {
    feedbackData = await Feedback.get(
      data.productInfoComponent.id,
      adminAccountId,
      data.feedbackComponent.totalValidNum,
      FEEDBACK_LIMIT
    );
  }

  /** Build the JSON response with aliexpress product details */
  const json = {
    title: data.productInfoComponent.subject,
    categoryId: data.productInfoComponent.categoryId,
    productId: data.productInfoComponent.id,
    totalAvailableQuantity: data.inventoryComponent.totalAvailQuantity,
    description: descriptionData,
    orders: data.tradeComponent.formatTradeCount,
    storeInfo: {
      name: data.sellerComponent.storeName,
      companyId: data.sellerComponent.companyId,
      storeNumber: data.sellerComponent.storeNum,
      followers: data.sellerComponent.followingNumber,
      ratingCount: data.sellerComponent.positiveNum,
      rating: data.sellerComponent.positiveRate
    },
    ratings: {
      totalStar: 5,
      averageStar: data.feedbackComponent.averageStar,
      totalStartCount: data.feedbackComponent.totalValidNum,
      fiveStarCount: data.feedbackComponent.fiveStarNum,
      fourStarCount: data.feedbackComponent.fourStarNum,
      threeStarCount: data.feedbackComponent.threeStarNum,
      twoStarCount: data.feedbackComponent.twoStarNum,
      oneStarCount: data.feedbackComponent.oneStarNum
    },
    images:
      (data.imageModule &&
        data.imageModule.imagePathList) ||
      [],
    feedback: feedbackData,
    variants: Variants.get(data.priceComponent.skuPriceList),
    specs: data.productPropComponent.props,
    currency: data.currencyComponent.currencyCode,
    originalPrice: {
      min: data.priceComponent.origPrice.minAmount.value,
      max: data.priceComponent.origPrice.maxAmount.value
    },
    salePrice: {
      min: data.priceComponent.discountPrice.minActivityAmount 
        ? data.priceComponent.discountPrice.minActivityAmount.value 
        : data.priceComponent.discountPrice.minAmount.value,
      max: data.priceComponent.discountPrice.maxActivityAmount 
        ? data.priceComponent.discountPrice.maxActivityAmount.value 
        : data.priceComponent.discountPrice.maxAmount.value,
    }
  };

  return json;
}

module.exports = AliexpressProductScraper;

This code updates the file to the new Aliexpress object names. 🙂

sudheer-ranga commented 9 months ago

@SunD004 @migijc @nicelir1996 I have release a new version. Please use v2.0.2. This takes care of scraping the new layout from aliexpress. All the issues have been fixed in the new version along with adding shipping info to the json response.

migijc commented 9 months ago

Amazing! thanks so much for the update!

On Wed, Dec 6, 2023 at 1:57 PM Sudheer Ranga @.***> wrote:

@SunD004 https://github.com/SunD004 @migijc https://github.com/migijc @nicelir1996 https://github.com/nicelir1996 I have release a new version. Please use v2.0.2. This takes care of scraping the new layout from aliexpress. All the issues have been fixed in the new version along with adding shipping info to the json response.

— Reply to this email directly, view it on GitHub https://github.com/sudheer-ranga/aliexpress-product-scraper/issues/37#issuecomment-1843514239, or unsubscribe https://github.com/notifications/unsubscribe-auth/AYBFMOJZF54PNGEBKBQNB63YIC5X5AVCNFSM6AAAAAA2UZGC6KVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNBTGUYTIMRTHE . You are receiving this because you were mentioned.Message ID: @.***>

sudheer-ranga commented 8 months ago

Closing this issue. If the latest update has problem, feel free to reopen or create new issue.

sudheer-ranga / aliexpress-product-scraper

not working #37