arugyani / shopIQ

LLM powered high-consideration research tool for UTD senior project
1 stars 0 forks source link

backend: web scraping: scraping product information #27

Open divyamk opened 7 months ago

divyamk commented 7 months ago

Product Information

divyamk commented 7 months ago

Scraped Product Information

all of the above have been scraped and added to a json

Example JSON Output

{
  "Insignia - 40\" Class LED Full HD TV": {
    "title": "Insignia - 40\" Class LED Full HD TV",
    "bullets": [
      "The 40\" LED screen provides enhanced viewing experience",
      "Resolution of 1080p delivers high-quality visuals",
      "HDMI and USB ports allow reliable connectivity with compatible devices"
    ],
    "description": "INSIGNIA- 40\" CLASS LED FULL HD 1080p (Full HD) resolution 60Hz refresh rateEnhance your viewing experience with this 40-inch Insignia LED TV. The 60Hz panel refresh rate reduces motion blur for high-quality pictures, while the 1080p resolution produces stunning visuals on the LCD screen. This Insignia LED TV has HDMI and USB ports for flexible connectivity to compatible external devices, and the headphone output allows for private listening.",
    "reviews": [
      "Good sound quality (747)",
      "View 747 reviews about Good sound quality. 79% of the reviews are positive.",
      "Easy to set up (625)",
      "View 625 reviews about Easy to set up. 96% of the reviews are positive.",
      "Lightweight (336)",
      "View 336 reviews about Lightweight. 100% of the reviews are positive.",
      "Easy to use (291)",
      "View 291 reviews about Easy to use. 95% of the reviews are positive.",
      "Quality build (146)",
      "View 146 reviews about Quality build. 71% of the reviews are positive.",
      "Fast input response (66)",
      "View 66 reviews about Fast input response. 59% of the reviews are positive.",
      "Quality remote (54)",
      "View 54 reviews about Quality remote. 83% of the reviews are positive.",
      "Attractive (38)",
      "View 38 reviews about Attractive. 97% of the reviews are positive.",
      "Vivid colors (30)",
      "View 30 reviews about Vivid colors. 81% of the reviews are positive.",
      "Display quality (19)",
      "View 19 reviews about Display quality. 58% of the reviews are positive.",
      "Minimal glare (3)",
      "View 3 reviews about Minimal glare. 100% of the reviews are negative."
    ],
    "price": "$129.99",
    "reviewScore": "4.6",
    "numReviews": "17,294 reviews"
  },
  "Insignia - 32\" Class LED HD TV": {
    "title": "Insignia - 32\" Class LED HD TV",
    "bullets": [
      "Number of DVI Inputs: 0",
      "Product Depth Without Stand: 2.9 inches",
      "Picture Quality Enhancement Technology: HDR Compatible",
      "Screen Mirroring Technology: Apple AirPlay",
      "Stand Included: Yes",
      "Number of DVI Inputs: 0",
      "Product Depth Without Stand: 2.9 inches",
      "Picture Quality Enhancement Technology: HDR Compatible",
      "Screen Mirroring Technology: Apple AirPlay",
      "Stand Included: Yes",
      "Curved Screen: No"
    ],
    "description": "Stay entertained with this 32\" Class LED HDTV. The 720p resolution renders stunning, detailed visuals on the LCD screen, while the HDMI and USB ports ensure flexible connectivity to compatible external devices. This Insignia 32\" Class LED HDTV has a 60Hz refresh rate that reduces motion blur for a smooth viewing experience. Display Type - LED. Screen Size: 31.5 inches. Dimensions: height 17 inches, Width 28.7 inches. screen - Size 31.5 inches.",
    "reviews": [
      "Good sound quality (747)",
      "View 747 reviews about Good sound quality. 79% of the reviews are positive.",
      "Easy to set up (625)",
      "View 625 reviews about Easy to set up. 96% of the reviews are positive.",
      "Lightweight (336)",
      "View 336 reviews about Lightweight. 100% of the reviews are positive.",
      "Easy to use (291)",
      "View 291 reviews about Easy to use. 95% of the reviews are positive.",
      "Quality build (146)",
      "View 146 reviews about Quality build. 71% of the reviews are positive.",
      "Fast input response (66)",
      "View 66 reviews about Fast input response. 59% of the reviews are positive.",
      "Quality remote (54)",
      "View 54 reviews about Quality remote. 83% of the reviews are positive.",
      "Attractive (38)",
      "View 38 reviews about Attractive. 97% of the reviews are positive.",
      "Vivid colors (30)",
      "View 30 reviews about Vivid colors. 81% of the reviews are positive.",
      "Display quality (19)",
      "View 19 reviews about Display quality. 58% of the reviews are positive.",
      "Minimal glare (3)",
      "View 3 reviews about Minimal glare. 100% of the reviews are negative."
    ],
    "price": "$80.00",
    "reviewScore": "4.6",
    "numReviews": "17,358 reviews"
  },
  "Insignia - 55\" Class F30 Series LED 4K UHD Smart Fire TV NS-55F301NA22": {
    "title": "Insignia - 55\" Class F30 Series LED 4K UHD Smart Fire TV NS-55F301NA22",
    "bullets": [
      "Brand: INSIGNIA",
      "Color: Black",
      "Binding: Electronics",
      "model number: NS-55F301NA22",
      "Part Number: NS-55F301NA22",
      "Brand: INSIGNIA",
      "Color: Black",
      "Binding: Electronics",
      "model number: NS-55F301NA22",
      "Part Number: NS-55F301NA22",
      "Package Dimensions: 52.0 x 34.0 x 6.3 inches"
    ],
    "description": null,
    "reviews": [
      "Good sound quality (550)",
      "View 550 reviews about Good sound quality. 93% of the reviews are positive.",
      "Easy to set up (350)",
      "View 350 reviews about Easy to set up. 95% of the reviews are positive.",
      "Easy to use (291)",
      "View 291 reviews about Easy to use. 94% of the reviews are positive.",
      "Quality build (81)",
      "View 81 reviews about Quality build. 76% of the reviews are positive.",
      "Fast input response (72)",
      "View 72 reviews about Fast input response. 69% of the reviews are positive.",
      "Lightweight (79)",
      "View 79 reviews about Lightweight. 95% of the reviews are positive.",
      "Remote quality (59)",
      "View 59 reviews about Remote quality. 59% of the reviews are positive.",
      "Attractive (33)",
      "View 33 reviews about Attractive. 100% of the reviews are positive.",
      "Vivid colors (27)",
      "View 27 reviews about Vivid colors. 87% of the reviews are positive.",
      "Display quality (13)",
      "View 13 reviews about Display quality. 62% of the reviews are positive.",
      "Minimal glare (9)",
      "View 9 reviews about Minimal glare. 56% of the reviews are positive."
    ],
    "price": "$386.12",
    "reviewScore": "4.6",
    "numReviews": "6,961 reviews"
  },
  "Samsung 85\" CU7000 Crystal UHD 4K Smart TV": {
    "title": "Samsung 85\" CU7000 Crystal UHD 4K Smart TV",
    "bullets": [
      "Crystal Processor With 4K UHD Upscaling.",
      "Mega Contrast.",
      "Object Trackign Sound Lite.",
      "*Q-Series and S-Series soundbars sold separately.",
      "Samsung Smart TV Hub.",
      "Crystal Processor With 4K UHD Upscaling.",
      "Mega Contrast.",
      "Object Trackign Sound Lite.",
      "*Q-Series and S-Series soundbars sold separately.",
      "Samsung Smart TV Hub.",
      "*Content services may vary by regions and are subject to change without notifications.",
      "Samsung Gaming Hub.",
      "Where gaming comes together. Stream your favorite games right from your TV. No console required.",
      "*High speed internet connection, additional gaming service subscriptions and compatible controller required."
    ],
    "description": "True to life color. Effortless connectivity. Dazzling 4K value. Samsung Crystal UHD is worth a look (and more). Effortlessly access TV shows, movies and ambient content using the Samsung Smart Hub, or find a range of great games on the Samsung Gaming Hub. * Enjoy content even more clear than it was created as its upgraded to 4K resolution. Color and contrast are also improved with our PurColor and Mega Contrast technologies that instantly analyze and adjust what you see on screen. As all the visual details shine through, you’ll be surrounded by 3D sound that moves with the action, engaging all your senses. *High speed internet connection, additional gaming service subscriptions and compatible controller required.",
    "reviews": [
      "Good sound quality (1212)",
      "View 1212 reviews about Good sound quality. 77% of the reviews are positive.",
      "Easy to use (816)",
      "View 816 reviews about Easy to use. 86% of the reviews are positive.",
      "Easy to set up (765)",
      "View 765 reviews about Easy to set up. 89% of the reviews are positive.",
      "Fast input response (403)",
      "View 403 reviews about Fast input response. 60% of the reviews are positive.",
      "Quality build (256)",
      "View 256 reviews about Quality build. 76% of the reviews are positive.",
      "Low quality remote (226)",
      "View 226 reviews about Low quality remote. 72% of the reviews are negative.",
      "Vivid colors (221)",
      "View 221 reviews about Vivid colors. 94% of the reviews are positive.",
      "Lightweight (179)",
      "View 179 reviews about Lightweight. 87% of the reviews are positive.",
      "Attractive (179)",
      "View 179 reviews about Attractive. 99% of the reviews are positive.",
      "Quality display (33)",
      "View 33 reviews about Quality display. 66% of the reviews are positive.",
      "Minimal glare (43)",
      "View 43 reviews about Minimal glare. 63% of the reviews are negative."
    ],
    "price": "$1,097.99",
    "reviewScore": "4.6",
    "numReviews": "19,667 reviews"
  },
  "Insignia Class N10 Series LED Full HD TV - 43 in": {
    "title": "Insignia Class N10 Series LED Full HD TV - 43 in",
    "bullets": [
      "Labor: 1 year UPC: 600603298233 Insignia",
      "43\" Class N10 Series LED Full HD TV"
    ],
    "description": null,
    "reviews": [
      "Good sound quality (89)",
      "View 89 reviews about Good sound quality. 85% of the reviews are positive.",
      "Easy to set up (55)",
      "View 55 reviews about Easy to set up. 95% of the reviews are positive.",
      "Easy to use (35)",
      "View 35 reviews about Easy to use. 97% of the reviews are positive.",
      "Lightweight (30)",
      "View 30 reviews about Lightweight. 100% of the reviews are positive.",
      "Quality build (12)",
      "View 12 reviews about Quality build. 75% of the reviews are positive.",
      "Visual appeal (8)",
      "View 8 reviews about Visual appeal. 100% of the reviews are positive.",
      "Fast input response (6)",
      "View 6 reviews about Fast input response. 67% of the reviews are positive.",
      "Remote quality (6)",
      "View 6 reviews about Remote quality. 67% of the reviews are positive.",
      "Color vividness (4)",
      "View 4 reviews about Color vividness. 100% of the reviews are positive.",
      "Display quality (1)",
      "View 1 review about Display quality. 100% of the reviews are negative."
    ],
    "price": "$184.49",
    "reviewScore": "4.6",
    "numReviews": "1,259 reviews"
  },
  "Hisense - 40\" Class A4 Series LED Full HD Smart Vidaa TV": {
    "title": "Hisense - 40\" Class A4 Series LED Full HD Smart Vidaa TV",
    "bullets": [
      "Sports mode",
      "Works with Alexa",
      "1080 resolution",
      "VIDAA OS UI",
      "Slim bezel design",
      "Sports mode",
      "Works with Alexa",
      "1080 resolution",
      "VIDAA OS UI",
      "Slim bezel design",
      "Motion Rate 120",
      "Game mode",
      "Right at the music scene"
    ],
    "description": "The Hisense A4 Series HD Smart TV is an entertainment must-have that features tons of builtin apps via the Hisense Smart Platform (VIDAA OS), making it easier than ever to personalize your entertainment experience. The A4 series also has DTS Virtual-X incorporated to deliver a crisp and immersive sound. It has all the features for you to keep up with all of your entertainment. The screen size also makes it ideal for smaller rooms and budgets.",
    "reviews": [
      "Good sound quality (23)",
      "View 23 reviews about Good sound quality. 92% of the reviews are positive.",
      "Easy to set up (12)",
      "View 12 reviews about Easy to set up. 80% of the reviews are positive.",
      "Easy to use (13)",
      "View 13 reviews about Easy to use. 93% of the reviews are positive.",
      "Fast input response (7)",
      "View 7 reviews about Fast input response. 57% of the reviews are positive.",
      "Quality build (6)",
      "View 6 reviews about Quality build. 86% of the reviews are positive.",
      "Lightweight (4)",
      "View 4 reviews about Lightweight. 100% of the reviews are positive.",
      "Color vividness (3)",
      "View 3 reviews about Color vividness. 100% of the reviews are positive.",
      "Remote quality (3)",
      "View 3 reviews about Remote quality. 67% of the reviews are negative.",
      "Visual appeal (1)",
      "View 1 review about Visual appeal. 100% of the reviews are positive."
    ],
    "price": "$129.99",
    "reviewScore": "4.5",
    "numReviews": "384 reviews"
  },
  "Insignia - 43\" Class F30 Series LED 4K UHD Smart Fire TV": {
    "title": "Insignia - 43\" Class F30 Series LED 4K UHD Smart Fire TV",
    "bullets": [
      "4K Ultra HD (2160p resolution)",
      "High Dynamic Range",
      "Alexa voice control",
      "Access thousands of shows with Fire TV",
      "DTS Studio Sound",
      "4K Ultra HD (2160p resolution)",
      "High Dynamic Range",
      "Alexa voice control",
      "Access thousands of shows with Fire TV",
      "DTS Studio Sound",
      "Supports HDMI ARC and HDMI eARC",
      "Versatile connections",
      "Parental controls"
    ],
    "description": null,
    "reviews": [
      "Good sound quality (569)",
      "View 569 reviews about Good sound quality. 89% of the reviews are positive.",
      "Easy to set up (375)",
      "View 375 reviews about Easy to set up. 90% of the reviews are positive.",
      "Easy to use (350)",
      "View 350 reviews about Easy to use. 94% of the reviews are positive.",
      "Fast input response (114)",
      "View 114 reviews about Fast input response. 69% of the reviews are positive.",
      "Quality build (93)",
      "View 93 reviews about Quality build. 79% of the reviews are positive.",
      "Lightweight (91)",
      "View 91 reviews about Lightweight. 93% of the reviews are positive.",
      "Quality remote (52)",
      "View 52 reviews about Quality remote. 70% of the reviews are positive.",
      "Vivid colors (58)",
      "View 58 reviews about Vivid colors. 91% of the reviews are positive.",
      "Attractive (30)",
      "View 30 reviews about Attractive. 100% of the reviews are positive.",
      "Minimal glare (10)",
      "View 10 reviews about Minimal glare. 70% of the reviews are positive.",
      "Display quality (9)",
      "View 9 reviews about Display quality. 67% of the reviews are positive."
    ],
    "price": "$173.00",
    "reviewScore": "4.6",
    "numReviews": "8,225 reviews"
  },
  "Insignia 24-Inch Class F20 Series Smart HD 720P Fire TV (NS-24F201NA23, 2022 Model)": {
    "title": "Insignia 24-Inch Class F20 Series Smart HD 720P Fire TV (NS-24F201NA23, 2022 Model)",
    "bullets": [
      "View Your Favorite Movies, Shows And Games In High Definition.alexa Voice Control.",
      "Watch Over 1 Million Streaming Movies And Tv Episodes With Access To Tens Of Thousands Of Channels, Apps, And Alexa Skills.supports Apple Airplay.",
      "Share Videos, Photos, Music And More From An Apple Device To Your Tv With The Touch Of A Button.supports Hdmi Arc.",
      "Sends Audio Directly From The Hdmi Jack To A Compatible Soundbar Or Av Receiver, Removing The Need For An Extra Cable."
    ],
    "description": "720p resolution - View your favorite movies, shows and games in high definition. Alexa voice control - The Alexa Voice Remote lets you easily control your entertainment, search across apps, switch inputs, and more using just your voice. Press and hold the voice button and ask Alexa to easily find, launch, and control content, and even switch to cable. Fire TV experience built-in - Watch over 1 Million streaming movies and TV episodes with access to tens of thousands of channels, apps, and Alexa skills. Supports Apple AirPlay - Share videos, photos, music and more from an Apple device to your TV with the touch of a button. Supports HDMI ARC - Sends audio directly from the HDMI jack to a compatible soundbar or AV receiver, removing the need for an extra cable. Access live over-the-air channels and streaming with this 24-inch TV—and control it all with your voice. Ask Alexa to launch apps, search for titles and more. An HD experience in 720p provides crystal clear images. You won’t miss a second with the Insignia NS-24F201NA23 24\" Class F20 Series LED HD Smart Fire TV.",
    "reviews": [
      "Good sound quality (235)",
      "View 235 reviews about Good sound quality. 82% of the reviews are positive.",
      "Easy to set up (223)",
      "View 223 reviews about Easy to set up. 87% of the reviews are positive.",
      "Easy to use (142)",
      "View 142 reviews about Easy to use. 86% of the reviews are positive.",
      "Lightweight (81)",
      "View 81 reviews about Lightweight. 99% of the reviews are positive.",
      "Fast input response (81)",
      "View 81 reviews about Fast input response. 58% of the reviews are positive.",
      "Quality build (47)",
      "View 47 reviews about Quality build. 76% of the reviews are positive.",
      "Remote quality (33)",
      "View 33 reviews about Remote quality. 64% of the reviews are positive.",
      "Attractive (16)",
      "View 16 reviews about Attractive. 94% of the reviews are positive.",
      "Color vividness (12)",
      "View 12 reviews about Color vividness. 75% of the reviews are positive.",
      "Display quality (9)",
      "View 9 reviews about Display quality. 67% of the reviews are negative.",
      "Minimal glare (6)",
      "View 6 reviews about Minimal glare. 67% of the reviews are positive."
    ],
    "price": "$89.99",
    "reviewScore": "4.6",
    "numReviews": "4,453 reviews"
  },
  "Insignia - 50\" Class F30 Series LED 4K UHD Smart Fire TV": {
    "title": "Insignia - 50\" Class F30 Series LED 4K UHD Smart Fire TV",
    "bullets": [
      "32” Class LED HD Smart Fire TV Edition TV",
      "Fire TV experience built-in",
      "Voice Remote with Alexa",
      "Keeps getting smarter",
      "Endless entertainment",
      "32” Class LED HD Smart Fire TV Edition TV",
      "Fire TV experience built-in",
      "Voice Remote with Alexa",
      "Keeps getting smarter",
      "Endless entertainment",
      "Get it all in one place",
      "Alexa-enabled"
    ],
    "description": null,
    "reviews": [
      "Ease of use (2)",
      "View 2 reviews about Ease of use. 50% of the reviews are negative.",
      "Sound quality (1)",
      "View 1 review about Sound quality. 100% of the reviews are positive.",
      "Fast input response (1)",
      "View 1 review about Fast input response. 100% of the reviews are negative."
    ],
    "price": "$79.16",
    "reviewScore": "3.5",
    "numReviews": "28 reviews"
  }
}
divyamk commented 7 months ago

Added helper methods for scraping and added image link