jef / streetmerchant

🤖 The world's easiest, most powerful stock checker
https://jef.buzz/streetmerchant
MIT License
4.96k stars 1.31k forks source link

403 error on mediamarkt.de and saturn.de #1623

Closed LINZO-AT closed 3 years ago

LINZO-AT commented 3 years ago

streetmerchant should check product availability on mediamarkt.de and saturn.de, but even with sleep times of min 600000ms and max 700000ms my IP adress gets banned after a few hours, for about 1 or 2 days.

Since Mediamarkt & Saturn offers the best deals for the 3000 series in the German-speaking region, I sat down to find a solution.

My possible solution is to get the data from the Mediamarkt and Saturn API, following the API URL I found: https://www.mediamarkt.de/api/v1/graphql?operationName=GetProductAvailabilities&variables=%7B%22ids%22:%5B%222691244%22%5D%7D&extensions=%7B%22pwa%22%3A%7B%22salesLine%22%3A%22Saturn%22%2C%22country%22%3A%22DE%22%2C%22language%22%3A%22de%22%7D%2C%22persistedQuery%22%3A%7B%22version%22%3A1%2C%22sha256Hash%22%3A%22810286f7ae9368cb54ccd122b21e453bd00ed7dbf534b8259b8bed68f8da999f%22%7D%7D

As my solution I tried to use the part of code used in the newegg-Store File, but with the mediamarkt API URL:

import {Store} from './store';
import fetch from 'node-fetch';

export const Mediamarkt: Store = {
    currency: '€',
    labels: {
        captcha: {
            container: 'p',
            text: ['Das ging uns leider zu schnell.']
        },
        maxPrice: {
            container: 'span[font-family="price"]',
            euroFormat: false
        },
        outOfStock: [
            {
                container: '#root',
                text: ['Dieser artikel ist aktuell nicht verfügbar.']
            },
            {
                container: '#root',
                text: ['Leider keine Lieferung möglich']
            },
            {
                container: '#root',
                text: ['Nicht verfügbar']
            },
            {
                container: '#root',
                text: ['Dieser Artikel ist dauerhaft ausverkauft']
            }
        ]
    },
    links: [
        {
            brand: 'test:brand',
            model: 'test:model',
            series: 'test:series',
            url: 'https://www.mediamarkt.de/de/product/-2641856.html'
        },
        {
            brand: 'asus',
            model: 'strix',
            itemNumber: '2676607',
            series: '3070',
            url: 'https://www.mediamarkt.de/de/product/-2676607.html'
        },
        {
            brand: 'asus',
            model: 'strix oc',
            itemNumber: '2691244',
            series: '3070',
            url: 'https://www.mediamarkt.de/de/product/-2691244.html'
        }
    ],
    name: 'mediamarkt',
    realTimeInventoryLookup: async (itemNumber: string) => {
        const request_url =
            'https://www.mediamarkt.de/api/v1/graphql?operationName=GetProductAvailabilities&variables=%7B%22ids%22:%5B%22' +
            itemNumber +
            '%22%5D%7D&extensions=%7B%22pwa%22%3A%7B%22salesLine%22%3A%22Saturn%22%2C%22country%22%3A%22DE%22%2C%22language%22%3A%22de%22%7D%2C%22persistedQuery%22%3A%7B%22version%22%3A1%2C%22sha256Hash%22%3A%22810286f7ae9368cb54ccd122b21e453bd00ed7dbf534b8259b8bed68f8da999f%22%7D%7D';
        const response = await fetch(request_url);
        const response_json = await response.json();
        return (
            response_json.data.availabilityType === "IN_WAREHOUSE"
        );
    }
};

Unforunately I am getting the following error:

[9:53:24 AM] error :: ✖ [mediamarkt] asus 3070 strix - invalid json response body at https://www.mediamarkt.de/api/v1/graphql?operationName=GetProductAvailabilities&variables=%7B%22ids%22:%5B%222676607%22%5D%7D&extensions=%7B%22pwa%22%3A%7B%22salesLine%22%3A%22Saturn%22%2C%22country%22%3A%22DE%22%2C%22language%22%3A%22de%22%7D%2C%22persistedQuery%22%3A%7B%22version%22%3A1%2C%22sha256Hash%22%3A%22810286f7ae9368cb54ccd122b21e453bd00ed7dbf534b8259b8bed68f8da999f%22%7D%7D reason: Unexpected token < in JSON at position 0

According to Google, the error occurs because HTML code is being fetched and not data from a JSON format. I would be very grateful if someone could help me implement the Mediamarkt API into this wonderful project!

Thanks in advance!

Best regards, LINZO

fmsimp commented 3 years ago

Hey Linzo i can not help you with this. I am totally new to this subject. Do you have any good advice for me, what is a must have in the dotenv?

github-actions[bot] commented 3 years ago

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days

github-actions[bot] commented 3 years ago

This issue has been closed because it is stale. Reopen if necessary.

FreeworkEarth commented 3 years ago

Have you find a solutuon for this issue?