DrKain / scrape-youtube

A lightning fast package to scrape YouTube search results
MIT License
112 stars 30 forks source link

No results for certain short search terms #33

Closed accesstechnology-mike closed 3 years ago

accesstechnology-mike commented 3 years ago

Describe the bug On a UK located server, the search term 'duggee' doesn't pull through any results. (same issue for many other terms, lego, cats, 'kids toys play' )

note: safesearch is enabled through the header options: { Cookie: "PREF=f2=8000000" }

To Reproduce Steps to reproduce the behavior:

  1. On UK server, search 'duggee'
  2. No returned results available

Expected behavior Duggee should pull through the 20 results listed here: https://www.youtube.com/results?search_query=duggee

image

Screenshots image

Versions:

Additional context 'duggee' results pull through on a machine located in Pakistan.

DrKain commented 3 years ago

Package: 0.2.6

I assume you mean 2.0.6. Update the package to 2.0.9 and see if the problem persists, there have been several updates that could be related

accesstechnology-mike commented 3 years ago

Package: 0.2.6

I assume you mean 2.0.6. Update the package to 2.0.9 and see if the problem persists, there have been several updates that could be related

Not a typo! Didn't realise is was so out of date. Will update app to latest version and report back.

accesstechnology-mike commented 3 years ago

updated to latest v2.0.9. Problem still persists unfortunately.

image

For reference, most other searches are working fine:

image

'duggee' never works, 'lego' is intermittent

DrKain commented 3 years ago

Alright thanks for reporting. I ran a few tests on a server I have in London and wasn't able to reproduce the issue you're having.
I'll need to dig around more during the weekend. If you could provide a sample of the code (showing how you're using the package, filtering the results ect) that would be helpful.

accesstechnology-mike commented 3 years ago

I'm sorry. I'm not used to node and forgot to actually re-build the app once it was updated.

It does seem to fine now on latest version. Thanks for helping me out.

On Wed, 3 Feb 2021, 22:20 Kain, notifications@github.com wrote:

Alright thanks for reporting. I ran a few tests on a server I have in London and wasn't able to reproduce the issue you're having. I'll need to dig around more during the weekend. If you could provide a sample of the code (showing how you're using the package, filtering the results ect) that would be helpful.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/DrKain/scrape-youtube/issues/33#issuecomment-772865848, or unsubscribe https://github.com/notifications/unsubscribe-auth/ANFXY4JML2277EUQK4LHN7TS5HD3DANCNFSM4XAUHV2Q .

accesstechnology-mike commented 3 years ago

Hi, this is happening again on 2.0.9:

image

accesstechnology-mike commented 3 years ago

Code here:

const express = require("express");
const { default: youtube } = require("scrape-youtube");

const cache = require("../helpers/cache");

const Router = express.Router();

Router.get("/search", async (req, res) => {
  try {
    let { q, cached } = req.query;
    const allowCached = cached !== "false";
    if (!q) return res.send({ error: "Invalid query!" });
    q = q.trim();
    if (allowCached) {
      const cachedResults = await cache.getResults(q);
      if (cachedResults && cachedResults.length !== 0) {
        return res.send({ results: cachedResults, cached: true });
      }
    }
    let freshResults = await youtube.search(
      q,
      { safeSearch: true },
      { safeSearch: true, headers: { Cookie: "PREF=f2=8000000" } }
    );
    freshResults = freshResults.videos.map((video) => ({
      id: video.id,
      title: video.title,
    }));
    res.send({ results: freshResults, length: freshResults.length });
    if (freshResults.length > 0) await cache.saveResults(q, freshResults);
  } catch (error) {
    console.error(error);
    res.send({ error: "Backend error." });
  }
});

module.exports = Router;
accesstechnology-mike commented 3 years ago

image cats / lego and both intermittent too