Search suggestions are rate limited

[datanomi]

Official Instance

• [X] The bug is reproducable on the official hosted instance or is API related.

Describe the bug

I get a "Could not parse json response" response when it's loading search suggestions. Search itself works but search suggestions do not. Suggestions work on the official instance and other instances as well, but it is API related.

To Reproduce

Do any new search that is not cached by Cloudflare at https://api-piped.mha.fi/opensearch/suggestions?query=

Expected behavior

Return search suggestions successfully

Logs/Errors

An error occoured in the path: /opensearch/suggestions
org.schabi.newpipe.extractor.exceptions.ParsingException: Could not parse json response
        at org.schabi.newpipe.extractor.services.youtube.extractors.YoutubeSuggestionExtractor.suggestionList(YoutubeSuggestionExtractor.java:78)
        at me.kavin.piped.utils.ResponseHelper.opensearchSuggestionsResponse(ResponseHelper.java:516)
        at me.kavin.piped.ServerLauncher.lambda$mainServlet$17(ServerLauncher.java:169)
        at io.activej.http.AsyncServlet.lambda$ofBlocking$0(AsyncServlet.java:43)
        at io.activej.promise.Promise.lambda$ofBlocking$12(Promise.java:250)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        at java.base/java.lang.Thread.run(Unknown Source)
Caused by: com.grack.nanojson.JsonParserException: Unexpected token 'ml' on line 1, char 1
        at com.grack.nanojson.JsonTokener.createParseException(Unknown Source)
        at com.grack.nanojson.JsonTokener.createHelpfulException(Unknown Source)
        at com.grack.nanojson.JsonTokener.advanceToToken(Unknown Source)
        at com.grack.nanojson.JsonParser.advanceToken(Unknown Source)
        at com.grack.nanojson.JsonParser.parse(Unknown Source)
        at com.grack.nanojson.JsonParser$JsonParserContext.from(Unknown Source)
        at org.schabi.newpipe.extractor.services.youtube.extractors.YoutubeSuggestionExtractor.suggestionList(YoutubeSuggestionExtractor.java:64)
        ... 7 more

Browser, and OS with Version.

Firefox 104 on Arch Linux

Additional context

No response

[FireMasterK]

Hi, could you try fetching https://suggestqueries.google.com/complete/search?client=youtube&jsonp=JP&ds=yt&gl=US&q=test from your instance's IP? I suspect google has some kind of blocking perhaps on this endpoint :thinking:

[datanomi]

Hi, could you try fetching https://suggestqueries.google.com/complete/search?client=youtube&jsonp=JP&ds=yt&gl=US&q=test from your instance's IP? I suspect google has some kind of blocking perhaps on this endpoint 🤔

Hey, that seems to be the case.

Sorry...
We're sorry...
... but your computer or network may be sending automated queries. To protect our users, we can't process your request right now.

So I guess I can't do anything about it then 😕

[FireMasterK]

Can you see if the suggestions from the regular youtube.com website work?

[FireMasterK]

Hi, there's a new update that should fix this, could you let me know if it works (after updating your instance)?

[ghost]

Search suggestions disappear for me as soon as I enter 3rd letter.

[jerlam06]

I have the exact same problem, same error in dev console.