search

ostrolucky / Simple-Subreddit-Image-Downloader

Bash script for downloading images posted in specific subreddit with minimal footprint
GNU General Public License v2.0
54 stars 6 forks source link

Getting: parse error: Invalid numeric literal at line 1, column 10 #16

Closed Luvirini closed 6 months ago

Luvirini commented 8 months ago

On running: ./download-subreddit-images.sh battlemaps new all 50 I get the following:

% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 1486 100 1486 0 0 99066 0 --:--:-- --:--:-- --:--:-- 99066 parse error: Invalid numeric literal at line 1, column 10 parse error: Invalid numeric literal at line 1, column 10 parse error: Invalid numeric literal at line 1, column 10 parse error: Invalid numeric literal at line 1, column 10

Running on: Ubuntu 20.04.6 LTS with jq: 1.6-1ubuntu0.20.04.1 and curl: 7.68.0-1ubuntu2.22

ostrolucky commented 8 months ago

It works for me. It probably has nothing to with the script, but your network connectivity. My guess would be you are using proxy, corporate firewall, vpn or something like that, which blocks this. What do you get when you run

curl https://www.reddit.com/r/battlemaps/new/.json?raw_json=1&t=all

?

Luvirini commented 8 months ago

I am not using anything special like that, running the script on linux server I control with a very basic FW policy and no proxy/vpn or such.

Seems that Reddit is blocking the connection for some reason. So I have to check into that.

<!doctype html>

Blocked

whoa there, pardner!

Your request has been blocked due to a network policy.

Try logging in or creating an account here to get back to browsing.

If you're running a script or application, please register or sign in with your developer credentials here. Additionally make sure your User-Agent is not empty and is something unique and descriptive and try again. if you're supplying an alternate User-Agent string, try changing back to default as that can sometimes result in a block.

You can read Reddit's Terms of Service here.

if you think that we've incorrectly blocked you or you would like to discuss easier ways to get the data you want, please file a ticket here.

when contacting us, please include your ip address which is: (My IP) and reddit account

ostrolucky commented 8 months ago

We can change the user agent that script uses if you want. But you'll need to play with it and tell me which one works (if any).

Luvirini commented 8 months ago

It seems that this is indeed a problem at my/reddit end. I set up a proxy on that server and got the same result, So I set it up on a different server in a different location and it works. So seems that reddit is blocking the IP

Thanks for you debugging help.

A possible enhancement for a future version would be to give a more understandable error if the returned file is not json.