generate-instances-json.sh is just stuck.

[shruuub]

So I don't know what happened, however, the ./generate-instances-json.sh script simply is stuck with no output. It worked before on the same machine (that was for ferrit), but now, somehow both dont work anymore. I doubt there's any fix as I cannot provide any logging information (there is none) but I still wanted to bring that up.

[Daniel-Valentine]

I doubt there's any fix as I cannot provide any logging information (there is none)

Try this:

bash -x ./generate-instances-json.sh [your args] >/tmp/libreddit-instances.txt 2>&1

and upload the log file generated in /tmp/libreddit-instances.txt. (Consider gzip-ing the log before uploading.)

[Daniel-Valentine]

@shruuub: Any update?

[shruuub]

Sorry, I've been quite busy, will try to do it today or tomorow

[shruuub]

Uh, uploading to github fails, I guess I just post it with all its clunkyness here:

+ set -o pipefail
++ date -I -u
+ TODAY=2022-11-15
+ DEPENDENCIES=(curl jq)
+ USER_AGENT=
+ [[ ./generate-instances-json.sh == \.\/\g\e\n\e\r\a\t\e\-\i\n\s\t\a\n\c\e\s\-\j\s\o\n\.\s\h ]]
+ main
+ local opt=
+ local OPTIND
+ local OPTARG
+ local failfast=n
+ local do_tor=y
+ get_opts=()
+ local -a get_opts
+ missing_deps=()
+ local -a missing_deps
+ local import_onions_from_file=
+ local input_file=/dev/stdin
+ local output_file=/dev/stdout
+ instance_entries=()
+ local -a instance_entries
+ imported_onions=()
+ local -a imported_onions
+ local instance_entry=
+ local -i rc=0
+ getopts :I:Tfhi:o: opt
+ IFS='
'
+ missing_deps=($(check_dependencies))
++ check_dependencies
++ local -i rc=0
++ for dep in "${DEPENDENCIES[@]}"
++ check_program curl
++ command -v curl
++ for dep in "${DEPENDENCIES[@]}"
++ check_program jq
++ command -v jq
++ return 0
+ [[ 0 -ne 0 ]]
+ [[ -n '' ]]
+ [[ y == \n ]]
+ can_tor
+ check_tor
+ pidof -q tor
+ check_program torsocks
+ command -v torsocks
+ [[ /dev/stdin != \/\d\e\v\/\s\t\d\i\n ]]
+ rows=()
+ local -a rows
+ mapfile rows

this is where it stops to do anything, and I eventually just quit via ctrl + c.

[Daniel-Valentine]

The script is waiting for input. You need to specify the input file with -i. From the helpdoc:

    -i INPUT_CSV
        Use INPUT_CSV as the input file. To read from stdin (the default
        behavior), either omit this option or provide `-i -`. Note that the
        argument provided to this option CANNOT be the same as the argument
        provided to -I.

This is generally how we execute the script:

./generate-instances-json.sh -I instances.json -i instances.txt -o instances.json

If you have Tor running, you can omit the arguments -I instances.json.

[shruuub]

sorry for the massive inconvinience, completly missed that!

[Daniel-Valentine]

@shruuub: Not a problem. In hindsight, the command-line interface for the script could have been more intuitive, but since this script is already out in the open we should probably keep it the way that it is.

That being said, I'd like to replace the script with a Python equivalent, and that is an opportunity for a better CLI interface.