Improve error handling

[lucamlouzada]

This issue is part of an effort to implement substantive improvements to the lab template, as discussed in #16.

In this issue, the goal is to work on error handling. The main points to be addressed per the plans for next steps are:

• Make.sh should report errors in Python / R / Stata / Latex scripts.
• Improve logging -- make.log should nicely report the start and end of make logging, the start and end of each script, and as much verbose feedback on what's going on within each script as possible.
• Error tracking should happen in the run command. It should be reported both in the terminal and in the log file.
• Detect and report all kinds errors, not only errors when running a program, but also the errors that stops the program from running and even when program is not available via the command line.
• If the desired output have not been generated but the program still runs well, report an error.

I am assigning myself to work on this.

[lucamlouzada]

I pushed some significant changes to each of the run_xx scripts, as well as to the make.sh scripts. The main changes and the reasoning behind them are as follows:

Run_xx scripts

• The previous logic of the shell scripts in the lines beginning with "# run program, add output to logfile" and "# report on errors" were not appropriately directing error messages to the log file. I changed this logic so now we store the output and errors in a variable and then treat this variable accordingly, showing only clean messages and warnings in the terminal and leaving all the detailed information to the log files. Note that for run_stata, this approach does not work and thus it is necessary to parse the stata .log files to identify errors appropriately.
• I added a specific error check to the run_xx scripts to let users know if the programs are missing from the command line
• I added an output check to the run_xx scripts. In the beginning and end of the code, the scripts capture the state of the output folder and the script then lists the files that were created
• I added a timestamp message that records the beginning and ending of each script
• I changed the way some comments were written to make the scripts more standardized and made minor changes such as adding "\n" to the beginning of some lines to that the messages displayed in the terminal and the log are cleaner

Make.sh

• The previous code included "set -e" in the beginning which means the script would exit upon any errors. However, this prevents us from making more detailed error handling (if this is activated, the error handling specified in the run_xx scripts does not work because when there are errors make.sh just exits the code). Therefore, I added a specific error handler function to capture errors that happen in the shell script itself, while errors related to the run_xx scripts are dealt within them as explained above. The error handler redirects the errors to the log file, so it is only defined after the log file has been created - "set -e" is active for the beginning of the code, in case there are errors in the first line of the shell script, then it is deactivated and the error handler is activated.
• The previous make.sh script ran all scripts in a subshell. I am not entirely sure of the reason why (the comment said this was so it would return to the other working directory after running). However, this makes it more difficult to deal with errors within the subshell, so I removed this aspect and made the script simpler by just setting the working directory back manually after running the code

[lucamlouzada]

I just added two pushes to this branch to reflect a discussion with @gentzkow in which we noticed some errors when running the scripts from a zsh shell instead of bash. There seems to be some incompatibilities due to small differences in syntax such that some commands yield different outputs in zsh and bash. With the small changes I made, zsh should work.

In the run_xx scripts (6240a8d), the change was a minor alteration in syntax for the regex search in the output folder.

In the make.sh scripts (98dc2cb), I simplified the way error handling was happening in this shell script. I removed "set -e" from the beginning and instead defined a trap right at the start of the code (without redirecting output to the logfile, so we don't need to wait until the logfile has been created to activate this trap as was the case in my previous edit). I also brought back the subshells and manually reactivated the trap inside the subshell to ensure proper error handling.

However, there still are inconsistencies in the way that each deals with errors - for example, errors in the run_xx scripts in zsh get propagated to the parent script (make.sh), such that the inner error handling does not function the same way as in bash. I will keep investigating this issue.