[BUG] reghdfe accidently deletes files from hard drive!

[poliquin]

Bug report

I am running Stata 15 03feb2020 on Ubuntu Linux, using reghdfe version 6.12.1 and ftools 2.48.0.

• When running reghdfe ... in either interactive mode or batch mode, the program should not delete files from the hard drive (except for temporary files that it owns). Running Stata in batch mode should produce a .log file in the working directory.
• reghdfe deletes all *.tmp and *.log files from the working directory.

Here is a reproducible example. Run this interactively...

clear all
set more off

! echo "Testing at $(date)" >> tests.log

sysuse auto
! ls *.log
// the log file is there

reghdfe price mpg, absorb(foreign)

! ls *.log
// no more log file! reghdfe deleted all *.tmp and *.log files from the working directory.
exit

I have found the culprit. The issue is the unlink_folder function in ftools, which is called by reghdfe here: https://github.com/sergiocorreia/reghdfe/blob/3c264a596a51968a0b32080bf709211a4d025443/current-code/reghdfe.ado#L71

That function is aggressively deleting files from the user's hard drive. When I comment out the unlink_folder line, everything works as expected.

[sergiocorreia]

Hi Chris,

I did a small change which hopefully should fix this. The line you mentioned should only be run when you are running reghdfe in parallel (e.g. reghdfe ... , parallel(2)) and otherwise will not run.

However, something that still baffles me a bit is that HDFE.parallel_dir should be empty in which case the unlink function should do nothing, so I suspect there is a deeper bug that I'll need to investigate more.

[poliquin]

I reinstalled from GitHub and now get the following error:

. clear all
. sysuse auto
(1978 automobile data)

. reghdfe price mpg, absorb(foreign)
* type mismatch:  exp.exp:  transmorphic found where struct expected
* r(3000);

Here's the trace with set trace on...

...
- cap syntax, worker [*]
- if (!c(rc)) {
  ParallelWorker, `options'
  exit
  }
- cap syntax anything(everything) [fw aw pw/], [*] VERSION(integer) [noWARN]
- if !c(rc) {
  _assert inlist(`version', 3, 5)
  if ("`warn'" != "nowarn") di as error "(running historical version of reghdfe: `version')"
  if ("`weight'"!="") local weightexp [`weight'=`exp']
  if (`version' == 3) {
  reghdfe3 `anything' `weightexp', `options'
  }
  else {
  reghdfe5 `anything' `weightexp', `options'
  }
  exit
  }
- if replay() {
  Replay `0'
  exit
  }
- loc keep_mata 0
- Cleanup 0 `keep_mata'
= Cleanup 0 0
  --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- begin reghdfe.Cleanup ---
  - args rc keep_mata
  - loc cleanup_folder = !`keep_mata' & ("$LAST_PARALLEL_DIR"!="")
  = loc cleanup_folder = !0 & (""!="")
  - mata: HDFE.parallel_dir
type mismatch:  exp.exp:  transmorphic found where struct expected

[leftvalue]

+1

type mismatch:  exp.exp:  transmorphic found where struct expected

dont know how to fix it. both fail on mac and docker container 'ceumicrodata/stata' i would appreciate it if you anyone could tell why it happend.

[sergiocorreia]

For some reason the python file failed to correctly update the /src folder, so the update only got through partially. Running "which reghdfe" should now return version 6.12.2 02Nov2021 instead of the June version.

Let me know if it works on your end so I can close

[poliquin]

Yes, it's working for me now and not deleting log files.

[leftvalue]

it works well for me now,thanks!