Failing When Merging IDX Files

[MineSafety]

On MacOS Mojave (10.14.5) and Python 3.7, $ python download_filings.py is failing on the last step merging the IDX files. This is the error I'm getting:

FileNotFoundError: [Errno 2] No such file or directory: '/Users/derajfast/anaconda3/envs/py-sec-edgar/lib/python3.7/site-packages/py_sec_edgar-0.1.0-py3.7.egg/refdata/merged_idx_files.pq'

I'm not sure if this is a bug, or if I'm doing something wrong (probably the latter since I'm very new to Python), but I believe this merged_idx_files.pqfile is supposed to be created automatically, but it doesn't seem to be.

Here is the full Terminal output, capturing installation and the error message: Output.zip

[ryansmccoy]

Hmm... It looks like an issue of wrong relative folder paths. I had decided to save the merged_idx_files.pq file in the the same directory where you cloned the code to (i.e py-sec-edgar/refdata/merged_idx_files.pq), but might make more sense saving it in the sec_gov/ directory. will take a look.

[MineSafety]

I figured it might be something like that. When I follow the README instructions:

$ git clone https://github.com/ryansmccoy/py-sec-edgar.git
$ cd py-sec-edgar
$ conda create -n py-sec-edgar python=3.6 pandas numpy lxml -y
$ source activate py-sec-edgar
$ sudo mkdir /sec_gov
$ sudo chown -R $USER:$USER /sec_gov
$ pip install -r requirements.txt

it creates the following directories:

/Users/user/anaconda3/envs/py-sec-edgar
/Users/user/py-sec-edgar/py_sec_edgar
/sec_gov

So should I be specifying an absolute path and then run setup.py and then download_filings.py from there?

(and sorry again, as I'm sure this is a pretty dumb/basic question)

[ryansmccoy]

No need to apologize.

You shouldn't need to run the setup.py (guessing you did?). It created a folder (like you said /Users/user/anaconda3/envs/py-sec-edgar) and i think that is messing things up.

First, delete that folder if it exists ( /Users/user/anaconda3/envs/py-sec-edgar).

Then, try pulling changes and re-running. I just made some changes that might fix the relative import issue.

[MineSafety]

Thanks, that did the trick. Very much appreciate the (fast) help.

One thing I noticed is that the /sec_gov/Archives folder doesn't get auto-created, so I had to manually add it. But after that, all seemed to work well.

[ryansmccoy]

No prob. Thanks