search

mccgr / edgar

Code to manage data related to SEC EDGAR
31 stars 15 forks source link

Check EDGAR_DIR #37

Closed iangow closed 5 years ago

iangow commented 5 years ago

For me it is

igow@igow-z640:/media/igow/2TB/edgar/data/883505$ echo $EDGAR_DIR 
/media/igow/2TB/
bdcallen commented 5 years ago

@iangow 

bdcallen@igow-z640:~/edgar$ echo $EDGAR_DIR
/home/shared

EDGAR_DIR on my account is the home/shared folder

iangow commented 5 years ago

But there's nothing EDGAR-related there:

igow@igow-z640:/home/shared$ ls
streetevents_project  uploads
iangow commented 5 years ago

Where are the files you're downloading going then?

bdcallen commented 5 years ago 
bdcallen@igow-z640:~$ ls -l /home/
total 40
drwxr-xr-x 13 aliciah2   aliciah2   4096 Nov 10 13:17 aliciah2
drwxr-xr-x 27 bdcallen   bdcallen   4096 Sep 13 13:38 bdcallen
drwxr-xr-x 11 dcarrasco  dcarrasco  4096 Nov  2 15:31 dcarrasco
drwxr-xr-x  6 dcochrane1 dcochrane1 4096 Sep  5 09:39 dcochrane1
drwxr-xr-x 37 igow       igow       4096 Nov 16 16:00 igow
drwxr-xr-x 18 jpk        jpk        4096 Nov 16 14:57 jpk
drwxr-xr-x 11 olivia     olivia     4096 Jun 26 09:27 olivia
drwxrwxr-x  4 igow       mccgr      4096 Jun 30 16:47 shared
drwxr-xr-x  5 shollander shollander 4096 Oct  5  2017 shollander
drwxr-xr-x  4 stewartc3  stewartc3  4096 Mar 24  2018 stewartc3

Something weird is going on. This says that the shared directory was last updated on June 30. But to record the downloading of a filing document, get_filings_docs checks whether the downloaded file exists and returns True if so, and False if not.

bdcallen commented 5 years ago

Ran this in my R console

> Sys.getenv("EDGAR_DIR")
[1] ""
iangow commented 5 years ago

So probably going in your working directory. Perhaps move edgar there over to /home/shared/ and then I can pick it up from there.

bdcallen commented 5 years ago

@iangow Indeed

> getwd()
[1] "/home/bdcallen/edgar"

but edgar doesn't list the documents

bdcallen@igow-z640:~/edgar$ ls -a
.                                 download_filing_docs.R       extract_cusips.py          get_13D_filings.R             get_filings.R               .gitignore                   .Rhistory.old    test.R
..                                download_filing_functions.R  extract_cusips.R           get_accession_nos.R           get_form_345_filing_docs.R  problem_finding.R            .Rproj.user      update_edgar.sh
analysis                          download_item5_docs.R        filing_docs.md             get_filer_ciks.R              get_item_5.R                process_345_xml_documents.R  .Rproj.user.old
analyze_filer_ciks.R              edgar2.RData                 filings                    get_filing_doc_functions.R    get_item_no_desc.R          .RData                       server_logs
download_8K_item_205_files.R      edgar.Rproj                  forms_345_xml_functions.R  get_filing_docs.R             get_item_nos.R              readme.md                    test.htm
download_filing_docs_functions.R  extract_cusips.pl            get_13D_filing_details.R   get_filings_modified_draft.R  .git                        .Rhistory                    test.pl
iangow commented 5 years ago

Contents of my ~/.Rprofile are:

Sys.setenv(EDGAR_DIR="/media/igow/2TB/")
Sys.setenv(SE_DIR="/home/shared/uploads/")
Sys.setenv(PGHOST="localhost")
Sys.setenv(PGDATABASE="crsp")
Sys.setenv(WRDS_ID="iangow")