Clarifications on the use of this API

MrJensen1 commented 8 years ago

Hello Okomarov

Sorry for my late answer. But I had to fix my account to WRDS to an Research Assistant before i could go further with MATLAB & WRDS (I wrote to you for 13 days ago).

My following questions is: I have made a Word/PDF in a Dropbox since I had printed something from my MATLAB command window and thought it will be easier for you to read :).

https://www.dropbox.com/sh/zd2o6hm93e0bhee/AADriA1_LCCfM5qhXXR5I-zea?dl=0

THANKS!

MrJensen1 commented 8 years ago

I have downloaded the latest .zip pagage from Github to setup MATLAB & WRDS

okomarov commented 8 years ago

Answering your questions: A1) If you can download data, the connection is indeed working. A2) Verbose means that it shoulf print more messages in the command window. You can set it to 0 to suppress most of the messages, in order to keep the command window clean A3) How do you pass specific queries? This package is built for bulk download of data. It is NOT an interface for building queries. For that, I recommend using the WRDS official API or their web-queries. A4) The syntax w.getDatasetInfo('libref.datasetname') requires you to specify a meaningful library and dataset, e.g. w.getDatasetInfo('CRSPA.DSF'). Similarly for w.cmd('some command that you want to execute on a unix server') and other syntaxes.

The idea behind my function, is to provide a FAST way to programmatically download the data. Hence, it only downloads full compressed datasets.

Hope this clarifies :)

MrJensen1 commented 8 years ago

Hi again

THANKS! :)

I have made a new PDF with some quick questions if you mind :).

Have a nice day!

/ Allan

------ Original Message ------ From: "Oleg Komarov" notifications@github.com To: "okomarov/wrds" wrds@noreply.github.com Cc: "MrJensen1" allangjerloev@gmail.com Sent: 03-03-2016 11:49:34 Subject: Re: [wrds] Hope you can help me :) (#8)

Answering your questions: A1) If you can download data, the connection is indeed working. A2) Verbose means that it shoulf print more messages in the command window. You can set it to 0 to suppress most of the messages, in order to keep the command window clean A3) How do you pass specific queries? This package is built for bulk download of data. It is NOT an interface for building queries. For that, I recommend using the WRDS official API or their wequeries. A4) The syntax w.getDatasetInfo('libref.datasetname') requires you to specify ome meaningful library and dataset, e.g. w.getDatasetInfo('CRSPA.DSF').

The idea behind my function, is to provide a FAST way to programmatically download the data. Hence, it only downloads full compressed datasets.

Hope this clarifies :)

— Reply to this email directly or view it on GitHub.

okomarov commented 8 years ago

@MrJensen1 I do not mind the pdfs, but it would be useful for others that might have the same type of questions, if they could read your questions directly in the body of the message. You can write in markup, and include snippets of code, and it should be easier than creating pdfs and uploading them to e.g. dropbox.

okomarov commented 8 years ago

Btw, I am not sure which pdf I should look into, the old one does not have any new questions, apart from the fact that if you get back information, then yes it is working.

Just a clarification, in my personal tests

bulk downloading the dataset,
importing it into Matlab, and
selecting it in Matlab,

is faster than using the alternative ufficial API that queries results throught the database toolbox.

MrJensen1 commented 8 years ago

Okay I will do that! :)

Hi again Thanks for your help! (I am not expert in MATLAB so the help is appreciated!) I does. I also need bulk of download, so your programming/script might fits perfect for me :).

Q.1 Is it possible to change the date range when downloading the bulk of download?

Q.2 Why do I get an error? (it worked well earlier today :S )

w.getDatasetInfo('CRSPA.MSI')
Retrieving dataset info for 'CRSPA.MSI'.
Request submitted to WRDS servers.
ans =
ERROR: Invalid file, /home/au/agj1988/tmp/cmd.sas.

Q.3 At The WRDS official API, I have problems with understanding their code to MATLAB Mabye you know how to rewrite the code?

query = exec(WRDS, ['SELECT date,dji FROM DJONES.DJDAILY (obs=100)']);
result = fetch(query);
result.Data

['SELECT date,dji FROM DJONES.DJDAILY (obs=100)'] I do not understand how to rewrite this piece of code to work? date – does I need to create variable? dji – does I need to create variable ?

(it is not your code but I hope you still can help)

Link: https://wrds-web.wharton.upenn.edu/wrds/support/Accessing%20and%20Manipulating%20the%20Data/_009MATLAB%20Programming/_004Sample%20Programs.cfm

Your help is appreciated! :)

MrJensen1 commented 8 years ago

?

okomarov commented 8 years ago

Let me see:

A1) No, bulk download means you get the dataset as is. You can then unzip the csv and use any tools you feel comfortable to do the selections, even excel.

A2) Thats a good question. The request is being sent but it is not going through. Do you still have the issue?

A3) Queries with ufficial API are built using SQL syntax. I would recommend using their web interface to query data, does not require your learnign SQL syntax.

MrJensen1 commented 8 years ago

Thanks!

But I still have the problem:

When running this code, it Looks like it is java problem?

`w.getDataset('CRSPA.MSI')

Retrieving 'CRSPA.MSI'. Request submitted to WRDS servers. Downloading file. Please, wait. Error using ssh2_main (line 169)

Error Transferring File SEE JAVA ERROR BELOW

Java exception occurred: java.io.IOException: Error during SCP transfer.

at ch.ethz.ssh2.SCPClient.get(SCPClient.java:703)

at ch.ethz.ssh2.SCPClient.get(SCPClient.java:596)

Caused by: java.io.IOException: Remote SCP error: scp: /home/au/agj1988/tmp/fc32d7e69_29f6_4ce0_ac26_8cb86be253f7.zip: No such file or directory

at ch.ethz.ssh2.SCPClient.receiveFiles(SCPClient.java:325)

at ch.ethz.ssh2.SCPClient.get(SCPClient.java:699)

... 1 more

Error in ssh2 (line 84) ssh2_struct = ssh2_main(ssh2_struct);

Error in scp (line 19) ssh2_struct = ssh2(ssh2_struct);

Error in scp_get (line 50) ssh2_struct = scp(ssh2_struct);

Error in wrds/getFile (line 114) obj.SSH2conn = scp_get(obj.SSH2conn, [rfname, rext], lpath, rpath);

Error in sas2csv (line 56) wrds.getFile(tmpzip, outfile);

Error in wrds/getDataset (line 14) sas2csv(wrds, libdataname, outfile);`

okomarov commented 8 years ago

Can you run the following commands and report back what output you get pls:

w.cmd('ls;ls tmp');

MrJensen1 commented 8 years ago

w.cmd('ls;ls tmp');

autoexec.sas sasuser.v94 tmp fadcda3cb_9ccb_4822_ba4b_01b465f3fb67.zip report.log

???

------ Original Message ------ From: "Oleg Komarov" notifications@github.com To: "okomarov/wrds" wrds@noreply.github.com Cc: "MrJensen1" allangjerloev@gmail.com Sent: 06-03-2016 02:14:39 Subject: Re: [wrds] Clarifications on the use of this API (#8)

Can you run the following commands and report back what output you get pls:

w.cmd('ls;ls tmp'); — Reply to this email directly or view it on GitHub.

okomarov commented 8 years ago

Try the following steps:

w.getDatasetNames('CRSPA')

check that MSI is in the output. If not, or if it comes out empty, it means you (or the institution) has no access to the Annual CRSP subscription (CRSPA). And I don't how you downloaded it in the first place earlier (maybe check again you are using the right account).

If you see MSI, then run the following again:

w.getDataset('CRSPA.MSI')

and after the error:

w.getFile('tmp/report.log');

Then, please copy-paste the full content of the report.log that you should find under you wrds/data folder.

MrJensen1 commented 8 years ago

Here is the report.log:

1 The SAS System 04:41 Thursday, March 3, 2016

NOTE: SAS (r) Proprietary Software 9.4 (TS1M2)

Licensed to UNIVERSITY OF PENNSYLVANIA - SFA T&R, Site 70001358.

NOTE: This session is executing on the SunOS 5.10 (SUN 64) platform.

NOTE: Updated analytical products:

SAS/STAT 13.2

SAS/ETS 13.2

SAS/OR 13.2

SAS/IML 13.2

SAS/QC 13.2

NOTE: Additional host information:

SUN SunOS SUN 64 5.10 Generic_150400-30 sun4u

You are running SAS 9. Some SAS 8 files will be automatically converted

by the V9 engine; others are incompatible. Please see

http://support.sas.com/rnd/migration/planning/platform/64bit.html

PROC MIGRATE will preserve current SAS file attributes and is

recommended for converting all your SAS libraries from any

SAS 8 release to SAS 9. For details and examples, please see

http://support.sas.com/rnd/migration/index.html

This message is contained in the SAS news file, and is presented upon

initialization. Edit the file "news" in the "misc/base" directory to

display site-specific news and information in the program log.

The command line option "-nonews" will prevent this display.

NOTE: SAS initialization used:

real time 0.14 seconds

cpu time 0.08 seconds

NOTE: AUTOEXEC processing beginning; file is /home/XX/USERNAME/autoexec.sas.

NOTE: Libref AUDIT was successfully assigned as follows:

Levels: 3

Engine(1): V9

Physical Name(1): /wrds/audit/sasdata/audit_comp

Engine(2): V9

Physical Name(2): /wrds/audit/sasdata/corp_legal

Engine(3): V9

Physical Name(3): /wrds/audit/sasdata/oia

NOTE: Libref BANK was successfully assigned as follows:

Levels: 3

Engine(1): V9

Physical Name(1): /wrds/bank/sasdata

Engine(2): V9

Physical Name(2): /wrds/bank/sasdata/commercial

Engine(3): V9

Physical Name(3): /wrds/bank/sasdata/holding

NOTE: Libref BLOCK was successfully assigned as follows:

Engine: V9

Physical Name: /wrds/blockholders/sasdata

NOTE: Libref BVD was successfully assigned as follows:

Levels: 7

Engine(1): V9

Physical Name(1): /wrds/bvd/sasdata/ama_v

Engine(2): V9

Physical Name(2): /wrds/bvd/sasdata/ama_l

Engine(3): V9

Physical Name(3): /wrds/bvd/sasdata/ama_m

Engine(4): V9

Physical Name(4): /wrds/bvd/sasdata/ama_s

Engine(5): V9

Physical Name(5): /wrds/bvd/sasdata/bs

Engine(6): V9

Physical Name(6): /wrds/bvd/sasdata/is

Engine(7): V9

Physical Name(7): /wrds/bvd/sasdata/os

NOTE: Libref BVDTRIAL was successfully assigned as follows:

Engine: V9

Physical Name: /wrds/bvd/sasdata/bs_trial

NOTE: Libref BVDSAMP was successfully assigned as follows:

Levels: 2

Engine(1): V9

Physical Name(1): /wrds/samples/sasdata/bvd_ama

Engine(2): V9

Physical Name(2): /wrds/samples/sasdata/bvd_bs

NOTE: Libref CBOE was successfully assigned as follows:

Engine: V9

Physical Name: /wrds/cboe/sasdata

NOTE: Libref CENTRIS was successfully assigned as follows:

Engine: V9

Physical Name: /wrds/centris/sasdata

NOTE: Libref CIQ was successfully assigned as follows:

Levels: 5

Engine(1): V9

Physical Name(1): /wrds/capitaliq/sasdata/pplintel

Engine(2): V9

Physical Name(2): /wrds/capitaliq/sasdata/helper

Engine(3): V9

Physical Name(3): /wrds/capitaliq/sasdata/keydev

Engine(4): V9

Physical Name(4): /wrds/capitaliq/sasdata/capstructure

Engine(5): V9

Physical Name(5): /wrds/capitaliq/sasdata/ratings

NOTE: Libref CIQSAMP was successfully assigned as follows:

Levels: 5

Engine(1): V9

Physical Name(1): /wrds/ciqsamp/sasdata/capstructure

Engine(2): V9

Physical Name(2): /wrds/ciqsamp/sasdata/helper

Engine(3): V9

Physical Name(3): /wrds/ciqsamp/sasdata/keydev

Engine(4): V9

Physical Name(4): /wrds/ciqsamp/sasdata/pplintel

Engine(5): V9

Physical Name(5): /wrds/ciqsamp/sasdata/ratings

NOTE: Libref CISDM was successfully assigned as follows:

Engine: V9

Physical Name: /wrds/morningstar/sasdata/cisdm

NOTE: Libref CISDMSMP was successfully assigned as follows:

Engine: V9

Physical Name: /wrds/samples/sasdata/morningstar

NOTE: Libref COMPB was successfully assigned as follows:

Engine: V9

Physical Name: /wrds/comp/sasdata/bank

NOTE: Libref COMPBD was successfully assigned as follows:

Engine: V9

Physical Name: /wrds/comp/sasdata/d_bank

NOTE: Libref COMPG was successfully assigned as follows:

Levels: 7

Engine(1): V9

Physical Name(1): /wrds/comp/sasdata/global

Engine(2): V9

Physical Name(2): /wrds/comp/sasdata/global/company

Engine(3): V9

Physical Name(3): /wrds/comp/sasdata/global/currency

Engine(4): V9

Physical Name(4): /wrds/comp/sasdata/global/dictionary

Engine(5): V9

Physical Name(5): /wrds/comp/sasdata/global/economy

Engine(6): V9

Physical Name(6): /wrds/comp/sasdata/global/index

Engine(7): V9

Physical Name(7): /wrds/comp/sasdata/global/security

NOTE: Libref COMPGD was successfully assigned as follows:

Levels: 7

Engine(1): V9

Physical Name(1): /wrds/comp/sasdata/d_global

Engine(2): V9

Physical Name(2): /wrds/comp/sasdata/d_global/company

Engine(3): V9

Physical Name(3): /wrds/comp/sasdata/d_global/currency

Engine(4): V9

Physical Name(4): /wrds/comp/sasdata/d_global/dictionary

Engine(5): V9

Physical Name(5): /wrds/comp/sasdata/d_global/economy

Engine(6): V9

Physical Name(6): /wrds/comp/sasdata/d_global/index

Engine(7): V9

Physical Name(7): /wrds/comp/sasdata/d_global/security

NOTE: Libref COMPA was successfully assigned as follows:

Levels: 14

Engine(1): V9

Physical Name(1): /wrds/comp/sasdata/naa

Engine(2): V9

Physical Name(2): /wrds/comp/sasdata/naa/company

Engine(3): V9

Physical Name(3): /wrds/comp/sasdata/naa/currency

Engine(4): V9

Physical Name(4): /wrds/comp/sasdata/naa/dictionary

Engine(5): V9

Physical Name(5): /wrds/comp/sasdata/naa/economy

Engine(6): V9

Physical Name(6): /wrds/comp/sasdata/naa/index

Engine(7): V9

Physical Name(7): /wrds/comp/sasdata/naa/industry

Engine(8): V9

Physical Name(8): /wrds/comp/sasdata/naa/insider

Engine(9): V9

Physical Name(9): /wrds/comp/sasdata/naa/institution

Engine(10): V9

Physical Name(10): /wrds/comp/sasdata/naa/pension

Engine(11): V9

Physical Name(11): /wrds/comp/sasdata/naa/rating

Engine(12): V9

Physical Name(12): /wrds/comp/sasdata/naa/security

Engine(13): V9

Physical Name(13): /wrds/comp/sasdata/naa/segments_current

Engine(14): V9

Physical Name(14): /wrds/comp/sasdata/naa/transparency

NOTE: Libref COMPM was successfully assigned as follows:

Levels: 14

Engine(1): V9

Physical Name(1): /wrds/comp/sasdata/nam

Engine(2): V9

Physical Name(2): /wrds/comp/sasdata/nam/company

Engine(3): V9

Physical Name(3): /wrds/comp/sasdata/nam/currency

Engine(4): V9

Physical Name(4): /wrds/comp/sasdata/nam/dictionary

Engine(5): V9

Physical Name(5): /wrds/comp/sasdata/nam/economy

Engine(6): V9

Physical Name(6): /wrds/comp/sasdata/nam/index

Engine(7): V9

Physical Name(7): /wrds/comp/sasdata/nam/industry

Engine(8): V9

Physical Name(8): /wrds/comp/sasdata/nam/insider

Engine(9): V9

Physical Name(9): /wrds/comp/sasdata/nam/institution

Engine(10): V9

Physical Name(10): /wrds/comp/sasdata/nam/pension

Engine(11): V9

Physical Name(11): /wrds/comp/sasdata/nam/rating

Engine(12): V9

Physical Name(12): /wrds/comp/sasdata/nam/security

Engine(13): V9

Physical Name(13): /wrds/comp/sasdata/nam/segments_current

Engine(14): V9

Physical Name(14): /wrds/comp/sasdata/nam/transparency

NOTE: Libref COMPMCUR was successfully assigned as follows:

Levels: 14

Engine(1): V9

Physical Name(1): /wrds/comp/sasdata/nam_current

Engine(2): V9

Physical Name(2): /wrds/comp/sasdata/nam_current/company

Engine(3): V9

Physical Name(3): /wrds/comp/sasdata/nam_current/currency

Engine(4): V9

Physical Name(4): /wrds/comp/sasdata/nam_current/dictionary

Engine(5): V9

Physical Name(5): /wrds/comp/sasdata/nam_current/economy

Engine(6): V9

Physical Name(6): /wrds/comp/sasdata/nam_current/index

Engine(7): V9

Physical Name(7): /wrds/comp/sasdata/nam_current/industry

Engine(8): V9

Physical Name(8): /wrds/comp/sasdata/nam_current/insider

Engine(9): V9

Physical Name(9): /wrds/comp/sasdata/nam_current/institution

Engine(10): V9

Physical Name(10): /wrds/comp/sasdata/nam_current/pension

Engine(11): V9

Physical Name(11): /wrds/comp/sasdata/nam_current/rating

Engine(12): V9

Physical Name(12): /wrds/comp/sasdata/nam_current/security

Engine(13): V9

Physical Name(13): /wrds/comp/sasdata/nam_current/segments_current

Engine(14): V9

Physical Name(14): /wrds/comp/sasdata/nam_current/transparency

NOTE: Libref COMPNAD was successfully assigned as follows:

Levels: 14

Engine(1): V9

Physical Name(1): /wrds/comp/sasdata/d_na

Engine(2): V9

Physical Name(2): /wrds/comp/sasdata/d_na/company

Engine(3): V9

Physical Name(3): /wrds/comp/sasdata/d_na/currency

Engine(4): V9

Physical Name(4): /wrds/comp/sasdata/d_na/dictionary

Engine(5): V9

Physical Name(5): /wrds/comp/sasdata/d_na/economy

Engine(6): V9

Physical Name(6): /wrds/comp/sasdata/d_na/index

Engine(7): V9

Physical Name(7): /wrds/comp/sasdata/d_na/industry

Engine(8): V9

Physical Name(8): /wrds/comp/sasdata/d_na/insider

Engine(9): V9

Physical Name(9): /wrds/comp/sasdata/d_na/institution

Engine(10): V9

Physical Name(10): /wrds/comp/sasdata/d_na/pension

Engine(11): V9

Physical Name(11): /wrds/comp/sasdata/d_na/rating

Engine(12): V9

Physical Name(12): /wrds/comp/sasdata/d_na/security

Engine(13): V9

Physical Name(13): /wrds/comp/sasdata/d_na/segments_current

Engine(14): V9

Physical Name(14): /wrds/comp/sasdata/d_na/transparency

NOTE: Libref COMPDCUR was successfully assigned as follows:

Levels: 14

Engine(1): V9

Physical Name(1): /wrds/comp/sasdata/d_na_current

Engine(2): V9

Physical Name(2): /wrds/comp/sasdata/d_na_current/company

Engine(3): V9

Physical Name(3): /wrds/comp/sasdata/d_na_current/currency

Engine(4): V9

Physical Name(4): /wrds/comp/sasdata/d_na_current/dictionary

Engine(5): V9

Physical Name(5): /wrds/comp/sasdata/d_na_current/economy

Engine(6): V9

Physical Name(6): /wrds/comp/sasdata/d_na_current/index

Engine(7): V9

Physical Name(7): /wrds/comp/sasdata/d_na_current/industry

Engine(8): V9

Physical Name(8): /wrds/comp/sasdata/d_na_current/insider

Engine(9): V9

Physical Name(9): /wrds/comp/sasdata/d_na_current/institution

Engine(10): V9

Physical Name(10): /wrds/comp/sasdata/d_na_current/pension

Engine(11): V9

Physical Name(11): /wrds/comp/sasdata/d_na_current/rating

Engine(12): V9

Physical Name(12): /wrds/comp/sasdata/d_na_current/security

Engine(13): V9

Physical Name(13): /wrds/comp/sasdata/d_na_current/segments_current

Engine(14): V9

Physical Name(14): /wrds/comp/sasdata/d_na_current/transparency

NOTE: Libref COMPSEG was successfully assigned as follows:

Engine: V9

Physical Name: /wrds/comp/sasdata/seghist

NOTE: Libref COMPSEGD was successfully assigned as follows:

Engine: V9

Physical Name: /wrds/comp/sasdata/d_seghist

NOTE: Libref EXECCOMP was successfully assigned as follows:

Engine: V9

Physical Name: /wrds/comp/sasdata/execcomp

NOTE: Libref COMP was successfully assigned as follows:

Levels: 26

Engine(1): V9

Physical Name(1): /wrds/comp/sasdata/seghist

Engine(2): V9

Physical Name(2): /wrds/comp/sasdata/nam

Engine(3): V9

Physical Name(3): /wrds/comp/sasdata/nam/company

Engine(4): V9

Physical Name(4): /wrds/comp/sasdata/nam/currency

Engine(5): V9

Physical Name(5): /wrds/comp/sasdata/nam/dictionary

Engine(6): V9

Physical Name(6): /wrds/comp/sasdata/nam/economy

Engine(7): V9

Physical Name(7): /wrds/comp/sasdata/nam/index

Engine(8): V9

Physical Name(8): /wrds/comp/sasdata/nam/industry

Engine(9): V9

Physical Name(9): /wrds/comp/sasdata/nam/insider

Engine(10): V9

Physical Name(10): /wrds/comp/sasdata/nam/institution

Engine(11): V9

Physical Name(11): /wrds/comp/sasdata/nam/pension

Engine(12): V9

Physical Name(12): /wrds/comp/sasdata/nam/rating

Engine(13): V9

Physical Name(13): /wrds/comp/sasdata/nam/security

Engine(14): V9

Physical Name(14): /wrds/comp/sasdata/nam/segments_current

Engine(15): V9

Physical Name(15): /wrds/comp/sasdata/nam/transparency

Engine(16): V9

Physical Name(16): /wrds/comp/sasdata/bank

Engine(17): V9

Physical Name(17): /wrds/comp/sasdata/global

Engine(18): V9

Physical Name(18): /wrds/comp/sasdata/global/company

Engine(19): V9

Physical Name(19): /wrds/comp/sasdata/global/currency

Engine(20): V9

Physical Name(20): /wrds/comp/sasdata/global/dictionary

Engine(21): V9

Physical Name(21): /wrds/comp/sasdata/global/economy

Engine(22): V9

Physical Name(22): /wrds/comp/sasdata/global/index

Engine(23): V9

Physical Name(23): /wrds/comp/sasdata/global/security

Engine(24): V9

Physical Name(24): /wrds/comp/sasdata/execcomp

Engine(25): V9

Physical Name(25): /wrds/comp/sasdata/filings

Engine(26): V9

Physical Name(26): /wrds/comp/sasdata/marginaltax

NOTE: Libref COMPD was successfully assigned as follows:

Levels: 23

Engine(1): V9

Physical Name(1): /wrds/comp/sasdata/d_seghist

Engine(2): V9

Physical Name(2): /wrds/comp/sasdata/d_na

Engine(3): V9

Physical Name(3): /wrds/comp/sasdata/d_na/company

Engine(4): V9

Physical Name(4): /wrds/comp/sasdata/d_na/currency

Engine(5): V9

Physical Name(5): /wrds/comp/sasdata/d_na/dictionary

Engine(6): V9

Physical Name(6): /wrds/comp/sasdata/d_na/economy

Engine(7): V9

Physical Name(7): /wrds/comp/sasdata/d_na/index

Engine(8): V9

Physical Name(8): /wrds/comp/sasdata/d_na/industry

Engine(9): V9

Physical Name(9): /wrds/comp/sasdata/d_na/insider

Engine(10): V9

Physical Name(10): /wrds/comp/sasdata/d_na/institution

Engine(11): V9

Physical Name(11): /wrds/comp/sasdata/d_na/pension

Engine(12): V9

Physical Name(12): /wrds/comp/sasdata/d_na/rating

Engine(13): V9

Physical Name(13): /wrds/comp/sasdata/d_na/security

Engine(14): V9

Physical Name(14): /wrds/comp/sasdata/d_na/segments_current

Engine(15): V9

Physical Name(15): /wrds/comp/sasdata/d_na/transparency

Engine(16): V9

Physical Name(16): /wrds/comp/sasdata/d_bank

Engine(17): V9

Physical Name(17): /wrds/comp/sasdata/d_global

Engine(18): V9

Physical Name(18): /wrds/comp/sasdata/d_global/company

Engine(19): V9

Physical Name(19): /wrds/comp/sasdata/d_global/currency

Engine(20): V9

Physical Name(20): /wrds/comp/sasdata/d_global/dictionary

Engine(21): V9

Physical Name(21): /wrds/comp/sasdata/d_global/economy

Engine(22): V9

Physical Name(22): /wrds/comp/sasdata/d_global/index

Engine(23): V9

Physical Name(23): /wrds/comp/sasdata/d_global/security

NOTE: Libref COMPH was successfully assigned as follows:

Levels: 3

Engine(1): V9

Physical Name(1): /wrds/comp/sasdata/ph

Engine(2): V9

Physical Name(2): /wrds/comp/sasdata/pit

Engine(3): V9

Physical Name(3): /wrds/comp/sasdata/urq

NOTE: Libref COMPSAMP was successfully assigned as follows:

Engine: V9

Physical Name: /wrds/compsamp/sasdata

NOTE: Libref COMPSNAP was successfully assigned as follows:

Engine: V9

Physical Name: /wrds/comp/sasdata/snapshot

NOTE: Libref COMSCORE was successfully assigned as follows:

Engine: V9

Physical Name: /wrds/comscore/sasdata

NOTE: Libref CRSPA was successfully assigned as follows:

Levels: 8

Engine(1): V9

Physical Name(1): /wrds/crsp/sasdata/a_stock

Engine(2): V9

Physical Name(2): /wrds/crsp/sasdata/a_stock62

Engine(3): V9

Physical Name(3): /wrds/crsp/sasdata/a_indexes

Engine(4): V9

Physical Name(4): /wrds/crsp/sasdata/a_ccm

Engine(5): V9

Physical Name(5): /wrds/crsp/sasdata/a_treasuries

Engine(6): V9

Physical Name(6): /wrds/crsp/sasdata/a_treas_bd

Engine(7): V9

Physical Name(7): /wrds/crsp/sasdata/a_treas_bm

Engine(8): V9

Physical Name(8): /wrds/crsp/sasdata/a_ziman

NOTE: Libref CRSPQ was successfully assigned as follows:

Levels: 9

Engine(1): V9

Physical Name(1): /wrds/crsp/sasdata/q_stock

Engine(2): V9

Physical Name(2): /wrds/crsp/sasdata/q_indexes

Engine(3): V9

Physical Name(3): /wrds/crsp/sasdata/q_ccm

Engine(4): V9

Physical Name(4): /wrds/crsp/sasdata/q_mutualfunds

Engine(5): V9

Physical Name(5): /wrds/crsp/sasdata/q_stock62

Engine(6): V9

Physical Name(6): /wrds/crsp/sasdata/q_treasuries

Engine(7): V9

Physical Name(7): /wrds/crsp/sasdata/q_treas_bd

Engine(8): V9

Physical Name(8): /wrds/crsp/sasdata/q_treas_bm

Engine(9): V9

Physical Name(9): /wrds/crsp/sasdata/q_ziman

NOTE: Libref CRSPM was successfully assigned as follows:

Levels: 8

Engine(1): V9

Physical Name(1): /wrds/crsp/sasdata/m_stock

Engine(2): V9

Physical Name(2): /wrds/crsp/sasdata/m_indexes

Engine(3): V9

Physical Name(3): /wrds/crsp/sasdata/m_ccm

Engine(4): V9

Physical Name(4): /wrds/crsp/sasdata/m_stock62

Engine(5): V9

Physical Name(5): /wrds/crsp/sasdata/m_treasuries

Engine(6): V9

Physical Name(6): /wrds/crsp/sasdata/m_treas_bd

Engine(7): V9

Physical Name(7): /wrds/crsp/sasdata/m_treas_bm

Engine(8): V9

Physical Name(8): /wrds/crsp/sasdata/m_ziman

NOTE: Libref CRSP was successfully assigned as follows:

Levels: 9

Engine(1): V9

Physical Name(1): /wrds/crsp/sasdata/a_stock

Engine(2): V9

Physical Name(2): /wrds/crsp/sasdata/a_stock62

Engine(3): V9

Physical Name(3): /wrds/crsp/sasdata/a_indexes

Engine(4): V9

Physical Name(4): /wrds/crsp/sasdata/a_ccm

Engine(5): V9

Physical Name(5): /wrds/crsp/sasdata/a_treasuries

Engine(6): V9

Physical Name(6): /wrds/crsp/sasdata/a_treas_bd

Engine(7): V9

Physical Name(7): /wrds/crsp/sasdata/a_treas_bm

Engine(8): V9

Physical Name(8): /wrds/crsp/sasdata/a_ziman

Engine(9): V9

Physical Name(9): /wrds/crsp/sasdata/q_mutualfunds

NOTE: Libref CRSPSAMP was successfully assigned as follows:

Engine: V9

Physical Name: /wrds/crspsamp/sasdata

NOTE: Libref CSMAR was successfully assigned as follows:

Levels: 3

Engine(1): V9

Physical Name(1): /wrds/csmar/sasdata/trading

Engine(2): V9

Physical Name(2): /wrds/csmar/sasdata/financial

Engine(3): V9

Physical Name(3): /wrds/csmar/sasdata/funds_mkt_research

NOTE: Libref CUSIPM was successfully assigned as follows:

Engine: V9

Physical Name: /wrds/cusip/sasdata

NOTE: Libref DEALSCAN was successfully assigned as follows:

Engine: V9

Physical Name: /wrds/tfn/sasdata/dealscan

NOTE: Libref DJONES was successfully assigned as follows:

Engine: V9

Physical Name: /wrds/djones/sasdata

NOTE: Libref DMEF was successfully assigned as follows:

Engine: V9

Physical Name: /wrds/dmef/sasdata

NOTE: Libref DOE was successfully assigned as follows:

Engine: V9

Physical Name: /wrds/doe/sasdata

NOTE: Libref EMDB was successfully assigned as follows:

Levels: 2

Engine(1): V9

Physical Name(1): /wrds/comp/sasdata/emdbd

Engine(2): V9

Physical Name(2): /wrds/comp/sasdata/emdbm

NOTE: Libref FACTSET was successfully assigned as follows:

Levels: 4

Engine(1): V9

Physical Name(1): /wrds/factset/sasdata/ff_usc

Engine(2): V9

Physical Name(2): /wrds/factset/sasdata/ff_int

Engine(3): V9

Physical Name(3): /wrds/factset/sasdata/own

Engine(4): V9

Physical Name(4): /wrds/factset/sasdata/common

NOTE: Libref FSSAMP was successfully assigned as follows:

Engine: V9

Physical Name: /wrds/samples/sasdata/factset

NOTE: Libref FDIC was successfully assigned as follows:

Engine: V9

Physical Name: /wrds/fdic/sasdata

NOTE: Libref FF was successfully assigned as follows:

Engine: V9

Physical Name: /wrds/ff/sasdata

NOTE: Libref FISD was successfully assigned as follows:

Levels: 2

Engine(1): V9

Physical Name(1): /wrds/mergent/sasdata/fisd

Engine(2): V9

Physical Name(2): /wrds/mergent/sasdata/naic

NOTE: Libref FISDSAMP was successfully assigned as follows:

Engine: V9

Physical Name: /wrds/fisdsamp/sasdata

NOTE: Libref FRB was successfully assigned as follows:

Engine: V9

Physical Name: /wrds/frb/sasdata

NOTE: Libref GINSIGHT was successfully assigned as follows:

Levels: 3

Engine(1): V9

Physical Name(1): /wrds/ginsight/sasdata/all

Engine(2): V9

Physical Name(2): /wrds/ginsight/sasdata/basic

Engine(3): V9

Physical Name(3): /wrds/ginsight/sasdata/common

NOTE: Libref GMI was successfully assigned as follows:

Engine: V9

Physical Name: /wrds/gmiratings/sasdata

NOTE: Libref GSI was successfully assigned as follows:

Engine: V9

Physical Name: /wrds/gsi/sasdata

NOTE: Libref GOVPX was successfully assigned as follows:

Engine: V9

Physical Name: /wrds/govpx/sasdata

NOTE: Libref HFR was successfully assigned as follows:

Engine: V9

Physical Name: /wrds/hfr/sasdata/hfrdb

NOTE: Libref HFRSAMP was successfully assigned as follows:

Engine: V9

Physical Name: /wrds/hfrsamp/sasdata/hfrdb

NOTE: Libref IBES was successfully assigned as follows:

Levels: 3

Engine(1): V9

Physical Name(1): /wrds/ibes/sasdata

Engine(2): V9

Physical Name(2): /wrds/ibes/sasdata/iga

Engine(3): V9

Physical Name(3): /wrds/ibes/sasdata/guidance

NOTE: Libref IRI was successfully assigned as follows:

Engine: V9

Physical Name: /wrds/iri/sasdata

NOTE: Libref ISSM was successfully assigned as follows:

Engine: V9

Physical Name: /wrds/issm/sasdata

NOTE: Libref KLD was successfully assigned as follows:

Engine: V9

Physical Name: /wrds/kld/sasdata

NOTE: Libref LSPD was successfully assigned as follows:

Levels: 2

Engine(1): V9

Physical Name(1): /wrds/lspd/sasdata/monthly

Engine(2): V9

Physical Name(2): /wrds/lspd/sasdata/daily

NOTE: Libref LSPDSAMP was successfully assigned as follows:

Engine: V9

Physical Name: /wrds/samples/sasdata/lspd

NOTE: Libref MARKIT was successfully assigned as follows:

Levels: 10

Engine(1): V9

Physical Name(1): /wrds/markit/sasdata/cds

Engine(2): V9

Physical Name(2): /wrds/markit/sasdata/red

Engine(3): V9

Physical Name(3): /wrds/markit/sasdata/cdx_composites

Engine(4): V9

Physical Name(4): /wrds/markit/sasdata/cdx_constituents

Engine(5): V9

Physical Name(5): /wrds/markit/sasdata/itraxx_asia_composites

Engine(6): V9

Physical Name(6): /wrds/markit/sasdata/itraxx_europe_composites

Engine(7): V9

Physical Name(7): /wrds/markit/sasdata/itraxx_sovx_composites

Engine(8): V9

Physical Name(8): /wrds/markit/sasdata/itraxx_asia_constituents

Engine(9): V9

Physical Name(9): /wrds/markit/sasdata/itraxx_europe_constituents

Engine(10): V9

Physical Name(10): /wrds/markit/sasdata/itraxx_sovx_constituents

NOTE: Libref MSFANLY was successfully assigned as follows:

Levels: 5

Engine(1): V9

Physical Name(1): /wrds/markit/sasdata/msf_analytics_bonds

Engine(2): V9

Physical Name(2): /wrds/markit/sasdata/msf_analytics_eqty_amer

Engine(3): V9

Physical Name(3): /wrds/markit/sasdata/msf_analytics_eqty_euro

Engine(4): V9

Physical Name(4): /wrds/markit/sasdata/msf_analytics_eqty_asia

Engine(5): V9

Physical Name(5): /wrds/markit/sasdata/msf_analytics_eqty_other

NOTE: Libref MSFINST was successfully assigned as follows:

Levels: 5

Engine(1): V9

Physical Name(1): /wrds/markit/sasdata/msf_inst_bonds

Engine(2): V9

Physical Name(2): /wrds/markit/sasdata/msf_inst_eqty_amer

Engine(3): V9

Physical Name(3): /wrds/markit/sasdata/msf_inst_eqty_asia

Engine(4): V9

Physical Name(4): /wrds/markit/sasdata/msf_inst_eqty_euro

Engine(5): V9

Physical Name(5): /wrds/markit/sasdata/msf_inst_eqty_other

NOTE: Libref MRKTSAMP was successfully assigned as follows:

Engine: V9

Physical Name: /wrds/samples/sasdata/markit/cdsred

NOTE: Libref MFL was successfully assigned as follows:

Engine: V9

Physical Name: /wrds/mfl/sasdata

NOTE: Libref OMTRIAL was successfully assigned as follows:

Levels: 2

Engine(1): V9

Physical Name(1): /wrds/samples/sasdata/optionmetrics

Engine(2): V9

Physical Name(2): /wrds/samples/sasdata/optionmetrics/europe

NOTE: Libref OPTIONM was successfully assigned as follows:

Levels: 2

Engine(1): V9

Physical Name(1): /wrds/optionm/sasdata

Engine(2): V9

Physical Name(2): /wrds/optionm/sasdata/europe

NOTE: Libref OTC was successfully assigned as follows:

Engine: V9

Physical Name: /wrds/otc/sasdata/endofday

NOTE: Libref PACAP was successfully assigned as follows:

Levels: 8

Engine(1): V9

Physical Name(1): /wrds/pacap/sasdata/hk

Engine(2): V9

Physical Name(2): /wrds/pacap/sasdata/id

Engine(3): V9

Physical Name(3): /wrds/pacap/sasdata/ja

Engine(4): V9

Physical Name(4): /wrds/pacap/sasdata/ks

Engine(5): V9

Physical Name(5): /wrds/pacap/sasdata/my

Engine(6): V9

Physical Name(6): /wrds/pacap/sasdata/sn

Engine(7): V9

Physical Name(7): /wrds/pacap/sasdata/tw

Engine(8): V9

Physical Name(8): /wrds/pacap/sasdata/th

NOTE: Libref PHLX was successfully assigned as follows:

Engine: V9

Physical Name: /wrds/phlx/sasdata

NOTE: Libref PUBLIC was successfully assigned as follows:

Levels: 4

Engine(1): V9

Physical Name(1): /wrds/public/sasdata/healthcare/hcup

Engine(2): V9

Physical Name(2): /wrds/public/sasdata/healthcare/meps

Engine(3): V9

Physical Name(3): /wrds/public/sasdata/macroecon/bea

Engine(4): V9

Physical Name(4): /wrds/public/sasdata/macroecon/bls

NOTE: Libref PWT was successfully assigned as follows:

Engine: V9

Physical Name: /wrds/pwt/sasdata

NOTE: Libref RENT was successfully assigned as follows:

Engine: V9

Physical Name: /wrds/rent/sasdata

NOTE: Libref RISK was successfully assigned as follows:

Levels: 4

Engine(1): V9

Physical Name(1): /wrds/riskmetrics/sasdata/directors

Engine(2): V9

Physical Name(2): /wrds/riskmetrics/sasdata/governance

Engine(3): V9

Physical Name(3): /wrds/riskmetrics/sasdata/proposals

Engine(4): V9

Physical Name(4): /wrds/riskmetrics/sasdata/votes

NOTE: Libref SECSAMP was successfully assigned as follows:

Engine: V9

Physical Name: /wrds/samples/sasdata/sec

NOTE: Libref SPRAT was successfully assigned as follows:

Engine: V9

Physical Name: /wrds/comp/sasdata/sprat

NOTE: Libref TAQ was successfully assigned as follows:

Engine: V9

Physical Name: /wrds/taq/sasdata

NOTE: Libref TAQSAMP was successfully assigned as follows:

Engine: V9

Physical Name: /wrds/taqsamp/sasdata

NOTE: Libref TFN was successfully assigned as follows:

Levels: 3

Engine(1): V9

Physical Name(1): /wrds/tfn/sasdata/s12

Engine(2): V9

Physical Name(2): /wrds/tfn/sasdata/s34

Engine(3): V9

Physical Name(3): /wrds/tfn/sasdata/insiders

NOTE: Libref TRACE was successfully assigned as follows:

Levels: 2

Engine(1): V9

Physical Name(1): /wrds/trace/sasdata/standard

Engine(2): V9

Physical Name(2): /wrds/trace/sasdata/enhanced

NOTE: Libref TROWN was successfully assigned as follows:

Engine: V9

Physical Name: /wrds/tfn/sasdata/ownership

NOTE: Libref ZACKS was successfully assigned as follows:

Engine: V9

Physical Name: /wrds/zacks/sasdata

NOTE: Libref ZACKSAMP was successfully assigned as follows:

Engine: V9

Physical Name: /wrds/samples/sasdata/zacks

NOTE: AUTOEXEC processing completed.

1 filename writer zip "~/tmp/fadcda3cb_9ccb_4822_ba4b_01b465f3fb67.zip"

1 ! member="CRSPA.DSF.csv";

2 proc sql noprint;

3 select '"'||trim(name)||'"'

4 into :names

5 separated by "','"

6 from dictionary.columns

7 where libname eq "CRSPA" and memname eq "DSF";

8 quit;

NOTE: PROCEDURE SQL used (Total process time):

real time 0.09 seconds

cpu time 0.04 seconds

------ Original Message ------ From: "Oleg Komarov" notifications@github.com To: "okomarov/wrds" wrds@noreply.github.com Cc: "MrJensen1" allangjerloev@gmail.com Sent: 06-03-2016 14:07:01 Subject: Re: [wrds] Clarifications on the use of this API (#8)

Try the following steps:

w.getDatasetNames('CRSPA') check that MSI is in the output. If not, or if it comes out empty, it means you (or the institution) has no access to the Annual CRSP subscription (CRSPA). And I don't how you downloaded it in the first place earlier (maybe check again you are using the right account).

If you see MSI, then run the following again:

w.getDataset('CRSPA.MSI') and after the error:

w.getFile('tmp/report.log'); Then, please paste the full content of the report.log that you should find under you wrds/data folder.

— Reply to this email directly or view it on GitHub.

okomarov commented 8 years ago

Your report has some anomalies, it says:

NOTE: AUTOEXEC processing beginning; file is /home/XX/USERNAME/autoexec.sas.

where XX/USERNAME/ should be your specific institution and username. You might have issues with your account, where I cannot help.

Then, you are trying to download CRSPA.DSF and not CRSPA.MSI as I suggested. Notice that DSF is

File Size (bytes)           14226817024

approximately 14GB in size. That is gonna take a looooong time to compress and download. In fact, you report does not show that the step completed.

Please check with getDatasetInfo the size of the datasets you are downloading, before doing so.

okomarov commented 8 years ago

@MrJensen1 I have modified the wrds() to connect by default to the cloud servers. You can try again with a newly downloaded version of wrds() and see if that helps.

Still, bear in mind that big files will take a while to process.

I will be closing this issue which you can reopen if you have additional queries.

okomarov / wrds

Clarifications on the use of this API #8