Minimize Dataset data fetched from server when performing a MergedDataset([...]).data() call:
If only one column index is requested, pass as query param column_index=x so that the server can give us back that one column
For more than 1 column specified, fetch all the data from the server, and filter out on the client side the columns the user has requested
NOTE: We have to handle multiple column index per Dataset in a MergedDataset on the client side because the API does not support multiple column_indexes for a Dataset data fetch
Don't make metadata calls when calling MergedDataset('[]').data()
Make metadata call only when user asks for an attribute of the MergedDataset, e.g., MergedDataset([...]).name
MergedDataList (which is the class that holds the dataset data response data for a MergedDataset):
instead of getting the start_date and end_date from the metadata, just set these values to what we see in the actual data we got back via the merged dataframe
Validations:
Do not perform client side validation of requested column indexes when performing MergedDataset().data()
Perform the validation after we retrieve the data
This prevents the forced
Analyst Quandl.get([....])
Fail silently when making single dataset calls where the column does not exist (i.e., insert a Not Found empty column in the data frame)
Fail silently when making multiple dataset calls where the column does not exist
Fail silently when dataset does not exist ONLY in multiple dataset scenario
Raise a NotFoundError for single dataset call and dataset does not exist
Fail silently means to insert an empty DataFrame column whenever column(s) fails to be retrieved
Developer flow MergedDataset and Dataset:
Raise a ColumnNotFound when dataset column does not exist
Raise a NotFoundError (as before) when dataset does not exist
mbasset
commented
8 years ago
Tracker Ticket: https://quandl.atlassian.net/browse/AP-1631
Description:
MergedDataset([...]).data()call:column_index=xso that the server can give us back that one columnMergedDataset('[]').data()MergedDataset, e.g.,MergedDataset([...]).namestart_dateandend_datefrom the metadata, just set these values to what we see in the actual data we got back via the merged dataframeQuandl.get([....])NotFoundErrorfor single dataset call and dataset does not existColumnNotFoundwhen dataset column does not existNotFoundError(as before) when dataset does not existNote to the reviewer:
Note to QA: