CartoDB / cartoframes

CARTO Python package for data scientists
BSD 3-Clause "New" or "Revised" License
250 stars 63 forks source link

Programatically subscribe to a dataset/geography #1341

Closed arredond closed 4 years ago

arredond commented 4 years ago

Especially for QA purposes, it'd be nice to be able to programatically subscribe to a dataset/geography without prompting the iframe and having to click Yes. Something like:

my_dataset.subscribe(skip_prompt=True)

Currently, performing QA involves manually subscribing to each dataset and geography, which may mean running the same cell and clicking on Yes dozens of times.

Also, right now we must update the metadata specifically to make subscription work. To keep things safe, @andy-esch wrote a Context Manager to do just this, and then revert the metadata to its original values, but we're having trouble making this work with the iframe prompt.

Jesus89 commented 4 years ago

Hi, there are legal implications adding a mode to skip the confirmation. My proposal is to add a subscription in CLI (y/N) and then if you need to automate, use external tools like pyautogui.

This is an example of a script to subscribe using CLI:

from cartoframes.auth import Credentials
from cartoframes.data.observatory import Catalog, Dataset, Geography

creds = Credentials('...')

dataset_name = 'ags_crimerisk_e9cfa4d4'

dataset = Dataset.get(dataset_name)
geography = Geography.get(dataset.geography)

print(dataset)
dataset.subscribe(creds)

print(geography)
geography.subscribe(creds)

print('Subscriptions:')
subs = Catalog().subscriptions(creds)
print(subs)

Output case n, case y:

$ python test-subscriptions.py <Dataset.get('ags_crimerisk_e9cfa4d4')> Subscription contract: You are about to subscribe to "carto-do.ags.demographics_crimerisk_usa_blockgroup_2015_yearly_2019". The cost of this dataset is $X. If you want to proceed, a Request will be sent to CARTO who will order the data and load it into your account. This dataset is available for Instant Order for your organization, so it will automatically process the order and you will get immediate access to the dataset. In order to proceed, we need you to agree to the License of the dataset available at this link: https://drive.google.com/file/d/1YMnFXjexEG-cCiWLUSo9d-oUGQZWztHh/view?usp=sharing. Do you want to proceed? (y/N): n The dataset "carto-do.ags.demographics_crimerisk_usa_blockgroup_2015_yearly_2019" has not been purchased <Geography.get('usct_blockgroup_f45b6b49')> Subscription already purchased: The geography "carto-do-public-data.usa_carto.geography_usa_blockgroup_2015" has already been purchased. Subscriptions: Datasets: [<Dataset.get('ags_retailpoten_aaf25a8c')>] Geographies: [<Geography.get('usct_blockgroup_f45b6b49')>, <Geography.get('ags_blockgroup_1c63771c')>]

$ python test-subscriptions.py <Dataset.get('ags_crimerisk_e9cfa4d4')> Subscription contract: You are about to subscribe to "carto-do.ags.demographics_crimerisk_usa_blockgroup_2015_yearly_2019". The cost of this dataset is $X. If you want to proceed, a Request will be sent to CARTO who will order the data and load it into your account. This dataset is available for Instant Order for your organization, so it will automatically process the order and you will get immediate access to the dataset. In order to proceed, we need you to agree to the License of the dataset available at this link: https://drive.google.com/file/d/1YMnFXjexEG-cCiWLUSo9d-oUGQZWztHh/view?usp=sharing. Do you want to proceed? (y/N): y Congrats! The dataset "carto-do.ags.demographics_crimerisk_usa_blockgroup_2015_yearly_2019" has been requested and it will be available in your account soon. <Geography.get('usct_blockgroup_f45b6b49')> Subscription already purchased: The geography "carto-do-public-data.usa_carto.geography_usa_blockgroup_2015" has already been purchased. Subscriptions: Datasets: [<Dataset.get('ags_crimerisk_e9cfa4d4')>, <Dataset.get('ags_retailpoten_aaf25a8c')>] Geographies: [<Geography.get('usct_blockgroup_f45b6b49')>, <Geography.get('ags_blockgroup_1c63771c')>]