Create subpackage for functionality reliant on tableauhyperapi.

[jaybythebay]

Background

This is a fix for https://github.com/hoverinc/tableau-utilities/issues/75. On newer apple computers installation fails because tableauhyperapi is not compatible with Apple Silicon machines: https://tableau.github.io/hyper-db/docs/installation#hardware-requirements

This PR:

• Changes hyperapi dependent functionality into a optionally installed subpackage
• Updates the cli and Airflow example to use new subpackage
• Change short cli commands to be all lowercase and multiple letters. This conforms to the style in the rest of the cli.
• Adds new attribute of max_varchar_size to Connection object. This appears in a Snowflake extract with Tableau 2024.1
• Adds Python Version specification as Pandas installation dependency will fail if the user has Python 3.7

Testing

Installation

pip install ./ this installed the tableau_utilities but did not install the tableauhyperapi dependency pip install ./[hyper] this DID install the tableauhyperapi dependency

CLI Commands

cli throws error when package is not installed

-ee flag

tableau_utilities --token_name $TABLEAU_PAT_NAME --token_secret $TABLEAU_PAT_VALUE --site $TABLEAU_SITE_NAME --server $TABLEAU_SERVER --location online --name call_center --project_name 'default' -f Datasource.tdsx datasource -ee

usage: tableau_utilities [-h] [--version] [-d] [-s SERVER] [-sn SITE_NAME] [--api_version API_VERSION] [-u USER] [-p PASSWORD] [-ts TOKEN_SECRET] [-tn TOKEN_NAME]
                         [--settings_path SETTINGS_PATH] [-o OUTPUT_DIR] [-c] [-l {local,online}] [-i ID] [-n NAME] [-pn PROJECT_NAME] [-f FILE_PATH]
                         [--definitions_csv DEFINITIONS_CSV] [--include_extract] [-tds] [--conn_user CONN_USER] [--conn_pw CONN_PW] [--conn_type CONN_TYPE]
                         [--conn_db CONN_DB] [--conn_schema CONN_SCHEMA] [--conn_host CONN_HOST] [--conn_role CONN_ROLE] [--conn_warehouse CONN_WAREHOUSE]
                         {server_info,server_operate,datasource,generate_config,csv_config,merge_config} ...
tableau_utilities: error: --filter_extract and --empty_extract require the tableau_utilities[hyper] subpackage.  See installation notes if you are on an Apple Silicon (Apple M1, Apple M2, ...)

-fe flag

tableau_utilities --token_name $TABLEAU_PAT_NAME --token_secret $TABLEAU_PAT_VALUE --site $TABLEAU_SITE_NAME --server $TABLEAU_SERVER --location online --name call_center --project_name 'default' -f Datasource.tdsx datasource -fe 'Cc Class = medium'

usage: tableau_utilities [-h] [--version] [-d] [-s SERVER] [-sn SITE_NAME] [--api_version API_VERSION] [-u USER] [-p PASSWORD] [-ts TOKEN_SECRET] [-tn TOKEN_NAME]
                         [--settings_path SETTINGS_PATH] [-o OUTPUT_DIR] [-c] [-l {local,online}] [-i ID] [-n NAME] [-pn PROJECT_NAME] [-f FILE_PATH]
                         [--definitions_csv DEFINITIONS_CSV] [--include_extract] [-tds] [--conn_user CONN_USER] [--conn_pw CONN_PW] [--conn_type CONN_TYPE]
                         [--conn_db CONN_DB] [--conn_schema CONN_SCHEMA] [--conn_host CONN_HOST] [--conn_role CONN_ROLE] [--conn_warehouse CONN_WAREHOUSE]
                         {server_info,server_operate,datasource,generate_config,csv_config,merge_config} ...
tableau_utilities: error: --filter_extract and --empty_extract require the tableau_utilities[hyper] subpackage.  See installation notes if you are on an Apple Silicon (Apple M1, Apple M2, ...)

Testing Functions WITH sub-page installed

Download empty extract

tableau_utilities --token_name $TABLEAU_PAT_NAME --token_secret $TABLEAU_PAT_VALUE --site $TABLEAU_SITE_NAME --server $TABLEAU_SERVER  --location online --name call_center --project_name 'default' -f Datasource.tdsx datasource -ee
...Downloading call_center...
✅  (Done in 1 sec) Downloaded Datasource: /Users/jayrosenthal/code/tableau-utilities/tmp_tdsx_and_config/call_center.tdsx

Added empty .hyper extract for /Users/jayrosenthal/code/tableau-utilities/tmp_tdsx_and_config/call_center.tdsx
...Saving datasource changes...
✅ (Done in 0 sec) Saved datasource changes: /Users/jayrosenthal/code/tableau-utilities/tmp_tdsx_and_config/call_center.tdsx

Publish Empty Extract

tableau_utilities --token_name $TABLEAU_PAT_NAME --token_secret $TABLEAU_PAT_VALUE --site $TABLEAU_SITE_NAME --server $TABLEAU_SERVER --name call_center --project_name 'default' -f '/Users/jayrosenthal/code/tableau-utilities/tmp_tdsx_and_config/call_center.tdsx' server_operate --publish datasource PUBLISHING DATASOURCE → ID: cda42c94-d287-4ab2-9935-225ceb3a7d6a ║ NAME: call_center ║ PROJECT NAME: default ✅ default / call_center:

Workbook can be created but the data is empty

Download with Filter

tableau_utilities --token_name $TABLEAU_PAT_NAME --token_secret $TABLEAU_PAT_VALUE --site $TABLEAU_SITE_NAME --server $TABLEAU_SERVER --location online --name call_center --project_name 'default' -f Datasource.tdsx --include_extract datasource -fe "\"CC_REC_START_DATE\" < '1/1/2001'" ...Downloading call_center... ✅ (Done in 1 sec) Downloaded Datasource: /Users/jayrosenthal/code/tableau-utilities/tmp_tdsx_and_config/call_center.tdsx

...Filtering extract data... ✅ (Done in 0 sec) Filtered extract data for /Users/jayrosenthal/code/tableau-utilities/tmp_tdsx_and_config/call_center.tdsx (venv) jaysmacbookhome:tableau-utilities jayrosenthal$

Publish filtered datasets

tableau_utilities --token_name $TABLEAU_PAT_NAME --token_secret $TABLEAU_PAT_VALUE --site $TABLEAU_SITE_NAME --server $TABLEAU_SERVER --name call_center --project_name 'default' -f '/Users/jayrosenthal/code/tableau-utilities/tmp_tdsx_and_config/call_center.tdsx' server_operate --publish datasource PUBLISHING DATASOURCE → ID: cda42c94-d287-4ab2-9935-225ceb3a7d6a ║ NAME: call_center ║ PROJECT NAME: default ✅ default / call_center:

Pre-filtered set has data in 1998 and 2000

Publish after filter on has data in 2001 and 2002

[jaybythebay]

@JustinGrilli this is ready for review. I tested all functionality in the cli and it works like a charm! Replacing the functions in Airflow and setting the new import should be really really easy.

If you're willing to merge this without testing Airflow that would be great. I have a couple of things to do before I can get Airflow working with extracts and all that stuff. If I'm a total insomniac this weekend then I'll try to get to it otherwise it'll be later this week.