bertrandmartel / tableau-scraping

Tableau scraper python library. R and Python scripts to scrape data from Tableau viz
MIT License
126 stars 20 forks source link

Multiple parameters in single call? #32

Closed paulrxn007 closed 2 years ago

paulrxn007 commented 2 years ago

Hello

I have not found in .getParameters() if there is a way to do multiple parameters. Is this something in your code that you haven't been able to do, or is it a Tableau issue?

I am trying to extra Ohio's Covid dashboard info and use the multiple parameters from the GUI, but have not figured out how to do it with your scraper.

Any help would be appreciated!

bertrandmartel commented 2 years ago

@paulrxn007 This library reproduces the same behaviour as in the UI, you can set one parameter after another like in the UI using consecutive setParameter.

For Ohio Covid dashboard I have an example to get every county data https://replit.com/@bertrandmartel/TableauCovidOhio using setParameter

paulrxn007 commented 2 years ago

hey @bertrandmartel

thanks for replying so fast and for the tip of consecutive setParameter.

What I am seeing now is this error from the counties loop...

Traceback (most recent call last): File "main.py", line 46, in workbook = maptotal.select("County ", county) File "/opt/virtualenvs/python3/lib/python3.8/site-packages/tableauscraper/TableauWorksheet.py", line 227, in select self.updateFullData(r) File "/opt/virtualenvs/python3/lib/python3.8/site-packages/tableauscraper/TableauWorksheet.py", line 39, in updateFullData dataSegments = presModel["dataDictionary"]["dataSegments"] KeyError: 'dataDictionary'

How do I figure out these dictionary levels? I always feel like I'm missing something

bertrandmartel commented 2 years ago

@paulrxn007 what is your code ?

paulrxn007 commented 2 years ago
from tableauscraper import TableauScraper as TS
import pandas as pd

url = "https://public.tableau.com/views/OverviewDashboard_15852499073250/DashboardOverview_1"

ts = TS()
ts.loads(url)

workbook = ts.getWorkbook()
searchfilter = ts.getWorksheet("Search Filter")
maptotal = ts.getWorksheet("County Map | Total")
linetotal = ts.getWorksheet("Line | Total Cases")
sideside = ts.getWorksheet("Side By Side Comparison")

pd.set_option('display.max_columns', None)
pd.set_option('display.max_rows', None)

parameters = workbook.getParameters()
print(parameters)

counties = [
  t["values"] for t in maptotal.getSelectableItems()
  if t["column"] == "County " ][0]

print(counties)

# set parameters column / value
workbook.setParameter("Parameter - Case Type", "Probable")
#workbook.setParameter("Parameter - Cumulative","Daily Count")
#workbook.setParameter("Parameter County","County of Residence")
workbook.setParameter("Parameter - Start Date",'2021-09-17')

for county in counties:
    print(county)
    workbook = maptotal.select("County ", county)
    print(workbook.data)
bertrandmartel commented 2 years ago

@paulrxn007 The output of setParameter is a new workbook. In order to persist the session with multiple call, you can reuse the workbook variable like this workbook = workbook.setParameter(...).

It seems that the date uses a format DD/MM/YYYY not sure why

The following sets the parameters and retrieves each county for the resulted workbook:

from tableauscraper import TableauScraper as TS
import pandas as pd

url = "https://public.tableau.com/views/OverviewDashboard_15852499073250/DashboardOverview_1"

ts = TS()
ts.loads(url)

workbook = ts.getWorkbook()
maptotal = workbook.getWorksheet("County Map | Total")

pd.set_option('display.max_columns', None)
pd.set_option('display.max_rows', None)

counties = [
    t["values"] for t in maptotal.getSelectableItems()
    if t["column"] == "County "][0]

print(counties)

# set parameters column / value
workbook = workbook.setParameter("Parameter - Case Type", "Probable")
workbook = workbook.setParameter("Parameter - Start Date", '17/09/2021')
maptotal = workbook.getWorksheet("County Map | Total")

for county in counties:
    print(county)
    tempWb = maptotal.select("County ", county)
    ws = tempWb.getWorksheet("Text | Total Cases")
    print(ws.data)

https://replit.com/@bertrandmartel/TableauCovidOhio2

paulrxn007 commented 2 years ago

@bertrandmartel I don't get how it works in your code, but when I copy from yours to mine, no go.

Maybe i need to start one from scratch

paulrxn007 commented 2 years ago

hey there @bertrandmartel Tableau expert,

So, now I am running into an issue with not finding the End Date parameter, but it is there. Any clues about that? I have tried different places for the Parameter, but no change.

_``` [{'column': 'Parameter County', 'values': ['County of Residence', 'County of Death'], 'parameterName': '[Parameters].[Parameter 8]'}, {'column': 'Parameter - Start Date', 'values': [], 'parameterName': '[Parameters].[Parameter 9]'}, {'column': 'Parameter - End Date', 'values': [], 'parameterName': '[Parameters].[Parameter - Start Date (copy)_1926696252560949248]'}, {'column': 'Parameter - Cumulative', 'values': ['Cumulative Count', 'Daily Count'], 'parameterName': '[Parameters].[Parameter 7]'}, {'column': 'Parameter - Comparison', 'values': ['Age', 'County', 'Sex'], 'parameterName': '[Parameters].[Parameter 1]'}, {'column': 'Parameter - Case Type', 'values': ['(All) Total Cases', 'Confirmed', 'Probable'], 'parameterName': '[Parameters].[Parameter 5]'}, {'column': 'Parameter - Map', 'values': ['Cases', 'Deaths', 'Hospitalizations'], 'parameterName': '[Parameters].[Parameter 2]'}] 2021-10-05 13:31:00,768 - tableauScraper - ERROR - column Parameter - End Date not found Wyandot 2021-10-05 13:31:03,153 - tableauScraper - WARNING - no data dictionary present in response AGG(Daily Case Count)-alias 0 72 Wood AGG(Daily Case Count)-alias 0 308 Williams

bertrandmartel commented 2 years ago

@paulrxn007 without your code, I can't understand what's the issue, adding Parameter - End Date parameter in the previous code works for me