bertrandmartel / tableau-scraping

Tableau scraper python library. R and Python scripts to scrape data from Tableau viz
MIT License
126 stars 20 forks source link

Can't filter using dropdown #29

Closed khrusco closed 2 years ago

khrusco commented 2 years ago

Hi! First of all, I'm really impressed about the lib, such a great job!

Could you help me with this code? I'm trying get data using some filters, but it seems the code can't read the dropdown. I already used the force_setFilter function mentioned in another issue, but I stil can't get the data from the dashboard

from tableauscraper import TableauScraper as TS

url = "https://tableaupub.ccee.org.br/t/PIDM/views/IndicadoresdeSeguranadoMercado/ConcentraodeNegociao"

ts = TS()
ts.loads(url)
workbook = ts.getWorkbook()

#the code returns the parameter values used below
workbook.getParameters()

workbook.setParameter('parDimensão', 'CNPJ') #select "CNPJ" option

#this code returns the name of worksheet used below
#workbook.getWorksheetNames()

ws = ts.getWorksheet('Hist Concentração de Negociação (2)') #reaching dashboard data

#by using this, i got the the filters, previously 'Agente', but now 'CNPJ'
#as mentioned on setParameters
#ws.getFilters()

#the code return the data without any filter previously applied
ws.setFilter('SG_PERF_AGEN (Consulta_SQL_personalizada)','00.095.840/0001-85') #can't use the dropdown filter
ws.data

tnks in advance :)

bertrandmartel commented 2 years ago

@khrusco Hello! When calling setParameter or setFilter, it returns a workbook class that needs to be used for further call since it persists the session.

Also, it seems the setFilter API call used in your Tableau dashboard doesn't accept the parameter membershipTarget here

I will add a boolean param to configure this:

from tableauscraper import TableauScraper as TS

url = "https://tableaupub.ccee.org.br/t/PIDM/views/IndicadoresdeSeguranadoMercado/ConcentraodeNegociao"

ts = TS()
ts.loads(url)
workbook = ts.getWorkbook()

# the code returns the parameter values used below
print(workbook.getParameters())

workbook = workbook.setParameter('parDimensão', 'CNPJ')  # select "CNPJ" option

ws = workbook.getWorksheet('Classe Contraparte')
# by using this, i got the the filters, previously 'Agente', but now 'CNPJ'
# as mentioned on setParameters
print(ws.getFilters())

# the code return the data without any filter previously applied
# can't use the dropdown filter
workbook = ws.setFilter(
    'Filtro Dimensão', '00.095.840/0001-85', membershipTarget=False)
print(workbook.getWorksheet('Hist Concentração de Negociação (2)').data)
bertrandmartel commented 2 years ago

@khrusco released in v0.1.20

khrusco commented 2 years ago

You are greaaat! tnks :D

anapaulazampier commented 1 year ago

Hii @bertrandmartel , first of all thank you so much for your work, it's amazing! I'm facing issues on the same dashboard as @khrusco. When calling getFilters() it only returns the first 200 values from the value field and I would like to create a database with all the "agentes". Here's my code:

import pandas as pd
from tableauscraper import TableauScraper as TS

url = "https://tableaupub.ccee.org.br/t/PIDM/views/IndicadoresdeSeguranadoMercado/ConcentraodeNegociao"

ts = TS()
ts.loads(url)
workbook = ts.getWorkbook()

print("parametros:")
print(workbook.getParameters())
print("------------------------")

workbook = workbook.setParameter('parDimensão', 'Agente')
ws = workbook.getWorksheet('Classe Contraparte')

print("filtros:")
print(ws.getFilters())
print("------------------------")

filtros = ws.getFilters()
agentes = filtros[0]['values']

appended_data = pd.DataFrame()
for x in agentes:
    print(x)
    wb = ws.setFilter('Filtro Dimensão', x , membershipTarget=False, filterDelta=True, dashboardFilter=True)
    df_aux = wb.getWorksheet('Hist Concentração de Negociação (2)').data
    df_aux['agente'] = x
    appended_data = appended_data.append(df_aux)

Thanks in advance for you help! :)