basedosdados / sdk

⚙️ Código de manutenção do datalake (metadados e pacotes de acesso) | 📖 Docs: https://basedosdados.github.io/mais/
https://info.basedosdados.org/links
MIT License
394 stars 88 forks source link

[infra] Erro em table_create- Python #996

Closed gustavoairestiago closed 10 months ago

gustavoairestiago commented 3 years ago

Descrição

Passos para reproduzir o comportamento:

  1. Executei
    
    wb = bd.Table(dataset_id='mundo_wb_learning_poverty', table_id='country')

sobe dado bigquery

wb.create(r'G:\Meu Drive\Base dos Dados\mundo_wb_learning_poverty\learningpoverty.csv', if_table_exists='replace', if_storage_data_exists='replace', if_table_config_exists='pass')

2. Traceback:

TypeError Traceback (most recent call last)

in 11 #sobe dado bigquery 12 ---> 13 wb.create(r'G:\Meu Drive\Base dos Dados\mundo_wb_learning_poverty\learningpoverty.csv', 14 if_table_exists='replace', 15 if_storage_data_exists='replace', ~\AppData\Roaming\Python\Python38\site-packages\basedosdados\upload\table.py in create(self, path, job_config_params, force_dataset, if_table_exists, if_storage_data_exists, if_table_config_exists, source_format, columns_config_url) 630 pass 631 --> 632 dataset_obj.create(if_exists="pass") 633 634 self.init( ~\AppData\Roaming\Python\Python38\site-packages\basedosdados\upload\dataset.py in create(self, mode, if_exists) 173 174 # Construct a full Dataset object to send to the API. --> 175 dataset_obj = self._setup_dataset_object(m["id"]) 176 177 # Send the dataset to the API for creation, with an explicit timeout. ~\AppData\Roaming\Python\Python38\site-packages\basedosdados\upload\dataset.py in _setup_dataset_object(self, dataset_id) 43 44 dataset = bigquery.Dataset(dataset_id) ---> 45 dataset.description = self._render_template( 46 Path("dataset/dataset_description.txt"), self.dataset_config 47 ) ~\AppData\Roaming\Python\Python38\site-packages\basedosdados\upload\base.py in _render_template(self, template_file, kargs) 316 def _render_template(self, template_file, kargs): 317 --> 318 return Template( 319 (self.templates / template_file).open("r", encoding="utf-8").read() 320 ).render(**kargs) TypeError: render() argument after ** must be a mapping, not NoneType ``` 3. A base está em : [https://console.cloud.google.com/storage/browser/basedosdados-dev/staging/mundo_wb_learning_poverty/country?](link da base no storage) 4. A base foi criada no storage mas não no staging do BigQuery
vmussa commented 3 years ago

adicionei o traceback reportado no Discord para entendermos melhor. Obg pelo report @gustavoairestiago!

lucasnascm commented 2 years ago
  1. Executei:
    dic = bd.Table(dataset_id='br_inep_saeb', table_id='dicionario')
    dic.create('E:/Users/Administrador/Documents/dicionario_saeb.csv',
            if_table_exists='replace', 
            if_storage_data_exists='replace', 
            if_table_config_exists='replace')
  2. Traceback:
    
    TypeError                                 Traceback (most recent call last)
    <ipython-input-6-3819610c30d7> in <module>
      1 dic = bd.Table(dataset_id='br_inep_saeb', table_id='dicionario')
    ----> 2 dic.create('E:/Users/Administrador/Documents/dicionario_saeb.csv',
      3             if_table_exists='replace',
      4             if_storage_data_exists='replace',
      5             if_table_config_exists='replace')

c:\users\nascm\appdata\local\programs\python\python39\lib\site-packages\basedosdados\upload\table.py in create(self, path, job_config_params, force_dataset, if_table_exists, if_storage_data_exists, if_table_config_exists, source_format, columns_config_url) 630 pass 631 --> 632 dataset_obj.create(if_exists="pass") 633 634 self.init(

c:\users\nascm\appdata\local\programs\python\python39\lib\site-packages\basedosdados\upload\dataset.py in create(self, mode, if_exists) 173 174 # Construct a full Dataset object to send to the API. --> 175 dataset_obj = self._setup_dataset_object(m["id"]) 176 177 # Send the dataset to the API for creation, with an explicit timeout.

c:\users\nascm\appdata\local\programs\python\python39\lib\site-packages\basedosdados\upload\dataset.py in _setup_dataset_object(self, dataset_id) 43 44 dataset = bigquery.Dataset(dataset_id) ---> 45 dataset.description = self._render_template( 46 Path("dataset/dataset_description.txt"), self.dataset_config 47 )

c:\users\nascm\appdata\local\programs\python\python39\lib\site-packages\basedosdados\upload\base.py in _render_template(self, template_file, kargs) 316 def _render_template(self, template_file, kargs): 317 --> 318 return Template( 319 (self.templates / template_file).open("r", encoding="utf-8").read() 320 ).render(**kargs)

TypeError: jinja2.environment.Template.render() argument after ** must be a mapping, not NoneType


3. Cria em `basedosdados-dev`, mas não cria nada na máquina local