search

rolozanod / googlefinance-sheets

Repo to retrieve historical financial data from Google Finance using a Google Cloud Computing account connection to the Sheets app.
MIT License
5 stars 2 forks source link

Unable to load Stock #3

Open mpainenz opened 2 years ago

mpainenz commented 2 years ago

Hi there,

I am facing an exception when running the test.py file which comes with the project.

Exception has occurred: BadRequest
400 Error while reading table: stocks, error message: CSV table references column position 7, but line starting at position:40 contains only 1 columns.

Location: US
Job ID: 73f41e76-e30d-45a3-b7fb-922682639401
  File "C:\Solus\googlefinance-sheets\gfs\google_finance.py", line 700, in gs_stocks
    stocks = list(query_job)  # Wait for the job to complete.
  File "C:\Solus\googlefinance-sheets\gfs\google_finance.py", line 827, in retrieve_stocks
    stocks = gs_stocks(path2json_service=path2json_service, bucket_name=bucket_name, table_id=table_id, fields="*", conds=conds)
  File "C:\Solus\googlefinance-sheets\test.py", line 21, in <module>
    stocks = google_finance.retrieve_stocks(

I can see that the Sheet is created in my Google Drive, and currently it is an empty sheet. I can't work out what may be going wrong here, do you have any suggestions?

Thanks

rolozanod commented 2 years ago

@mpainenz, sorry for the late response. COVID finally got me.

Yes, few questions/ideas. The sheet was created so permissions should be ok. Filling the sheet might be the problem because it tries to read data but finds nothing at all. Let see if it is pointing to the right file in the drive.

  1. The script empties the Google sheet before dumping new data. If the API fails to fill the sheet, you will find it empty. Can you see if the sheet history displays any activity please.
  2. The error might come from versions differences. Which python version are you using? I will replicate you stack, saw in the other issue you are on windows. Can you paste your requirements please?
mpainenz commented 2 years ago

Hi @rolozanod

Thanks for getting back to me, and sorry to hear about the COVID.

I'm on Python 3.10.4

My requirements output is quite long from pip freeze, but the ones that I have which are in the requirements.txt file from this project, match exactly the requirements.txt file from the project :

pandas==1.4.2 google-api-core==2.7.1 google-api-python-client==2.43.0 google-auth==2.6.0 google-auth-httplib2==0.1.0 google-auth-oauthlib==0.4.6 google-cloud-bigquery==3.0.1 google-cloud-bigquery-storage==2.13.1 google-cloud-core==2.2.3 google-cloud-storage==2.2.1 google-crc32c==1.3.0 google-pasta==0.2.0 google-resumable-media==2.3.2 googleapis-common-protos==1.56.0

In the sheet history, there is nothing other than the initial creation of the Sheet, and any manual troubleshooting changes I've made.

Would the terraform_setup function initially create data in the sheet? I assume not, but I did have problems with the initial terraform (as per the other issues), and did find that I had come Google objects missing (which I had to manually build). I wouldn't be suprised if I might be missing objects in Cloud. For some reason Terraform didn't create any Tables, but the Data set was there.

Could you perhaps point me to where to look at the part which writes to the Google Sheet initially? I'm still trying to understand how the data flows from Sheet, to Bigquery etc.

Thanks again, Mark

rolozanod commented 2 years ago

Hi @mpainenz, just uploaded a version with documented manual instructions. Under the README file there are two options for the GCP setup. Please follow the manual setup instructions. It worked for me, please let me know anything you need. Sorry for the delay.