search

guanquann / Stocksera

Finance application that provides more than 60 different alternative data to retail investors
MIT License
639 stars 102 forks source link

get_latest_insider_trading.pyn error unable to decipher #42

Closed CodeInFilth closed 1 month ago

CodeInFilth commented 1 year ago 
Traceback (most recent call last):
  File "tasks_to_run.py", line 280, in <module>
    get_latest_insider_trading.main()
  File "/code/scheduled_tasks/discover/get_latest_insider_trading.py", line 89, in main
    latest_insider_trading()
  File "/code/scheduled_tasks/discover/get_latest_insider_trading.py", line 27, in latest_insider_trading
    insider_trader = finsider.get_insider()
  File "/usr/local/lib/python3.7/site-packages/finvizfinance/insider.py", line 85, in get_insider
    info_dict["SEC Form 4 Link"] = col.find("a").attrs["href"]
AttributeError: 'NoneType' object has no attribute 'attrs'

I cant figure this one out any help appreciated, i am on a older version 4 commits back mannually updating, so that is the updated code. Could the finvizfinance have any other associated files i need to change?

guanquann commented 1 year ago

You may need to update your library to finnhub-python==2.4.18

guanquann commented 1 year ago

Let me know if it works!

CodeInFilth commented 1 year ago

thanks will update u shortly

CodeInFilth commented 1 year ago

Let me know if it works!

Still getting the same error. 

root@121:/var/www/vhosts/areus.ai/python_apps/filthy-app/deveopment/website/Areus#
docker compose run --rm web python tasks_to_run.py
Trying to connect to database...
Connection successful!
Version 7.2.0 of praw is outdated. Version 7.7.0 was released Saturday February 25, 2023.
LATEST_INSIDER_TRADING
latest buys
Traceback (most recent call last):
  File "tasks_to_run.py", line 280, in <module>
    get_latest_insider_trading.main()
  File "/code/scheduled_tasks/discover/get_latest_insider_trading.py", line 89, in main
    latest_insider_trading()
  File "/code/scheduled_tasks/discover/get_latest_insider_trading.py", line 27, in latest_insider_trading
    insider_trader = finsider.get_insider()
  File "/usr/local/lib/python3.7/site-packages/finvizfinance/insider.py", line 85, in get_insider
    info_dict["SEC Form 4 Link"] = col.find("a").attrs["href"]
AttributeError: 'NoneType' object has no attribute 'attrs'

I also was getting an error on the short volume and was able to fix like this

Error:

Trying to connect to database...
Connection successful!
Version 7.2.0 of praw is outdated. Version 7.7.0 was released Saturday February 25, 2023.
SHORT_VOL
Looking at 2023-04-24
Looking at 2023-04-25
Looking at 2023-04-26
Looking at 2023-04-27
Looking at 2023-04-28
Looking at 2023-04-29
No data for 2023-04-29

Looking at 2023-04-30
No data for 2023-04-30

Looking at 2023-05-01
Looking at 2023-05-02
Looking at 2023-05-03
Looking at 2023-05-04
Looking at 2023-05-05
Looking at 2023-05-06
No data for 2023-05-06

Looking at 2023-05-07
No data for 2023-05-07

Looking at 2023-05-08
Looking at 2023-05-09
Looking at 2023-05-10
Looking at 2023-05-11
Looking at 2023-05-12
Looking at 2023-05-13
No data for 2023-05-13

Looking at 2023-05-14
No data for 2023-05-14

Looking at 2023-05-15
Looking at 2023-05-16
Looking at 2023-05-17
Looking at 2023-05-18
Looking at 2023-05-19
Looking at 2023-05-20
No data for 2023-05-20

Looking at 2023-05-21
No data for 2023-05-21

Looking at 2023-05-22
Looking at 2023-05-23
Looking at 2023-05-24
No data for 2023-05-24

            Date Ticker  Short Vol  Short Exempt Vol  Total Vol  % Shorted  nan
0     2023-04-24      A   126036.0              53.0   220284.0      57.22  NaN
1     2023-04-24     AA   664042.0            2813.0  1927503.0      34.45  NaN
2     2023-04-24   AAAU   956616.0              59.0  1972984.0      48.49  NaN
3     2023-04-24    AAC     2391.0              35.0     6703.0      35.67  NaN
4     2023-04-24   AACG      282.0               0.0     3222.0       8.75  NaN
...          ...    ...        ...               ...        ...        ...  ...
9730  2023-05-23   ZVSA   428545.0           24161.0  1176727.0      36.42  NaN
9731  2023-05-23    ZWS   249132.0               0.0   353320.0      70.51  NaN
9732  2023-05-23   ZYME    31232.0               0.0   100450.0      31.09  NaN
9733  2023-05-23   ZYNE    41870.0               0.0    55356.0      75.64  NaN
9734  2023-05-23   ZYXI   379252.0               0.0   394645.0      96.10  NaN

[213816 rows x 7 columns]
0
50000
100000
150000
200000
https://cdn.finra.org/equity/regsho/daily/CNMSshvol20230523.txt
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/mysql/connector/connection_cext.py", line 526, in cmd_query
    query_attrs=self._query_attrs)
_mysql_connector.MySQLInterfaceError: Column count doesn't match value count at row 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "tasks_to_run.py", line 242, in <module>
    get_short_volume.main()
  File "/code/scheduled_tasks/stocks/get_short_volume.py", line 99, in main
    get_daily_data_finra()
  File "/code/scheduled_tasks/stocks/get_short_volume.py", line 73, in get_daily_data_finra
    cur.executemany("INSERT IGNORE INTO short_volume VALUES (%s, %s, %s, %s, %s, %s)", df.values.tolist())
  File "/usr/local/lib/python3.7/site-packages/mysql/connector/cursor_cext.py", line 363, in executemany
    self.execute(operation, params)
  File "/usr/local/lib/python3.7/site-packages/mysql/connector/cursor_cext.py", line 271, in execute
    raw_as_string=self._raw_as_string)
  File "/usr/local/lib/python3.7/site-packages/mysql/connector/connection_cext.py", line 529, in cmd_query
    sqlstate=exc.sqlstate)
mysql.connector.errors.DataError: 1136 (21S01): Column count doesn't match value count at row 1

Fix:

  1. Added a nan column to the short_volume table in mysql
  2. edited line 73 on get_short_volume from: cur.executemany("INSERT IGNORE INTO short_volume VALUES (%s, %s, %s, %s, %s, %s, %s)", df.values.tolist()) to this cur.executemany("INSERT IGNORE INTO short_volume VALUES (%s, %s, %s, %s, %s, %s, 'nan')", df.values.tolist())

still dont get the information in the DB but its in the local files so no issue in reality.

CodeInFilth commented 1 year ago

Lol sorry this is my 8th issue request i have submitted on here. Hope its more helpful than a nuisance.

guanquann commented 1 year ago

Sorry, I made a minor mistake in the library. Update version finvizfinance to 0.13.0, not finnhub-python. You can refer to https://github.com/lit26/finvizfinance/issues/60.

CodeInFilth commented 1 year ago

very strange im going to have to poke around I am on 0.13.0

I tried these commands.

pip3 install finvizfinance==0.13.0 -U

that didnt work so i tried this

docker compose run --rm web pip3 install finvizfinance==0.13.0 -U

that didnt work either when i turn only the get_latest_insider_trading to True and run 

docker compose run --rm web python tasks_to_run.py

any direction would be helpful but no worries ill poke around

tankisstank commented 10 months ago

For me, I need to update finvizfinance==0.14.7 to make it work. And one more thing, I realize that my internet is BLOCKING request from cdn.finra.org, so I need to use under VPN to work. Feeling so nut.