shilewenuw / get_all_tickers

Get all publicly traded tickers with this library
MIT License
273 stars 123 forks source link

New NASDAQ fix is not in PyPi #18

Closed paulmcq closed 3 years ago

paulmcq commented 3 years ago

The current code in PyPi is version 1.7 from August 2020. It does not work, as it uses the obsolete NASDAQ api. See PR#17, and Issues #11 and #12. Does anyone intend to upgrade PyPi?

shilewenuw commented 3 years ago

I'll do that tonight

shilewenuw commented 3 years ago

I meant to do this 2 days ago. I'll do this in a few hours. Sorry I've been pretty busy with work.

shilewenuw commented 3 years ago

done

paulmcq commented 3 years ago

Thank you very much for this. Pypi indeed lists version 1.8 now, and pip upgrade does install that.

However, get_tickers.py still has the old Nasdaq url at line 132:

response = requests.get(' https://old.nasdaq.com/screening/companies-by-name.aspx', headers=headers, params=params(exchange))

Do you have any clues for me?

I tried this to make sure it was not an error in my code: (panda) stonk python $VIRTUAL_ENV/lib/python3.8/site-packages/get_all_tickers/get_tickers.py Traceback (most recent call last): File "/home/paulmcq/.virtualenvs/panda/lib/python3.8/site-packages/get_all_tickers/get_tickers.py", line 187, in tickers = get_tickers() File "/home/paulmcq/.virtualenvs/panda/lib/python3.8/site-packages/get_all_tickers/get_tickers.py", line 73, in get_tickers tickers_list.extend(exchange2list('nyse')) File "/home/paulmcq/.virtualenvs/panda/lib/python3.8/site-packages/get_all_tickers/get_tickers.py", line 138, in exchange2list df = exchange2df(exchange) File "/home/paulmcq/.virtualenvs/panda/lib/python3.8/site-packages/get_all_tickers/get_tickers.py", line 134, in exchange2df df = pd.read_csv(data, sep=",") File "/home/paulmcq/.virtualenvs/panda/lib/python3.8/site-packages/pandas/io/parsers.py", line 688, in read_csv return _read(filepath_or_buffer, kwds) File "/home/paulmcq/.virtualenvs/panda/lib/python3.8/site-packages/pandas/io/parsers.py", line 460, in _read data = parser.read(nrows) File "/home/paulmcq/.virtualenvs/panda/lib/python3.8/site-packages/pandas/io/parsers.py", line 1198, in read ret = self._engine.read(nrows) File "/home/paulmcq/.virtualenvs/panda/lib/python3.8/site-packages/pandas/io/parsers.py", line 2157, in read data = self._reader.read(nrows) File "pandas/_libs/parsers.pyx", line 847, in pandas._libs.parsers.TextReader.read File "pandas/_libs/parsers.pyx", line 862, in pandas._libs.parsers.TextReader._read_low_memory File "pandas/_libs/parsers.pyx", line 918, in pandas._libs.parsers.TextReader._read_rows File "pandas/_libs/parsers.pyx", line 905, in pandas._libs.parsers.TextReader._tokenize_rows File "pandas/_libs/parsers.pyx", line 2042, in pandas._libs.parsers.raise_parser_error pandas.errors.ParserError: Error tokenizing data. C error: Expected 1 fields in line 5, saw 46

On Wed, Jul 14, 2021 at 9:01 PM Shile Wen @.***> wrote:

Closed #18 https://github.com/shilewenuw/get_all_tickers/issues/18.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/shilewenuw/get_all_tickers/issues/18#event-5022757601, or unsubscribe https://github.com/notifications/unsubscribe-auth/AD3AQGPHWGGPJOGMCXSXPBLTXY6QNANCNFSM5AHCY7TA .

shilewenuw commented 3 years ago

I merged whatever Rik Brown wrote. I'll take a look over the weekend, however, if you figure out a fix to the source code beforehand, I can merge your PR.

paulmcq commented 3 years ago

I downloaded get_all_tickers-1.8.tar.gz from PyPi. The files in there have 2020 dates, as shown in atch. Probably just the wrong version got copied into the tarball?

Thanks, again, for your work, ~Paul

On Thu, Jul 15, 2021 at 10:59 AM Shile Wen @.***> wrote:

I merged whatever Rik Brown wrote. I'll take a look over the weekend, however, if you figure out a fix to the source code beforehand, I can merge your PR.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/shilewenuw/get_all_tickers/issues/18#issuecomment-880815295, or unsubscribe https://github.com/notifications/unsubscribe-auth/AD3AQGN7OJKYHWBSEQO6T63TX4AVZANCNFSM5AHCY7TA .

paulmcq commented 3 years ago

I should have looked closer: the PyPI tarball is NOT the same as the github release: https://github.com/shilewenuw/get_all_tickers/releases/tag/v_18 Both the zip and tgz have 2021 timestamps.

So the PyPI v_18 is NOT consistent with github. I have no idea how to fix that, so relying on owner to fix...

BTW, the tarball on PyPI does NOT have the structure of the github tarball

On Thu, Jul 15, 2021 at 11:50 AM Paul McQuesten @.***> wrote:

I downloaded get_all_tickers-1.8.tar.gz from PyPi. The files in there have 2020 dates, as shown in atch. Probably just the wrong version got copied into the tarball?

Thanks, again, for your work, ~Paul

On Thu, Jul 15, 2021 at 10:59 AM Shile Wen @.***> wrote:

I merged whatever Rik Brown wrote. I'll take a look over the weekend, however, if you figure out a fix to the source code beforehand, I can merge your PR.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/shilewenuw/get_all_tickers/issues/18#issuecomment-880815295, or unsubscribe https://github.com/notifications/unsubscribe-auth/AD3AQGN7OJKYHWBSEQO6T63TX4AVZANCNFSM5AHCY7TA .