pranjal-joshi / Screeni-py

A Python-based stock screener to find stocks with potential breakout probability from NSE India.
MIT License
552 stars 195 forks source link

Unable to get F&O Stock List #240

Closed ankur2ankur closed 2 months ago

ankur2ankur commented 4 months ago

Unable to get F&O Stocks.

I lookup the code & found that the url "https://archives.nseindia.com/content/fo/fo_mktlots.csv" returning 404 status code. is there any other alternative resource for the same?

pranjal-joshi commented 4 months ago

Seems like they've shutdown the archives URL Need to figure out the replacement for this on new NSE and then push it in the next release

sandeep255 commented 4 months ago

We can make use of these.

https://nsearchives.nseindia.com/content/fo/NSE_FO_contract_06052024.csv.gz

pranjal-joshi commented 4 months ago

Thanks @sandeep255 for pointing this out. Will try to implement with the new data source.

ankur2ankur commented 4 months ago

suggestion from @sandeep255 is welcomed, but its implementation will be expensive. because its a archive file (need to extract it every time to read) and it has more than 146K rows in it (will take time to arrange & sort data).

ankur2ankur commented 4 months ago

Found new info, look at this: https://www.nseindia.com/products-services/equity-derivatives-contract-information

github-actions[bot] commented 3 months ago

This Issue is marked as Stale due to Inactivity. This Issue will be Closed soon.

ksshaha commented 3 months ago

Unable to get F&O Stocks.

I lookup the code & found that the url "https://archives.nseindia.com/content/fo/fo_mktlots.csv" returning 404 status code. is there any other alternative resource for the same?

I am also in search of this.....do you get any luck on this?

ankur2ankur commented 3 months ago

Unable to get F&O Stocks. I lookup the code & found that the url "https://archives.nseindia.com/content/fo/fo_mktlots.csv" returning 404 status code. is there any other alternative resource for the same?

I am also in search of this.....do you get any luck on this?

Nope, but i have made my own csv of F&O stocks, hosted in my storage and using that file. That is working for me temporarily. if @pranjal-joshi wants, then I can Send Pull Request.

alternate Solution, we can use list of static list in code to get F&O scripts.

Here is code snapshot of my changes. image

pranjal-joshi commented 3 months ago

Hi @ankur2ankur

Static file is not an optimized approach as NSE keeps changing lot sizes and Stock codes in this list quite often!

Thankfully @sandeep255 has suggested the URL above in which we can replace datestamp dynamically to pull the data.

We can make use of these.

https://nsearchives.nseindia.com/content/fo/NSE_FO_contract_06052024.csv.gz

As of now it works well from the browser but I am not able to pull the same Gzip from python using Requests library (Most likely because NSE wants some particular headers/cookies along with the request to validate if the requests are originating from browsers)

If anyone can find a workaround for the above idea, I can accept a PR for this if it passes all the workflows.

Thanks!

ankur2ankur commented 3 months ago

I am not master in python, but this code should work (I have added cookie):

import requests

url = "https://nsearchives.nseindia.com/content/fo/NSE_FO_contract_06052024.csv.gz"

payload = {}
headers = {
  'Cookie': '_abck=762933D34495D0CAA23489F894B021EC~-1~YAAQrCdzaNUz1b+PAQAANzNEzQus/TIIx6aW/La96DbWdpDIcDK745ktGbQECFoAmOu4LL7X6Hh0dc8xIta9FLAYT3VJXbNelOWCHyoOOFexpuq90WY5OPgk4d48y9niPVGMOZeslZGJwovYtITfMsSBJjkMCfaZBdRN1ESBPRZNslEmsb/uKL0O8dMpnnDaEcosUsgAXro1wRxJPy5OfLQwswgUBjNacafHvRlay3L3oQ2oz+cKIdt4SUOiF6RrjYwrQ0+7apQnV8ZENfckwyDXpEhc7PVQeUAgP/KtuKMZziKrdp+YSX+9ng9KdrsNwDI1hbjrqQAvmVeNuT7rDx9Yvuj3i8DjyWYXiEsKhogAebpWdf4OfJn5Tw==~-1~-1~-1; bm_sz=56574DCF2D9307C210A11CC0C1FEF50C~YAAQrCdzaNYz1b+PAQAANzNEzRfhDQCxz2ewCGQkfF+jPX3Q7cnrtKnqgKnxR4/lOiQ+kbia67AKh3ppygk4F2vw8fiUHQMJdP9FxM2MApt20Pt8hDEzO0ywstLBdm+gNh1LObIvSI7cUl11E2XpXgR+Y0XoMtaZpKW2Zt1/ghWLqHSn9pvyDzjqWrEkg6Mg9n00AqGbdkkA8DTkcfjCel9CREptqxLAroZFCaWTkEY2dBd6kYjD2E1eisiINOgk0fdxVOyfye8BK/zcVnSPDPNe/f93X7DWilLJ8IrCqg0ilgehfwoIiatlRCH3gmSHBUQT+e5MZvqtrvgCKkZq1nWRpN6cpgcUgKPmK9dpug==~4403769~3753027'
}

response = requests.request("GET", url, headers=headers, data=payload)

print(response.text)
pranjal-joshi commented 3 months ago

@ankur2ankur

Have you obtained this cookie through the browser? What is the expiration for this cookie? Is there any way to get this cookie from NSE through Python requests using the user-agent method?

ankur2ankur commented 3 months ago

Got the cookie from postman. In Postman there are 2 methods to obtain file: httpclient & request. Above mentioned is though request.

github-actions[bot] commented 3 months ago

This Issue is marked as Stale due to Inactivity. This Issue will be Closed soon.