Legality of the API

[Arkoprabho]

I stumbled across this link that says and i quote "You may not conduct any systematic or automated data collection activities (including scraping, data mining, data extraction and data harvesting) on or in relation to our website without our express written consent." Does this API follow the norms and the terms of NSE?

[mehul90]

Did I just read scraping ?? Well, Beautiful Soup does exactly that 🙇🏼

Guess it's time to refactor the code to fetch data from let's say zerodha or other legal sources, while still keeping the existing code and showing a warning everytime a user decides to use this path to scrap the NSE website.

[Arkoprabho]

But wouldn't it cause problem as this is an automated data collection, and there are no written consents. After all this API is providing a platform to collect the data. I'm sure the sites like the ones you mentioned, dont allow fetching data in a manner similar to what this API does. I couldn't find a way to use Zerodha yet, but say moneycontrol explicitly says that the data that they provide on the website is theirs and cannot be used outside of the website.

[Arkoprabho]

@swapniljariwala Can you kindly look into it? And share whatever details you get

[pannet1]

@Arkoprabho,

What you are talking about is a standard disclaimer. Without going to the live data part .... we are just downloading the data from history section in an automated way. It is very tedious to do certain things like manually checking the index price on a particular day, then decide the option strike price, buy hedge and repeat this to the entire period. Your fear is not helping this project. If you are concerned about the legality please don't use. The same people working for the exchange are accused of giving preference to certain brokers. Consider the plight of poor investors who wanted to invest or get returns from stock market (that is definitely rigged)

[pannet1]

-deleted-

[Arkoprabho]

@pannet1

That link, points to the terms of use, this is the disclaimer.

I am no expert in financial systems or the way it works. I am simply a software dev interested in contributing to the project. But at the same time I have very little to no interest in getting stuck in legal procedures hindering my career. And I am sure other devs would share a similar thought process I understand some things are tedious, and that is exactly why programming is so useful. From where I am right now, I would probably not use this API for any commercial or personal monetary gains. The only use I have of this API is to learn and then further contribute to thi,s and upcoming projects. But given that the terms of use strictly says any automated data collection is prohibited, it spiked concerns.

[pannet1]

@Arkoprabho

It all makes sense now when I see that you are not a trader in financial markets. As programming professional your concerns are correct and in the right direction. I have been working with NSEPY for most of past few days and i am very happy with the results. Here is a recent working program and there is also another module (nse_fno.py) which I developed based on NSEPY which this program depends. I am not inclding the module now. may be sometimes later. `from datetime import date, time, datetime, timedelta import math from nsepy.derivatives import get_expiry_date from nse_fno import * import os import sys

symbol = "BANKNIFTY" lot_size = 40

if len(sys.argv) < 4: exit

arg_mon = str(sys.argv[1]) arg_day = str(sys.argv[2]) arg_fun = str(sys.argv[3])

beg_date = date(2016,int(arg_mon),int(arg_day)) beg_time = datetime.combine(beg_date, time()) month_expiry = get_expiry_date(beg_time.year, beg_time.month)

leg_beg_time = next_working_day(beg_time) weekly_expiry = next_weekly_expiry(leg_beg_time)

print(len(sys.argv)) if len(sys.argv) == 5: base_strike = int(sys.argv[4])
else: future = future_price(symbol, leg_beg_time, leg_beg_time, month_expiry, csvname="future") base_strike = (math.trunc(future / 100)) * 100

if arg_fun == "bc": monthly = option_price(symbol, beg_time, base_strike, month_expiry, str_trans_type="BUY :", str_option="CE", csvname="monthly") monthly *= lot_size write_to_csv([beg_date,"BUY", "CE", base_strike,monthly])

if arg_fun == "sc": monthly = option_price(symbol, beg_time, base_strike, month_expiry, str_trans_type="SELL:", str_option="CE", csvname="monthly") monthly *= lot_size write_to_csv([beg_date,"SELL", "CE", base_strike,monthly])

if arg_fun == "bp": monthly = option_price(symbol, beg_time, base_strike, month_expiry, str_trans_type="BUY :", str_option="PE", csvname="monthly") monthly *= lot_size write_to_csv([beg_date,"BUY" ,"PE", base_strike,monthly])

if arg_fun == "sp": monthly = option_price(symbol, beg_time, base_strike, month_expiry, str_trans_type="SELL:", str_option="PE", csvname="monthly") monthly *= lot_size write_to_csv([beg_date,"SELL", "PE", base_strike,monthly])`

Note: I am a total newbie to Python.

[swapniljariwala]

I'm updating the license based on another python library Scrapy which is one of the scraping library, (to include Limitation of Liability)

[amansingh9097]

well I've been creating web crawler and web scrappers for years now and the first thing I look for on any website before writing a web crawler/scrapper is their /robots.txt file. But strangely when I looked at https://www.nseindia.com/robots.txt it clearly said it disallows any kind of web crawling to any of their sub-pages.

User-agent: *

Disallow: /static/htmls/
Disallow: /static/src/
Disallow: /server/
Disallow: /api/

Sitemap: https://www.nseindia.com/sitemap.xml
Sitemap: https://www.nseindia.com/sitemap-stocks.xml

Now as much as I find this repo really helpful, being a software engineer myself I also can't deny the fact, that no matter what library you use (scrappy or beautifulsoup or what not) its still considered illegal if you're doing it against their permission.

Having said that, as long as the NSEIndia doesn't finds out, I'm still gonna use it because heck it why shouldn't I??? Someone has put it so much effort into this. But to the contributors of this repo, I'd just be careful. Peach!