virresh / StockD

EOD Stock data downloader.
https://virresh.github.io/projects/stockd
GNU General Public License v2.0
11 stars 4 forks source link

Index translation table #3

Closed virresh closed 4 years ago

virresh commented 4 years ago

NSE has a bad habit of changing names of Indices frequently

Major trading analysis tools do not accept spaces in symbol names and thus we need some kind of translation Also there is the problem of old index symbols and new ones

It would be great if someone can list a table with corresponding names I'll incorporate it into a new release, however, I will need to know how to map them

kalyan4512 commented 4 years ago

ok will certainly try to list major indices but if u try to rename as per our database wud be good . as there r many insdices so it is important to have option to select indices. this is main criteria. as like nseeodfixed. take ur time to develop . i think it is going tobe greta software but pls dont delete nseeodfixed from ur help list untill ur project develop

kalyan4512 commented 4 years ago

eg nifty 50 is nifty spot symbol but in our database it is renamed as NIFTY

virresh commented 4 years ago

important to have option to select indices

I understand the concern of having option to select indices, but this option has no performance benefits. In order to download even one index, the data of every other index needs to be downloaded.

In my opinion, it would be better to download all the indices and have them inside your database. It will then be possible for a user to not worry about lack of data on other indices in case they wish to analyse them later on

This will also make it easy in future in case NSE adds more indices/removes indices. No change in software will be needed.

So what I'm proposing is maintaining a translation table. As you mentioned about nifty 50, the table will simply map an index name to another name which is present in most user's database already

This table would look as follows:

Actual Index Name New Name
Nifty 50 NIFTY
CNX Nifty NIFTY
... ...

As of now, these are underscore mapped. i.e Nifty 50 will be saved as Nifty_50 and so on

With the translation table, we will convert them into new name if available, otherwise use underscore mapping if an entry in table is not found

This makes the software future-proof

kalyan4512 commented 4 years ago

in my layman view there will be a problem of searching indices in charting software and many indices will appear on searching ,so keeping indices in limited nos is preferred. we are also using one downloader from getbhavcopy.com ,which in latest update bring almost all the indices and that delaying download. any way i have no such idea about programming so can't understand what difficulties u may face to map indices and limit its nos. just request u think about it. as max ppl who r using free downloader using nseeodfixed. ur software is far better with more customization. pls just think about indices as nseeodfixed. and also may be java 8 will stay for longer so until java 8 is there pls help in case any issue in nseeodfixed.. i have sent many user ur software for their feedback

On Fri, Jul 10, 2020 at 11:21 AM Viresh Gupta notifications@github.com wrote:

important to have option to select indices

I understand the concern of having option to select indices, but this option has no performance benefits. In order to download even one index, the data of every other index needs to be downloaded.

In my opinion, it would be better to download all the indices and have them inside your database. It will then be possible for a user to not worry about lack of data on other indices in case they wish to analyse them later on

This will also make it easy in future in case NSE adds more indices/removes indices. No change in software will be needed.

So what I'm proposing is maintaining a translation table. As you mentioned about nifty 50, the table will simply map an index name to another name which is present in most user's database already

This table would look as follows: Actual Index Name New Name Nifty 50 NIFTY CNX Nifty NIFTY ... ...

As of now, these are underscore mapped. i.e Nifty 50 will be saved as Nifty_50 and so on

With the translation table, we will convert them into new name if available, otherwise use underscore mapping if an entry in table is not found

This makes the software future-proof

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/virresh/StockD/issues/3#issuecomment-656496882, or unsubscribe https://github.com/notifications/unsubscribe-auth/AOSAUDN45TZZSSOQFI6JBHDR22T4ZANCNFSM4OWHDM5Q .

virresh commented 4 years ago

I'll add options for the user to choose whether they want to rename indices or not and which all indices they want in the data

This should solve all the issues with regards to your usage

kalyan4512 commented 4 years ago

thanx as u r considering this for indices . .No hurry take ur time. but if u dont mind nseeodfixed has such script(programming portion) for indices but may be in other language. any way for this software we can also can switch to 64 bit in future but limited to window 7(not window 10) due to my hardware. so may be java version may not be a problem if it is necessary to switch to java 9. so may be rectifying nseeodfixed also dont have any problem if any issue arise. any way thanx for responding my query in ur busy time

On Sun, Jul 12, 2020 at 2:03 AM Viresh Gupta notifications@github.com wrote:

I'll add options for the user to choose whether they want to rename indices or not and which all indices they want in the data

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/virresh/StockD/issues/3#issuecomment-657125411, or unsubscribe https://github.com/notifications/unsubscribe-auth/AOSAUDNJZ4ECP6MYDHGLY3LR3DECXANCNFSM4OWHDM5Q .

virresh commented 4 years ago

Based on NSE EOD's source code Here is a translation table

Actual Index Name New Name
CNX NIFTY NIFTY
NIFTY 50 NIFTY
CNX NIFTY JUNIOR JUNIOR
NIFTY NEXT 50 JUNIOR
BANK NIFTY BANKNIFTY
NIFTY BANK BANKNIFTY
CNX 100 NSE100
NIFTY 100 NSE100
CNX MIDCAP NSEMIDCAP
NIFTY MIDCAP 100 NSEMIDCAP
CNX IT NSEIT
NIFTY IT NSEIT
CNX 500 NSE500
NIFTY 500 NSE500
CNX DEFTY NSEDEFTY
NIFTY MIDCAP 50 MIDCAP50
S&P ESG INDIA INDEX NSESG
CNX NIFTY SHARIAH NSESHARIAH
NIFTY50 SHARIAH NSESHARIAH
CNX 500 SHARIAH SHARIAH500
NIFTY500 SHARIAH SHARIAH500
CNX INFRA NSEINFRA
NIFTY INFRA NSEINFRA
CNX REALITY NSEREALTY
NIFTY REALITY NSEREALTY
CNX ENERGY NSEENERGY
NIFTY ENERGY NSEENERGY
CNX FMCG NSEFMCG
NIFTY FMCG NSEFMCG
CNX MNC NSEMNC
NIFTY MNC NSEMNC
CNX PHARMA NSEPHARMA
NIFTY PHARMA NSEPHARMA
CNX PSE NSEPSE
NIFTY PSE NSEPSE
CNX PSU BANK NSEPSUBANK
NIFTY PSU BANK NSEPSUBANK
CNX SERVICE NSESERVICE
NIFTY SERV SECTOR NSESERVICE
CNX AUTO NSEAUTO
NIFTY AUTO NSEAUTO
CNX FINANCE NSEFINANCE
NIFTY FINANCE NSEFINANCE
CNX METAL NSEMETAL
NIFTY METAL NSEMETAL
INDIA VIX VIX
However, I have not found any translation for following indices (these are present in the data so many people would want these): Actual Index Name
Nifty 200
Nifty 500
Nifty Midcap 50
NIFTY Smallcap 100
Nifty50 Dividend Points
Nifty Auto
Nifty Bank
Nifty Energy
Nifty Financial Services
Nifty Media
Nifty Metal
Nifty India Consumption
Nifty Commodities
Nifty Dividend Opportunities 50
Nifty Infrastructure
Nifty Services Sector
Nifty50 Shariah
Nifty500 Shariah
Nifty Low Volatility 50
Nifty Alpha 50
Nifty High Beta 50
Nifty100 Equal Weight
Nifty100 Liquid 15
Nifty CPSE
Nifty50 Value 20
Nifty Midcap Liquid 15
Nifty Shariah 25
Nifty Growth Sectors 15
Nifty50 TR 1x Inverse
Nifty50 TR 2x Leverage
Nifty50 PR 1x Inverse
Nifty50 PR 2x Leverage
NIFTY100 Quality 30
Nifty 50 Futures TR Index
Nifty 50 Arbitrage
NIFTY50 Equal Weight
Nifty100 Low Volatility 30
NIFTY Alpha Low-Volatility 30
NIFTY Quality Low-Volatility 30
NIFTY Alpha Quality Low-Volatility 30
NIFTY Alpha Quality Value Low-Volatility 30
NIFTY200 Quality 30
NIFTY LargeMidcap 250
NIFTY SME EMERGE
Nifty Oil & Gas
NIFTY100 ESG
NIFTY100 Enhanced ESG
NIFTY500 Value 50
NIFTY100 Alpha 30
Nifty 50 Futures Index
Nifty Consumer Durables
Nifty Midcap 150
Nifty MidSmallcap 400
Nifty Smallcap 50
Nifty Smallcap 250
Nifty Private Bank
Nifty Tata Group 25% Cap
Nifty Tata Group
Nifty Mahindra Group
Nifty Aditya Birla Group
Nifty 8-13 yr G-Sec
Nifty 4-8 yr G-Sec Index
Nifty 11-15 yr G-Sec Index
Nifty 15 yr and above G-Sec Index
Nifty Composite G-sec Index
Nifty 10 yr Benchmark G-Sec
Nifty 10 yr Benchmark G-Sec (Clean Price)
Nifty 1D Rate Index
Nifty50 USD

@kalyan4512 please go through the first table and let me know if there are any mistakes

Also have a look at the second table.

These indices will be removed from the data when you will apply filter.

I believe they contain many sectoral indices which would be used by traders, e.g the Nifty Private Bank, Nifty Infrastructure, Nifty Media, Nifty Metal etc and removing them would be an issue for some people.

virresh commented 4 years ago

Issue solved with release of StockD v4.2: https://github.com/virresh/StockD/releases/tag/4.2 Implemented index filtering and renaming via index map.

All capabilities of NSEEODFIXED are now in StockD and hence no more updates to NSEEODFIXED are required anymore.