search

virresh / nseeod

NSE EOD Data Downloader
http://nse-eod-downloader.blogspot.com
2 stars 0 forks source link

nse f&o data is now available in original nseindia.com today but not in www1.nseindia.com #4

Open kalyan4512 opened 4 years ago

kalyan4512 commented 4 years ago

today observed that f&o still can not be downloaded ,when checking saw that it is not available(2/3/2020 data) in www1.nseindia.com but is available in nseindia.com but here file size is bigger(i guess). sometimes i saw that they upload f&o bhabcopy latter , if that no problem.

kalyan4512 commented 4 years ago

hi virresh ,r u active here? we need this utility very much ,in that case the users can contribute,if u dont mind as per their capacity as i think, so if u check this comment pls give ur view .

virresh commented 4 years ago

Hi @kalyan4512 I'm a bit busy with university work. Though please let me know the exact problems. I'll try to get them solved when time allows, can't make promises though. All contributions are much appreciated :sweat_smile:

kalyan4512 commented 4 years ago

hi virresh how are u now. any way first of all again thanx for ur contribution for this app. now the problem is that it is now a bit slow to download also currency is not downloading. if u check from ur end. any way if no problem u can give ur bank credential so that users can can contribute as they can.. u can also give the bank credential in a safe way in ur site or any suitable site. but contribution amount may vary . i will recommend every users to contribute.

On Fri, May 1, 2020 at 4:44 PM Viresh Gupta notifications@github.com wrote:

Hi @kalyan4512 https://github.com/kalyan4512 I'm a bit busy with university work. Though please let me know the exact problems. I'll try to get them solved when time allows, can't make promises though. All contributions are much appreciated 😅

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/virresh/nseeod/issues/4#issuecomment-622348421, or unsubscribe https://github.com/notifications/unsubscribe-auth/AOSAUDPKUCLQFTJCABZJYLDRPKVKBANCNFSM4K7WGUVA .

virresh commented 4 years ago

Hi Kalyan. Thanks for the monetary contribution offer :sweat_smile: , I'll set up some project sponsorship programme in future keeping this in mind to help fund this project and keep it free to use for all.

Thanks for being an active user. I'll try to look into the issue of currency downloads and latency and try to work on a new release in a couple of months.

kalyan4512 commented 4 years ago

thanx for ur help. we want this app sud perform smoothly. though currency is not main issue now. it is latency right now and also if any other issue arise in future. so want u stay with us . in that view i with many other users have a view to contribute for this project ,what ever amount we can. regard kalyan. hope u r well

On Sat, May 2, 2020 at 5:28 PM Viresh Gupta notifications@github.com wrote:

Hi Kalyan. Thanks for the monetary contribution offer 😅 , I'll set up some project sponsorship programme in future keeping this in mind to help fund this project and keep it free to use for all.

Thanks for being an active user. I'll try to look into the issue of currency downloads and latency and try to work on a new release in a couple of months.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/virresh/nseeod/issues/4#issuecomment-622942492, or unsubscribe https://github.com/notifications/unsubscribe-auth/AOSAUDOH3DFUAIRTWMXMN73RPQDG3ANCNFSM4K7WGUVA .

virresh commented 4 years ago

Can you elaborate a bit about the latency issue? I tried benchmarking the performance and a day's bhavcopy was downloaded and extracted in under a second on a decent internet connection (~2 MBps which is equal to approx 16 Mbps)?

kalyan4512 commented 4 years ago

mainly it is taking time during indices. i select cash with indices and future both .when downloading msg coming. in nse eod data downloader window it is stuck for considerable time before indices. i think nse indices taking time to download . mainly current day download. next day it is ok most of the time. i am using jiofy wifi connection in my desktop. if it is downloading ok from ur end then may be it need good bandwidth as jiofy speed is not so good nax time. just clarifying my problem. i have 2 desktop but both r using jiofy wifi with wifi adapter. both desktop is displaying such problem ,mainly current day download around 7 pm to 10 pm.

On Sun, May 3, 2020 at 3:14 PM Viresh Gupta notifications@github.com wrote:

Can you elaborate a bit about the latency issue? I tried benchmarking the performance and a day's bhavcopy was downloaded and extracted in under a second on a decent internet connection (~2 MBps which is equal to approx 16 Mbps)?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/virresh/nseeod/issues/4#issuecomment-623082981, or unsubscribe https://github.com/notifications/unsubscribe-auth/AOSAUDPSZYRQI42TZUNM3CLRPU4I7ANCNFSM4K7WGUVA .

virresh commented 4 years ago

Hi kalyan, I replicated the issue

Problem seems to be the usage of www1.nseindia.com server Here is relevant benchmarks with wget on linux:

From the new server:

$ time wget https://archives.nseindia.com/content/historical/EQUITIES/2020/MAY/cm05MAY2020bhav.csv.zip
--2020-05-05 21:53:39--  https://archives.nseindia.com/content/historical/EQUITIES/2020/MAY/cm05MAY2020bhav.csv.zip
Resolving archives.nseindia.com (archives.nseindia.com)... 23.54.90.113, 2600:140f:5:994::2630, 2600:140f:5:9a2::2630
Connecting to archives.nseindia.com (archives.nseindia.com)|23.54.90.113|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 67756 (66K) [application/zip]
Saving to: ‘cm05MAY2020bhav.csv.zip.1’

cm05MAY2020bhav.csv.zip.1                            100%[=====================================================================================================================>]  66.17K  --.-KB/s    in 0.02s   

2020-05-05 21:53:39 (2.82 MB/s) - ‘cm05MAY2020bhav.csv.zip.1’ saved [67756/67756]

real    0m0.192s
user    0m0.003s
sys 0m0.009s

From the old server:

$ time wget https://www1.nseindia.com/content/historical/EQUITIES/2020/MAY/cm04MAY2020bhav.csv.zip
--2020-05-05 21:53:54--  https://www1.nseindia.com/content/historical/EQUITIES/2020/MAY/cm04MAY2020bhav.csv.zip
Resolving www1.nseindia.com (www1.nseindia.com)... 23.54.90.113
Connecting to www1.nseindia.com (www1.nseindia.com)|23.54.90.113|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 68907 (67K) [application/zip]
Saving to: ‘cm04MAY2020bhav.csv.zip.1’

cm04MAY2020bhav.csv.zip.1                            100%[=====================================================================================================================>]  67.29K  --.-KB/s    in 0.03s   

2020-05-05 21:53:59 (2.15 MB/s) - ‘cm04MAY2020bhav.csv.zip.1’ saved [68907/68907]

real    0m4.295s
user    0m0.010s
sys 0m0.021s

I'll try to rectify the latency issue by switching to the newer sources for data Can't give an estimate on how long will it take me to implement the solution though

kalyan4512 commented 4 years ago

thanx a lot . take ur time . some times it download ok. previously when it download from nseindia.com speed was ok . we could download multiple days with good speed

On Tue, May 5, 2020 at 9:58 PM Viresh Gupta notifications@github.com wrote:

Hi kalyan, I replicated the issue

Problem seems to be the usage of www1.nseindia.com server Here is relevant benchmarks with wget on linux:

From the new server:

$ time wget https://archives.nseindia.com/content/historical/EQUITIES/2020/MAY/cm05MAY2020bhav.csv.zip

--2020-05-05 21:53:39-- https://archives.nseindia.com/content/historical/EQUITIES/2020/MAY/cm05MAY2020bhav.csv.zip

Resolving archives.nseindia.com (archives.nseindia.com)... 23.54.90.113, 2600:140f:5:994::2630, 2600:140f:5:9a2::2630

Connecting to archives.nseindia.com (archives.nseindia.com)|23.54.90.113|:443... connected.

HTTP request sent, awaiting response... 200 OK

Length: 67756 (66K) [application/zip]

Saving to: ‘cm05MAY2020bhav.csv.zip.1’

cm05MAY2020bhav.csv.zip.1 100%[=====================================================================================================================>] 66.17K --.-KB/s in 0.02s

2020-05-05 21:53:39 (2.82 MB/s) - ‘cm05MAY2020bhav.csv.zip.1’ saved [67756/67756]

real 0m0.192s

user 0m0.003s

sys 0m0.009s

From the old server:

$ time wget https://www1.nseindia.com/content/historical/EQUITIES/2020/MAY/cm04MAY2020bhav.csv.zip

--2020-05-05 21:53:54-- https://www1.nseindia.com/content/historical/EQUITIES/2020/MAY/cm04MAY2020bhav.csv.zip

Resolving www1.nseindia.com (www1.nseindia.com)... 23.54.90.113

Connecting to www1.nseindia.com (www1.nseindia.com)|23.54.90.113|:443... connected.

HTTP request sent, awaiting response... 200 OK

Length: 68907 (67K) [application/zip]

Saving to: ‘cm04MAY2020bhav.csv.zip.1’

cm04MAY2020bhav.csv.zip.1 100%[=====================================================================================================================>] 67.29K --.-KB/s in 0.03s

2020-05-05 21:53:59 (2.15 MB/s) - ‘cm04MAY2020bhav.csv.zip.1’ saved [68907/68907]

real 0m4.295s

user 0m0.010s

sys 0m0.021s

I'll try to rectify the latency issue by switching to the newer sources for data Can't give an estimate on how long will it take me to implement the solution though

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/virresh/nseeod/issues/4#issuecomment-624159031, or unsubscribe https://github.com/notifications/unsubscribe-auth/AOSAUDOR522CFCS7PVFJ2J3RQA5EBANCNFSM4K7WGUVA .

kalyan4512 commented 4 years ago

hi virresh ,if u help this time ,it is taking time now to download,mainly indices ,it seems. i dont know where to write,if u want u can share ur bank credential or paytm etc . if u answer ,then may be i can get it in my email or if u have any site. regard kalyan

virresh commented 4 years ago

Hi Kalyan, I'm trying to re-develop the software for more customizability so that people can load different settings easily. You can track the progress here: https://github.com/virresh/StockD/ It is far from complete, I hope to be able to finish it by this month. Sorry for the inconvenience and thanks for the support!

kalyan4512 commented 4 years ago

format of data eg ticker,date(yyyymmdd),open,high,low,close,volume,oi(in case of future). sud keep intact as like recent software, as we have database according to that format. it is a request. now the downloading speed is a problem. any one can download multiple days data quickly if required,that is also preferred. any way u r helping us in a great way . u dont know how important is the software for us.

On Fri, Jun 12, 2020 at 11:00 PM Viresh Gupta notifications@github.com wrote:

Hi Kalyan, I'm trying to re-develop the software for more customizability so that people can load different settings easily. You can track the progress here: https://github.com/virresh/StockD/ It is far from complete, I hope to be able to finish it by this month. Sorry for the inconvenience and thanks for the support!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/virresh/nseeod/issues/4#issuecomment-643397287, or unsubscribe https://github.com/notifications/unsubscribe-auth/AOSAUDKYMPRKDJM7IBAXTDTRWJQ33ANCNFSM4K7WGUVA .

virresh commented 4 years ago

format of data eg ticker,date(yyyymmdd),open,high,low,close,volume,oi(in case of future). sud keep intact as like recent software, as we have database according to that format

Sure, I'll try to keep output format such that it's fully compatible with nse-eod-data-downloader.