BennyThadikaran / eod2

A fully automated script to download and update NSE EOD Historical stock, index and delivery data with added features
GNU General Public License v3.0
68 stars 24 forks source link

pansari stock #176

Closed cotton1234 closed 1 month ago

cotton1234 commented 1 month ago

Hello Benny,

I was looking at the pansari stock. In the same, we have data from 2021 but if you see the same in broker terminal it has data from 2016.

I have seen other stocks also , so is there something which we can do about it.

Thanks Saurabh

BennyThadikaran commented 1 month ago

You're right. I looked a random bhavcopy from 2019 and Pansari is definitely there.

But back then, Pansari was a SME stock and is now part of equity series.

The file would be named pansari_sme.csv. After switching to EQ, it looks for pansari.csv. Not finding one, it creates a new file and starts updating the data there. Finally if pansari_sme.csv was not updated more than 365 days, it is deleted.

PANSARI EQ 01-FEB-2022 INE697V01011
PANSARI SM 01-MAR-2019 INE697V01011

Should there be a continuation of price when stock switches from SME to EQ?

If yes, then everytime a EQ series stock file is missing, it will look for the same file with _sme added and only then create a new file. If the _sme file is found, it will rename the file and continue appending data to it.

Let me know if this seems fine.

SME stock data will need to be recompiled to make old data available again and then changes in EOD2. I can make these changes over the next two weeks.

cotton1234 commented 1 month ago

You're right. I looked a random bhavcopy from 2019 and Pansari is definitely there.

But back then, Pansari was a SME stock and is now part of equity series.

The file would be named pansari_sme.csv. After switching to EQ, it looks for pansari.csv. Not finding one, it creates a new file and starts updating the data there. Finally if pansari_sme.csv was not updated more than 365 days, it is deleted.

PANSARI   EQ 01-FEB-2022 INE697V01011
PANSARI   SM 01-MAR-2019 INE697V01011

Should there be a continuation of price when stock switches from SME to EQ?

If yes, then everytime a EQ series stock file is missing, it will look for the same file with _sme added and only then create a new file. If the _sme file is found, it will rename the file and continue appending data to it.

Let me know if this seems fine.

SME stock data will need to be recompiled to make old data available again and then changes in EOD2. I can make these changes over the next two weeks.

Hello Benny,

SME stocks are traded in lots but since retail participation is less in these stocks and when company wants more participation they will switch the same to EQ. So price will not change or adjust as the face value is still same , only lot changes.

If yes, then everytime a EQ series stock file is missing, it will look for the same file with _sme added and only then create a new file. If the _sme file is found, it will rename the file and continue appending data to it.

So this is correct.

I hope i answer your question.

You can take your time, no hurry.

Thanks Saurabh

BennyThadikaran commented 1 month ago

Pansari stock now has data from 2016 Screenshot_2024-09-07_20-02-37

Some work is still pending. So maybe final update can be pushed on Monday.

cotton1234 commented 1 month ago

Pansari stock now has data from 2016 Screenshot_2024-09-07_20-02-37

Some work is still pending. So maybe final update can be pushed on Monday.

Thanks Benny

BennyThadikaran commented 1 month ago

All changes done. Update both EOD2 and eod2_data to get the latest changes.

cotton1234 commented 1 month ago

Thanks Benny, Just updated the same. Testing the same. Will update.

cotton1234 commented 1 month ago

Works Good Benny. All Set. Thanks

BennyThadikaran commented 1 month ago

The earlier update was missing delivery data. Bit of a mess up during compilation. eod2_data is now updated. You can pull the latest changes. Let me know if you face any issues.

cotton1234 commented 1 month ago

Hello Benny,

Getting this after the update.

(.venv) saurabhgarg@MacBook-Pro eod2 % git pull remote: Enumerating objects: 10, done. remote: Counting objects: 100% (10/10), done. remote: Compressing objects: 100% (6/6), done. remote: Total 8 (delta 6), reused 2 (delta 2), pack-reused 0 (from 0) Unpacking objects: 100% (8/8), 2.15 KiB | 274.00 KiB/s, done. From https://github.com/BennyThadikaran/eod2 9ce73e711..51a7744e4 main -> origin/main 8800d5561..3d85cad13 dev -> origin/dev Updating 9ce73e711..51a7744e4 Fast-forward src/eod2_data | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (.venv) saurabhgarg@MacBook-Pro eod2 % cd src/eod2_data (.venv) saurabhgarg@MacBook-Pro eod2_data % git reset --hard HEAD Updating files: 100% (2598/2598), done. HEAD is now at 5497e2037 Merge pull request #70 from BennyThadikaran/dev (.venv) saurabhgarg@MacBook-Pro eod2_data % git clean -f (.venv) saurabhgarg@MacBook-Pro eod2_data % git checkout main Already on 'main' Your branch is up to date with 'origin/main'. (.venv) saurabhgarg@MacBook-Pro eod2_data % git pull remote: Enumerating objects: 2670, done. remote: Counting objects: 100% (2670/2670), done. remote: Compressing objects: 100% (106/106), done. remote: Total 2670 (delta 2564), reused 2669 (delta 2564), pack-reused 0 (from 0) Receiving objects: 100% (2670/2670), 15.34 MiB | 4.99 MiB/s, done. Resolving deltas: 100% (2564/2564), completed with 2564 local objects. From https://github.com/BennyThadikaran/eod2_data

Thanks Saurabh

BennyThadikaran commented 1 month ago

Just run git reset --hard HEAD~1 followed by git pull.

First command will remove the last commit and discard any changes.

Finally pull the latest changes.

On 11 September 2024 02:28:57 UTC, Saurabh @.***> wrote:

Hello Benny,

Getting this after the update.

(.venv) @. eod2 % git pull remote: Enumerating objects: 10, done. remote: Counting objects: 100% (10/10), done. remote: Compressing objects: 100% (6/6), done. remote: Total 8 (delta 6), reused 2 (delta 2), pack-reused 0 (from 0) Unpacking objects: 100% (8/8), 2.15 KiB | 274.00 KiB/s, done. From https://github.com/BennyThadikaran/eod2 9ce73e711..51a7744e4 main -> origin/main 8800d5561..3d85cad13 dev -> origin/dev Updating 9ce73e711..51a7744e4 Fast-forward src/eod2_data | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (.venv) @. eod2 % cd src/eod2_data (.venv) @. eod2_data % git reset --hard HEAD Updating files: 100% (2598/2598), done. HEAD is now at 5497e2037 Merge pull request #70 from BennyThadikaran/dev (.venv) @. eod2_data % git clean -f (.venv) @. eod2_data % git checkout main Already on 'main' Your branch is up to date with 'origin/main'. (.venv) @. eod2_data % git pull remote: Enumerating objects: 2670, done. remote: Counting objects: 100% (2670/2670), done. remote: Compressing objects: 100% (106/106), done. remote: Total 2670 (delta 2564), reused 2669 (delta 2564), pack-reused 0 (from 0) Receiving objects: 100% (2670/2670), 15.34 MiB | 4.99 MiB/s, done. Resolving deltas: 100% (2564/2564), completed with 2564 local objects. From https://github.com/BennyThadikaran/eod2_data

  • 5497e2037...9a6d5eae5 main -> origin/main (forced update)
  • a7f4c7fd5...90aa3f711 dev -> origin/dev (forced update) hint: You have divergent branches and need to specify how to reconcile them. hint: You can do so by running one of the following commands sometime before hint: your next pull: hint: hint: git config pull.rebase false # merge hint: git config pull.rebase true # rebase hint: git config pull.ff only # fast-forward only hint: hint: You can replace "git config" with "git config --global" to set a default hint: preference for all repositories. You can also pass --rebase, --no-rebase, hint: or --ff-only on the command line to override the configured default per hint: invocation. fatal: Need to specify how to reconcile divergent branches.

Thanks Saurabh

-- Reply to this email directly or view it on GitHub: https://github.com/BennyThadikaran/eod2/issues/176#issuecomment-2342490405 You are receiving this because you commented.

Message ID: @.***> Regards, Benny Thadikaran

cotton1234 commented 1 month ago

Just run git reset --hard HEAD~1 followed by git pull. First command will remove the last commit and discard any changes. Finally pull the latest changes. On 11 September 2024 02:28:57 UTC, Saurabh @.> wrote: Hello Benny, Getting this after the update. (.venv) @. eod2 % git pull remote: Enumerating objects: 10, done. remote: Counting objects: 100% (10/10), done. remote: Compressing objects: 100% (6/6), done. remote: Total 8 (delta 6), reused 2 (delta 2), pack-reused 0 (from 0) Unpacking objects: 100% (8/8), 2.15 KiB | 274.00 KiB/s, done. From https://github.com/BennyThadikaran/eod2 9ce73e7..51a7744 main -> origin/main 8800d55..3d85cad dev -> origin/dev Updating 9ce73e7..51a7744 Fast-forward src/eod2_data | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (.venv) @. eod2 % cd src/eod2_data (.venv) @. eod2_data % git reset --hard HEAD Updating files: 100% (2598/2598), done. HEAD is now at 5497e2037 Merge pull request #70 from BennyThadikaran/dev (.venv) @. eod2_data % git clean -f (.venv) @. eod2_data % git checkout main Already on 'main' Your branch is up to date with 'origin/main'. (.venv) @. eod2_data % git pull remote: Enumerating objects: 2670, done. remote: Counting objects: 100% (2670/2670), done. remote: Compressing objects: 100% (106/106), done. remote: Total 2670 (delta 2564), reused 2669 (delta 2564), pack-reused 0 (from 0) Receiving objects: 100% (2670/2670), 15.34 MiB | 4.99 MiB/s, done. Resolving deltas: 100% (2564/2564), completed with 2564 local objects. From https://github.com/BennyThadikaran/eod2_data + 5497e2037...9a6d5eae5 main -> origin/main (forced update) + a7f4c7fd5...90aa3f711 dev -> origin/dev (forced update) hint: You have divergent branches and need to specify how to reconcile them. hint: You can do so by running one of the following commands sometime before hint: your next pull: hint: hint: git config pull.rebase false # merge hint: git config pull.rebase true # rebase hint: git config pull.ff only # fast-forward only hint: hint: You can replace "git config" with "git config --global" to set a default hint: preference for all repositories. You can also pass --rebase, --no-rebase, hint: or --ff-only on the command line to override the configured default per hint: invocation. fatal: Need to specify how to reconcile divergent branches. Thanks Saurabh -- Reply to this email directly or view it on GitHub: #176 (comment) You are receiving this because you commented. Message ID: @.> Regards, Benny Thadikaran

2670 files changed, 1326169 insertions(+), 1346162 deletions(-) create mode 100644 daily/galaprec.csv create mode 100644 daily/jeyyam_sme.csv delete mode 100644 daily/lghl_sme.csv delete mode 100644 daily/reliable_sme.csv delete mode 100644 daily/tarachand_sme.csv delete mode 100644 daily/thejo_sme.csv (.venv) saurabhgarg@MacBook-Pro eod2_data % cd .. (.venv) saurabhgarg@MacBook-Pro src % ls data dget.py error.log nseBhav nseIndices plot.py defs eod2_data init.py nseDelivery nse_cookies.pkl plugin (.venv) saurabhgarg@MacBook-Pro src % python3 init.py sh: color: command not found INFO: 11-09-2024 08:42 - defs.defs - All Up To Date. Check again after 07:00PM for today's EOD data

Is it fine now , should it not have downloaded the yesterday eod file and synced it?

Thanks Saurabh

BennyThadikaran commented 1 month ago

Its already updated till yesterday. Nothing more to do. 👍

On 11 September 2024 03:17:38 UTC, Saurabh @.***> wrote:

Just run git reset --hard HEAD~1 followed by git pull. First command will remove the last commit and discard any changes. Finally pull the latest changes. On 11 September 2024 02:28:57 UTC, Saurabh @.> wrote: Hello Benny, Getting this after the update. (.venv) @. eod2 % git pull remote: Enumerating objects: 10, done. remote: Counting objects: 100% (10/10), done. remote: Compressing objects: 100% (6/6), done. remote: Total 8 (delta 6), reused 2 (delta 2), pack-reused 0 (from 0) Unpacking objects: 100% (8/8), 2.15 KiB | 274.00 KiB/s, done. From https://github.com/BennyThadikaran/eod2 9ce73e7..51a7744 main -> origin/main 8800d55..3d85cad dev -> origin/dev Updating 9ce73e7..51a7744 Fast-forward src/eod2_data | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (.venv) @. eod2 % cd src/eod2_data (.venv) @. eod2_data % git reset --hard HEAD Updating files: 100% (2598/2598), done. HEAD is now at 5497e2037 Merge pull request #70 from BennyThadikaran/dev (.venv) @. eod2_data % git clean -f (.venv) @. eod2_data % git checkout main Already on 'main' Your branch is up to date with 'origin/main'. (.venv) @. eod2_data % git pull remote: Enumerating objects: 2670, done. remote: Counting objects: 100% (2670/2670), done. remote: Compressing objects: 100% (106/106), done. remote: Total 2670 (delta 2564), reused 2669 (delta 2564), pack-reused 0 (from 0) Receiving objects: 100% (2670/2670), 15.34 MiB | 4.99 MiB/s, done. Resolving deltas: 100% (2564/2564), completed with 2564 local objects. From https://github.com/BennyThadikaran/eod2_data + 5497e2037...9a6d5eae5 main -> origin/main (forced update) + a7f4c7fd5...90aa3f711 dev -> origin/dev (forced update) hint: You have divergent branches and need to specify how to reconcile them. hint: You can do so by running one of the following commands sometime before hint: your next pull: hint: hint: git config pull.rebase false # merge hint: git config pull.rebase true # rebase hint: git config pull.ff only # fast-forward only hint: hint: You can replace "git config" with "git config --global" to set a default hint: preference for all repositories. You can also pass --rebase, --no-rebase, hint: or --ff-only on the command line to override the configured default per hint: invocation. fatal: Need to specify how to reconcile divergent branches. Thanks Saurabh -- Reply to this email directly or view it on GitHub: #176 (comment) You are receiving this because you commented. Message ID: @.> Regards, Benny Thadikaran

2670 files changed, 1326169 insertions(+), 1346162 deletions(-) create mode 100644 daily/galaprec.csv create mode 100644 daily/jeyyam_sme.csv delete mode 100644 daily/lghl_sme.csv delete mode 100644 daily/reliable_sme.csv delete mode 100644 daily/tarachand_sme.csv delete mode 100644 daily/thejo_sme.csv (.venv) @. eod2_data % cd .. (.venv) @. src % ls data dget.py error.log nseBhav nseIndices plot.py defs eod2_data init.py nseDelivery nse_cookies.pkl plugin (.venv) @.*** src % python3 init.py sh: color: command not found INFO: 11-09-2024 08:42 - defs.defs - All Up To Date. Check again after 07:00PM for today's EOD data

Is it fine now , should it not have downloaded the yesterday eod file and synced it?

Thanks Saurabh

-- Reply to this email directly or view it on GitHub: https://github.com/BennyThadikaran/eod2/issues/176#issuecomment-2342536135 You are receiving this because you commented.

Message ID: @.***> Regards, Benny Thadikaran