search

matplotlib / mplfinance

Financial Markets Data Visualization using Matplotlib
https://pypi.org/project/mplfinance/
Other
3.48k stars 613 forks source link

Feature Request: Candlestick Footprint Chart #572

Open quantfreedom opened 1 year ago

quantfreedom commented 1 year ago

i want to create a footprint chart like this one where i am able to make an open high low close but also show how much trading volume was at each y value of the candle

here is an example of what i am talking about

image

its very easy to get the data ... getting and understanding how to organize the data on our end is very very easy ... you can see what a csv file would look like here ... https://public.bybit.com/trading/BTCUSD/

lets say i wanted to create a 5 min candle ... for the info i would just add up all the sell orders at each price point of the candle and do the same for the buy orders ... then i have a dataframe within the candle that tells me the sell column and buy column ... and you can get the open high low close even from that data ... the open is the first trade at the start of the candle then the high is the highest point ... low is the lowest point and the close is the last candle ...

this info helps us visualize at what price point in the candle has the most trade volume ... footprint charts are used in high level trading ... I will be making a ton of videos on this subject if we can get one of these open source plotting platforms to help create something like this

all we would need is send dataframes inside a master dataframe ... so you have the overall dataframe that houses all of the candles and inside each candle is a dataframe of all of the sell and buy data at each price of the candle

ultimately would be nice for it to look like this ... but for now the top one would be ultra sexy image

DanielGoldfarb commented 1 year ago

@The1iAmNeo Very interesting! I will take a closer look in a week or so (busy with another project for the next week or so). At that time I will look into how best to implement this, and where it should fall in priorities. How are your python coding skills? Perhaps you can help with some of the coding? Thanks. All the best. --Daniel

quantfreedom commented 1 year ago

@DanielGoldfarb hey thanks for taking an interest ... it would really help out the algo trading community so much.

my coding skills aren't the greatest but would love to chat and see what i could help with ... only been coding for like 2 months so not sure what i could do lol ... but i am willing to try because having this would be a life saver

quantfreedom commented 1 year ago

@DanielGoldfarb hey ... wanted to check back in and see if you were still interested in doing this?

kzaheer09 commented 1 year ago

Would be interested to see this up and running, please let know how I can help , not a good coder in python though

andrewrgarcia commented 1 year ago

@QuantFreedom1022 I'd like to help to code this Candlestick Footprint Chart method, but I need some clarification. Are we treating the bid-traded and ask-traded volumes as those indicated by the Buy and Sell labels in the 'side' column of the provided csv files? Also, is there a specific number of market orders that must be plotted for each footprint or a rule associated with such?

quantfreedom commented 1 year ago

@DanielGoldfarb ... wanted to check back in with you on this ... would like to start working on this soon

@andrewrgarcia yeah i am down to work together on this ... i already have the code and the csv files needed ... just need the plotting to do its job

DanielGoldfarb commented 1 year ago

I am very involved right now working on a project for Jupytercon. I am happy for you two to begin moving forward with this, and I can put some time into it. I would like to see some details regarding design and what you are planning, so as to fit nicely into the big picture and long term plans for mplfinance. You can post here, or if you prefer, email me at dgoldfarb.github@gmail.com

The first thing I'd like know is what is the minimum expectation for the data (what columns are a must; which are optional). If the data is going to look like this, https://public.bybit.com/trading/BTCUSD/, then it seems to me that it doesn't necessarily fit into the standard mpf.plot(df,**kwargs) function, which expects simply OHLC(V) data. Perhaps we write a wrapper around mpf.plot()? Or, if you have in mind just adding a new type= to mpf.plot(), maybe that can work as well, but I'd like to see what that might look like inside.

Looking forward to hearing from you. All the best. --Daniel

andrewrgarcia commented 1 year ago

@DanielGoldfarb ... wanted to check back in with you on this ... would like to start working on this soon

@andrewrgarcia yeah i am down to work together on this ... i already have the code and the csv files needed ... just need the plotting to do its job

@QuantFreedom1022 Nice. You can start by attaching the files and the code I'll see what I can do. If you're familiar with Git, I can open a new branch and you can add your material on the fork, your choice.

quantfreedom commented 1 year ago

@andrewrgarcia and @DanielGoldfarb ... here is the link to my github repo for the footprints ... and also here is a quick loom video of me explaining a little of how it works https://github.com/QuantFreedom1022/Footprints https://www.loom.com/share/76ccbf0a46374256b798f2a237f41f78

andrewrgarcia commented 1 year ago

@andrewrgarcia and @DanielGoldfarb ... here is the link to my github repo for the footprints ... and also here is a quick loom video of me explaining a little of how it works https://github.com/QuantFreedom1022/Footprints https://www.loom.com/share/76ccbf0a46374256b798f2a237f41f78

@QuantFreedom1022 I forked your repository here to shorten and revise your footprint code. My revised version does not use numba yet it runs faster than your former code. It also removes the sell / buy count issue you mention in your video and thus, appropriately reduces the trade count (see csv output screen captures for the revised code and your code). Still, please check the code and provide me some feedback if needed before I submit a PR to merge it to your repository

quantfreedom commented 1 year ago

@andrewrgarcia WOW ... ummm so this code is way way way above my skill level lol ... so i feel it almost impossible to give any feedback lol ...

but i did take your code and add it to my repo so that way we have your code and i also have my code ... but yeah lets move forward with yours seeing as how it is faster and better in all aspects.

also i would love to jump on a quick 10 or so screen share call to talk about what we are both thinking as far as moving the project forward now that we have a function that can organize the data to start to build the footprint charts

andrewrgarcia commented 1 year ago

@andrewrgarcia WOW ... ummm so this code is way way way above my skill level lol ... so i feel it almost impossible to give any feedback lol ...

but i did take your code and add it to my repo so that way we have your code and i also have my code ... but yeah lets move forward with yours seeing as how it is faster and better in all aspects.

also i would love to jump on a quick 10 or so screen share call to talk about what we are both thinking as far as moving the project forward now that we have a function that can organize the data to start to build the footprint charts

@QuantFreedom1022 Hey, it's possible you're new to open source. Can you please remove the andrew_footprint.py file you created by copying my code and instead approve my Pull Request (PR) ? I just sent you one. (My PR keeps your .ipynb code intact)

Best.

quantfreedom commented 1 year ago

@andrewrgarcia ok i just deleted your file and approved your pull request ... so everything should be up to date ... again for me i would prefer a quick 10 min screen share call for next steps but its up to you ... if you would like to setup a time to talk you can let me know your email and we can set that up or we can just do it here.

if not then what are your thoughts on what to do next

andrewrgarcia commented 1 year ago

@andrewrgarcia WOW ... ummm so this code is way way way above my skill level lol ... so i feel it almost impossible to give any feedback lol ...

but i did take your code and add it to my repo so that way we have your code and i also have my code ... but yeah lets move forward with yours seeing as how it is faster and better in all aspects.

also i would love to jump on a quick 10 or so screen share call to talk about what we are both thinking as far as moving the project forward now that we have a function that can organize the data to start to build the footprint charts

@QuantFreedom1022 I think the biggest challenge now would be on how to lump the timestamps into groups with the volume data so that footprints are made. My first thought was to use an argmax and/or argmin function, but if you have any other ideas and/or can sketch it, that would help.

The other thing would be on the side of the new footprints.py code to make it more lean. Your initial algorithm used several global variables, and because of the lengthiness of it, the new code inherits all these variables as constructors to the Printer class i.e. the variables below the init() function; some of these variables do not need to be constructors as some of them may be local to the specific code section they are used on. It would be great if you could identify the ones which do not need to be so "global" as this was originally your algorithm.

quantfreedom commented 1 year ago

@andrewrgarcia okay well I'm going to have to take some time to read through your code and figure out exactly what it's doing because I've never done object-oriented programming before so I have to educate myself a little bit and then once I feel like I have a grasp of that I'll make a loom video explaining anything that I see and also maybe some suggestions as well on how to put the footprints together

But this would be good because I'm at a point with my other project that I have to learn an object oriented programming so this is a good simple code to practice figuring out what's going on

andrewrgarcia commented 1 year ago

@andrewrgarcia okay well I'm going to have to take some time to read through your code and figure out exactly what it's doing because I've never done object-oriented programming before so I have to educate myself a little bit and then once I feel like I have a grasp of that I'll make a loom video explaining anything that I see and also maybe some suggestions as well on how to put the footprints together

But this would be good because I'm at a point with my other project that I have to learn an object oriented programming so this is a good simple code to practice figuring out what's going on

@QuantFreedom1022 That's great. The second paragraph is more of a request. I would prioritize the stuff I mentioned on the first paragraph (ideas/algos/drawn sketches for the lumping of timestamps with volumes). This can be done without learning the new code and more by looking at the csv that you generate with your former code and how to put the info together for the footprints

quantfreedom commented 1 year ago

@andrewrgarcia ok sounds good ... ill have a look at it as soon as I can and let u know any ideas I have

andrewrgarcia commented 1 year ago

i want to create a footprint chart like this one where i am able to make an open high low close but also show how much trading volume was at each y value of the candle

here is an example of what i am talking about

image

its very easy to get the data ... getting and understanding how to organize the data on our end is very very easy ... you can see what a csv file would look like here ... https://public.bybit.com/trading/BTCUSD/

lets say i wanted to create a 5 min candle ... for the info i would just add up all the sell orders at each price point of the candle and do the same for the buy orders ... then i have a dataframe within the candle that tells me the sell column and buy column ... and you can get the open high low close even from that data ... the open is the first trade at the start of the candle then the high is the highest point ... low is the lowest point and the close is the last candle ...

this info helps us visualize at what price point in the candle has the most trade volume ... footprint charts are used in high level trading ... I will be making a ton of videos on this subject if we can get one of these open source plotting platforms to help create something like this

all we would need is send dataframes inside a master dataframe ... so you have the overall dataframe that houses all of the candles and inside each candle is a dataframe of all of the sell and buy data at each price of the candle

ultimately would be nice for it to look like this ... but for now the top one would be ultra sexy

image

I did this but in a more elegant way (my last PR in candle builder.py). Instead of a dataframe within a master dataframe, the code creates a "3-d" dataframe so to speak, where each open and close time is now unique and each entry for every other column is now a list of ask bid and volumes thereof. The next steps would be making the actual graphs and answering the questions I posted in this new PR.

Philip030601 commented 7 months ago

Hi all, my name us philip and loojs like im comming in late. I was planning on coding some footprint logic and came accross this project. Do u guys still need help? Im an intermediate developer at best but very resourceful and have a tendency to get things done. Look forward to you reply/ies.

Much thanks

quantfreedom commented 7 months ago

Hi all, my name us philip and loojs like im comming in late. I was planning on coding some footprint logic and came accross this project. Do u guys still need help? Im an intermediate developer at best but very resourceful and have a tendency to get things done. Look forward to you reply/ies.

Much thanks

@Philip030601 yeah man that would be great ... i need to get back into this and working with someone else on it would help me get back into this project ... could you send me your discord or telegram or email or something and then we can setup a time to chat about what is needed to be done and share some ideas on what we want it to look like

Philip030601 commented 7 months ago

Hi quantfreedom.

Hope your doing well. Im in pst time and available to talk after 6pm almost every night. Let me know of your availability and will plan on it.

Thanks

On Tue, Nov 14, 2023, 6:48 PM Quant Freedom 1022 @.***> wrote:

Hi all, my name us philip and loojs like im comming in late. I was planning on coding some footprint logic and came accross this project. Do u guys still need help? Im an intermediate developer at best but very resourceful and have a tendency to get things done. Look forward to you reply/ies.

Much thanks

@Philip030601 https://github.com/Philip030601 yeah man that would be great ... i need to get back into this and working with someone else on it would help me get back into this project ... could you send me your discord or telegram or email or something and then we can setup a time to chat about what is needed to be done and share some ideas on what we want it to look like

— Reply to this email directly, view it on GitHub https://github.com/matplotlib/mplfinance/issues/572#issuecomment-1811725537, or unsubscribe https://github.com/notifications/unsubscribe-auth/AYZZYII7KWKIOV7FZKURBNLYEQUP5AVCNFSM6AAAAAASLTJOBWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMJRG4ZDKNJTG4 . You are receiving this because you were mentioned.Message ID: @.***>

quantfreedom commented 7 months ago

@Philip030601 ok ... let me know a contact for you that would be good ... either telegram, email or discord

Philip030601 commented 7 months ago

@Philip030601 ok ... let me know a contact for you that would be good ... either telegram, email or discord

Hi quant. My email is sflush369@gmail.com.

quantfreedom commented 7 months ago

@andrewrgarcia not sure if you are still interested in this but i am picking the project back up ... and i now understand about classes and all the stuff i didn't know about 6 months ago ... so i can actually be useful this time ... let me know if you are still interested in trying to finish this

quantfreedom commented 7 months ago

I found a person who has something close to what i want ... you can check out the project here https://github.com/murtazayusuf/OrderflowChart

andrewrgarcia commented 7 months ago

I am happy to contribute with you guys, but contingent on 1) that we all follow good open-source / version control practices so that we all get credit for our contributions and 2) we start from scratch with an object-oriented structure with concise code with little to no additional dependencies (packages). Mplfinance is a lean package and does not use numba, which in principle can make it quite heavy. In addition, I didn't find the implementation of numba helpful as I made it just as fast without it. Code cannot be clunky, it needs to be sleek

quantfreedom commented 7 months ago

@andrewrgarcia yeah i am happy to do so

1: this will be my first time trying to work with someone over the internet with only typing and also using github for version controls and practices and all that stuff ... so if you are willing to work with someone that is 100% new to this then i am fine but i have absolutely no idea about what good version control practices are ... i don't even know rule number one

2: yes i can do this but the concise code with little to no dependencies not sure ... all i know is i want to use numpy and as little pandas as possible because pandas is so slow ... but you are smarter than me so i am ok with using what ever dependencies that you want

We already have the project open at https://github.com/QuantFreedom1022/Footprints .. but i have no idea where to start ... or where should we start talking about this idea?

if you could lead this that would be great because i had no idea how to lead a project with only typing over the internet ... i am more of a lets hope on zoom ... talk for 15 mins and then get to work

KMey-create commented 7 months ago

Hi there, I will need some help with this, where can I get this type of data? I was trying Tradermade.com, but there are Bid/Ask/Mid columns

quantfreedom commented 7 months ago

@KMey-create i get mine from binance https://www.binance.com/en/landing/data

Philip030601 commented 7 months ago

Quant,

Happy belated thanksgiving. I thought the data had to be time and sales type in tick format.

Regards,

On Fri, Nov 24, 2023, 2:23 PM Quant Freedom 1022 @.***> wrote:

@KMey-create https://github.com/KMey-create i get mine from binance https://www.binance.com/en/landing/data

— Reply to this email directly, view it on GitHub https://github.com/matplotlib/mplfinance/issues/572#issuecomment-1826114089, or unsubscribe https://github.com/notifications/unsubscribe-auth/AYZZYIMW337VL2V47EVVNWDYGEM4NAVCNFSM6AAAAAASLTJOBWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMRWGEYTIMBYHE . You are receiving this because you were mentioned.Message ID: @.***>

Vimal-Seshadri-Raguraman commented 7 months ago

@QuantFreedom1022 @andrewrgarcia Hello everyone, I recently stumbled upon this discussion while exploring ways to create a footprint chart in Python, specifically for stock data, not cryptocurrency. I'm eager to contribute and offer assistance in any way I can. While I wouldn't classify myself as an expert programmer, I believe I have a good understanding and can navigate my way through this. Feel free to reach out to me at rsehshadri@gmail.com for any further discussion or help.

quantfreedom commented 7 months ago

@Vimal-Seshadri-Raguraman i don't think Andrew will be working in this but I am still interested ... pls email me and we can setup a time to talk ... quantfreedom1022@gmail.com

jamesdidit72 commented 1 week ago

Has there been any progress with this?

quantfreedom commented 1 week ago

@jamesdidit72 image we have made our own version if you or anyone else wants to help build it out more, you can join my discord and message me there ... the link to my linktree with my discord is on my github profile

once the concept is built out more would be nice to present it to matplotlib and see if we could work together on this idea