Get trader to profitability in at least one market

[owocki]

One could presumably increase the margin for a classifier such that, to classify a sequence of price movements as a BUY or SELL, the predicted movement must be higher than what it is now (0.002% increase/decrease threshold)

[owocki]

ACK if you are interested in contributing (one could contribute in the following ways: code, servers, test capital, intros to wall st quants).

[NDuma]

@owocki We've recently set up a Slack group where there are a few people working on separate API angles for money management and trading; starting with Polo. One of them is working with NeuPy for MLP NNets. Would like to help with where I can; and, see if anything they are doing might aid this project as well.

[hft-singularity]

Thank you for open sourcing the project but.. lets be honest here. You only open sourced it because you couldn't make it profitable and you wait for someone that will do it for you, but even if anyone will manage to do it using your code, why share it with you? You wouldn't open source it, if it wasn't profitable yourself. That's why there is no other open sourced solution, that is profitable and never will be. Trust me i know a lot in this topic.

And most important point, profitable trading software while open sourced will become non-profitable when hundreds/thousands will start using it. Period. Every team that made profitable trading software will tell you that.

It's really a fun toy project, but nothing more. Fees are not a problem at all, doesn't matter if fee will go up or not it's just another variable and proper algorithm will adapt to new reality and you will still make money.

I can tell you what you need for profitable trading software:

1. Minimum millions of dollars in infrastructure.
2. Minimum millions of dollars for software engineers.
3. Millions of dollars for ML specialists, but not someone out from school, i am talking about people which names you can see often on published papers in the ML field. You know how much research is being done on trading software and not published? you would be amazed what kind of results those research give and how top trading algorithms look like. Unfortunately that's all i can say because of NDA.

And then months of hard work of a world class team developing state of the art software that will NEVER be open sourced.

Good luck anyway!

[grahamannett]

@hft-singularity what do you get out writing a post like that?

[hft-singularity]

@grahamannett

You mean showing how real life looks like ? And showing what you need to expect when you try to achieve something that complex like successful trading software? or what exactly you mean by "writing a post like that" ?

Don't believe me ? You don't need to, ask anyone that worked in a team like i described. Just ask. Or we can meet here in a year from now and see if what i said was true. Which ever works better for you.

[grahamannett]

@hft-singularity where did I disagree with you? why are you being incredibly hostile and belligerent over a public github repo?

[hft-singularity]

@grahamannett

I was harsh, but hostile? if you felt i was hostile toward you then i am sorry, it wasn't my intent.

[amingilani]

@grahamannett oh man, I was totally looking forward to tinkering with something like this. Only, I can't do python, and my job keeps me from spending too much time on fun projects.

@hft-singularity aren't you a game theorist. Regardless, you're wrong, some people like helping others on the internet, especially for profit. Also, collaboration is the whole point of open source. Your comment wasn't constructive or helpful in any way.

[zenlambda]

A LICENSE file might help when inviting contributions, so people know what the ground rules are.

[kordless]

@hft-singularity you really should stop speaking for others. It's a bad habit and discouraged here.

Joined on Mar 26, 2016

Figures.

[owocki]

@NDuma yes, please send an invite to ksowocki@gmail.com!

@hft-singularity Thanks for your comment. You posit an interesting question about human nature, and about motivations in open source software.

If someone gave you an half-written instruction manual and a toolkit that could turn an ordinary goose into a golden goose, (1) Would you have a go at it? (2) Would you share or hoard the golden eggs if you figured out the rest of the instructions? (3) If you did share them, would the sum of the contributions compound and create more profit for everyone? Would the global markets for golden eggs come crashing down due to increased supply? And if so, how soon until that happens?

I do not presume to know the answer, but the undergraduate anthropology minor in me says that watching the 'pull requests' tab of this repo will yield some interesting clues. You are welcome to open another Issue and continue debating this there.

@amingilani python is easy to pick up. i'll happily coach you through it!

@zenlambda thanks, ill open up another issue; hopefully i can get to that today.

[ghost]

I have an idea related to this paper published in 2014, " http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2259133 ". I hope to post more in a bit when I finish working on it, but for now thanks for releasing this information! :) I hope to be able to contribute more in a bit.

[rmendes900]

Hi @owocki , I was checking your code and looks very nice by the way. The problem is that you missed a major vector that is the trader strategy according to the market sentiment. Probably if your learning machine was doing the tests considering that, at this time you supposed to be with a lot more coins. Well, I am a Trader and Coder, I can help you on that. Let me know. I will fork your code and see how can I contribute.

[prathik]

@hft-singularity Would I open source a profitable trading code? Only if I trusted the community to add more value to the code and help improve the algorithm and detect bugs. It doesn't matter if there are a few black sheep given that the overall number of contributors add value to the code, it is the I scratch my back and you scratch mine principle.

[jeff-hykin]

Hey @owocki

Your work is pretty impressive, that many transactions while statistically breaking even shows at least some promise. If anything it seems like just tweaking the code to refuse trades more often (higher confidence threshold) would result in something closer to a profit.

I was considering something similar two days ago after talking with another machine learner (one most likely similar to yourself). We came to the conclusion that just using price data, there isn’t a successful way to beat out the market (for obvious reasons some of which @hfs-singularity discussed). I’m completely with you though on creating one for the experience and fun of it along with potential profits. I’ll definitely be looking over the code in the next few days.

[cavemangraham]

@rmendes900 That was my first thought too. Perhaps adding market sentiment data pulled from news/reddit/twitter could be added to help refine the "confidence levels" of trades during certain time periods.

[owocki]

@jeff-hykin @cavemangraham we're doing some discussion of this on the slack channel https://github.com/owocki/pytrader/issues/23 if you guys would like to join in. i'll likely be putting some time in on this this weekend.

[kristerfl]

All I am saying is that Synergetics (the original stuff of "Bucky" Fuller) is the key; Tensegrity structures of the market, spherical hierarchy at al.

This is definatelly what that famous person; forgot his name; who became rich at NYC Stock used as analysis, no co-incidence it happened after Synergetics Vol 1. was published.

It's not so much millions of money as stated above her as millions of brain-cells understanding the 4000 pages of Synergetics Vol 1 and 2. Which when you have a project like this it becomes more apparent what kind of tool it can be. At least that's how it went for me.

You may complain Synergetics is about Energy not money but which is the the difference between the two, that is even stated in the actual books; which subtitles are "explorations in the geometry of thinking"..

Just my 2 cents.

[TheDeafMute]

Hi - I don't see any code?

[Snipa22]

@TheDeafMute Code's available on this github repo https://github.com/owocki/pytrader. Some basic install instructions are in the main README.md.

[TheDeafMute]

ty

[jpramirez]

ey Guys, i have been following both the code growth and evolution and this comments, i will join the channel if it's still up. But, im not seeing any comments on Social intelligence, Bitcoin and, any other Blockchain open public distributed markets are not followed by any mathematic rules only. Social Quantification is a must here, Blockchain prices are attached to mass sociology behavior, and in order to achieve a more certain price prediction, news outlets, and overall chatter needs to be studied... I will be working on a separate module to add at least an index of variation that includes the rest of the sources needed to make a valid predicition.

[owocki]

@jpramirez https://github.com/owocki/pytrader/pull/75 is the latest on the social data front. we've got the system pulling social data, but nothing open source that puts it into the model yet.

[jpramirez]

Ok, i will work something out and throw some ideas in there. Will be fun, thanks!

[kristerfl]

Don't forget this?

https://github.com/kylehg/sentiment-analysis

łł-ĸkĸĸxĸxĸ

Krister Lilleland kristerfl@operamail.com

On Tue, May 3, 2016, at 02:20 AM, Juan Pablo Ramirez wrote:

Ok, i will work something out and throw some ideas in there. Will be fun, thanks! — You are receiving this because you commented. Reply to this email directly or view it on GitHub[1]

Links:

1. https://github.com/owocki/pytrader/issues/1#issuecomment-216402472

[photonherder]

Maybe trade on more than one exchange and go for arbitrage?

[ai-nick]

so you start at 1 btc the net(in reference to fees not tax) profit is 2.441 (200+%), so actually your able to predict with a great deal of accuracy your just getting literally eaten alive by fees, since its polo you change the granularity of the time series in your api call so maybe trying longer ticker lengths and cherry picking the coins that have enough history to support the same levels of accuracy with an equally scaled forcast you should be able to profitable by basically training to hit bigger, longer spreads with more capital to avoid fees, idk just my two cents

[heyJonBray]

That many transactions should have only been executed in a backtest environment, where the historical candles are fed to the program in sequence so it has no knowledge of the next trades. On any exchange, particularly ones which utilize a proprietary token for transactions, executing that many trades will result in so many fees that you're completely eating up your profits.

While 20k+ trades is necessary to determine confidence, a realistic profitable strategy, even one that traded many different asset pairs, should have multiple orders of magnitude less trades, 4-6 per asset per 2 month period unless you get good scalping channels in which case we're looking at that number going up to 20, if that.

[mellertson]

I am working on resurrecting this project. I have two servers in a data center, which we can use to run PyTrader on, inside docker containers. Additionally, I see there are several cron jobs, which look like they do things like update balances, get data, and the like. For another project, which I own, I have developed Celery tasks (aka services) which use WSS to get data in real-time from multiple exchanges. I should be able to replace the cron jobs with real-time data from WSS feeds, which should give us the capability to make the trading strategy more responsive.

That being said, I see one very simple solution to get this algorithm to profitability. Just take it as is (meaning don't adjust any ML settings nor trade strategy) and scale it out to multiple markets on multiple exchanges. And then reduce the fees to as close to zero as possible. There is a is a way to just that. The following explains how to reduce the fees.

Most exchanges use a fee schedule, where fees are dynamically reduced if trading volume is above a given threshold for the previous 30 days. As a reference, I have copied the fee schedule for Bittrex and Coinbase Pro below.

Bittrex Fee Schedule

Bittrex's official fee schedule is published here.

Coinbase Pro

Coinbase Pro's official fee schedule is published here.

Proposed Solution

If we invested enough to do more than $50K in trade volume on Bittrex and more than $10K on Coinbase Pro the fees would be reduced. This seems like a simple arithmetic problem to me where we just need to figure out what trading volume level we need to be at to reduce the fees enough to make the strategy profitable. Then just iterate over the ML and trade strategy code to make incremental improvements, which should in theory increase profits.

In his book Advances in Financial Machine Learning Dr. Marcos Lopez De Prado, who manages several multibillion-dollar funds using ML algorithms, articulates in the first chapter why many ML strategies fail. Here's an excerpt found in section 1.2.2,

If you have been asked to develop ML strategies on your own, the odds are stacked against you. It takes almost as much effort to product one true investment strategy as to produce a hundred, and the complexities are overwhelming: data curation and processing, HPC infrastructure, software development, feature analysis, execution simulators, backtesting, etc. Even if the firm provides you with shared services in those areas, you are like a worker at a BMW factory who has been asked to build an entire car by using all the workshops around you. One week you need to be a master welder, another week an electrician, another week a mechanical engineer, another week a painter... You will try, fail, and circle back to welding. How does that make sense?

It’s a huge undertaking, and I know because what Dr. De Prado described has been my experience these last 6 years as I have endeavored to do what it looks like many of you have also worked to do in this project, and perhaps other projects on your own.

In section 1.3.1 Dr. De Prado goes on to identify six roles needed to produce an effective strategy. They are:

1. Data Curators
2. Feature Analysis
3. Strategist
4. Backtesters
5. Deployment Team
6. Portfolio Oversight

Two of my former colleagues have a great deal of expertise in Portfilio Oversight. They worked for years on the floor of Wall Street, so they are quite knowledgeable. I on the other hand have expertise in the areas of Data Curators, Backtesters, Deployment Team, and I own two servers which we can use. And I will gladly chip in for a third if needs be. But, I am just one guy and cannot effectively fulfill all three roles I mentioned. If you guys are truly serious about making this project work, I propose we all join forces and brace ourselves. Because, in addition to server infrastructure, a well organized team, and smart minds, it will take lots of capital to make this work, as the results from Kevin Owocki's excellent work seem to indicate, which he has generously shared with us here.

So, this begs the question, "how do we raise millions of dollars to invest in an ML algorithm?" Well, what if we create a ledger, which is published to all investors / contributors indicating who has invested how much and when. And we all pool our money together in a sort of crowd sourced investment fund? My understanding is we can invest up to $100K without requiring financial certifications by the SEC (like a Series 7). And if we were able to raise that kind of money, I dare say we could attract someone who is Series 7 certified, so that we could legally accept investments greater than $100k. I even know a couple of great guys who I would be willing to talk to if there is sufficient interest.

If enough people are interested in helping with the work, and this project is resurrected I would suggest we all brace ourselves because I imagine many who work in the financial sector would love to see us fall on our own swords. But, there is strength in numbers and with so many bright minds working together I have to believe we could create something truly amazing together.