Closed guanlisheng closed 8 years ago
I think I have worked out a way of achieving this by introducing a new table: TRANSFER_TRANS_V1
This table would tie the CHECKINGACCOUNT_V1 table to any other account type. This would extend the transfer functionality of the CHECKINGACCOUNT_V1 table, allowing transfer transaction to other account types such as stocks or assets. This would also allow purchase or sale from these account types by transferring money from existing accounts.
This table could also maintain a link to the currency, to allow a transaction to change its currency.
I am still working out the details before I put it into practice.
why a brand new table here ?
Continuing from issues: Support transfer operation for stock account #326, Assets should be closable or have an end date #361, and
As stated, Assets and Stocks require similar behaviour, even though they appear to be different. I will explain in next comment. This is the reason for closing the other issues.
Assets Assets are created in the asset register. We can think if each record of an asset register as an account. As an account, we can buy, or sell this item, or even generate income from this item or close it. In order to do this we need to maintain transactions in a transaction table.
Stocks Stocks are generated as an Account. In each Stock account we add Shares to maintain a Stocks portfolio. Similar to assets, we can think of these shares as Share accounts belonging to a Stock account, where we buy shares, sell shares or generate income from shares in the form of dividends. We can even close one share account within the stock account. In order to do this we need to maintain transactions in a transaction table.
From both Stocks and Assets we come to the same conclusion. We need to maintain transactions in a transaction table. This is why all the requests are similar, and can all be achieved with one solution.
We already have a transaction table which we call "CheckingAccount_V1". To maintain backward compatibility in the database, we require minimum disruption to the existing tables. The simplest and clean way is the introduction of a new table, as suggested in the feature (Stocks And Assets): expand functionality.
This table unites transactions with existing accounts, and allows transfers or straight transactions to occur for stocks, assets or anything else in the future.
I have no strong preference here for buying and selling. my question is that you just closed a request issue to support transfer for stock . Thought?
oops, I suspect I closed it accidently. This issue is not supposed to be closed.
Initial setup for this set of features has now been implemented, and can be reviewed on https://github.com/moneymanagerex/moneymanagerex/tree/stef145g_dev
This is by no means complete, but demonstrates the approach taken. For Assets, there is still another panel to be created that will display all transactions for an individual asset, as well as refining the user interface functionality.
Stocks will work in a similar way, and only the underlying structure is set up. The user interface will work in a similar way, but a bit more complicated for the user interface.
The whole implementation may take quite some time to complete.
This ticket is for stock and would we explain the approach for stock firstly. My feeling is that we are make things complex and complicated.
In order to find a solution, we need to understand the problem. Requests:
Problem:
\ Solution ** As shown in the appropriate diagrams, both have a similar structure. The additional table provides the linkage, as well as adding missing elements in the existing structure, shown marked in red.
Assets A single asset has many transactions contained in the existing checking table Provides a one to many transaction which link to an account.
Stocks A single Stock has many transactions contained in the existing checking table Provides a one to many transaction which link to an account.
The addition to the existing structure provides
There is nothing complicated about this solution, but it does solve a lot of issues. Combining all issues to this topic makes explanation easier. @guanlisheng Please provide your proposed solution. in a way that allows the coding of your solution.
Other interested users are welcomed in providing feedback on these issues, and some possible clarification as well.
The Stocks part of this new feature is now advanced enough to become useful. As a result, documentation has been provided to explain the way it works.
I thought you have stopped this incorrect approach and are focusing on the v1.2 release and UX improvement.
LiSheng
在 2015年3月8日,下午1:18,Stefano notifications@github.com 写道:
The Stocks part of this new feature is now advanced enough to become useful. As a result, documentation has been provided to explain the way it works.
— Reply to this email directly or view it on GitHub.
My approach is NOT incorrect. I have requested to provide your solution but have failed to do so, so I have continued my development as a side line, and for others to evaluate.
As I continue development, I am discovering the benefits of my approach, because it caters for all the items I need Stocks and Shares to accomplish. I have provided documentation in order to make it easer for others to evaluate this approach.
This solution has taken a long time in coming, mainly because of the original coding structure and the time required to develop. Thanks to the new internal structures which you (LiSheng) have assisted in providing, my original idea has now become much simpler to implement within the existing database structure.
I have left my approach in a separate personal branch so it becomes my responsibility to maintain. This way it does not become out-dated that it becomes difficult or impossible to merge into the master branch at some future date, because of coding conflicts, as has occurred with the previous development branch approach. This does not prevent others from switching to my branch and discovering what has been done, and provide positive feedback and constructive ideas for team working.
I the developer, will still have I, the user that will use this development, so my work will not be lost. As for 1.2.x release, I thought I made that clear in other communications.
I urge all other stake holders to express your opinion on this issue, either positive or negative. This way I know if I am developing this for strict personal use or the wider audience.
Stefano.
IMHO
First of all we need released stable 1.2.0 It's quite ready after some cherry-picks.
Then we should discuss how to continue to develop.
I suggest to release 1.3 as is then stop develop new features.
Then we should create new DB structure for support items like Buy/Sell Stocks, Dividents, Invoices Multy currency Assets etc. And after that - develop new features.
--Nikolay
2015-03-09 3:07 GMT+03:00 Stefano notifications@github.com:
My approach is NOT incorrect. I have requested to provide your solution but have failed to do so, so I have continued my development as a side line, and for others to evaluate.
As I continue development, I am discovering the benefits of my approach, because it caters for all the items I need Stocks and Shares to accomplish. I have provided documentation in order to make it easer for others to evaluate this approach.
This solution has taken a long time in coming, mainly because of the original coding structure and the time required to develop. Thanks to the new internal structures which you (LiSheng) have assisted in providing, my original idea has now become much simpler to implement within the existing database structure.
I have left my approach in a separate personal branch so it becomes my responsibility to maintain. This way it does not become out-dated that it becomes difficult or impossible to merge into the master branch at some future date, because of coding conflicts, as has occurred with the previous development branch approach. This does not prevent others from switching to my branch and discovering what has been done, and provide positive feedback and constructive ideas for team working.
I the developer, will still have I, the user that will use this development, so my work will not be lost. As for 1.2.x release, I thought I made that clear in other communications.
I urge all other stake holders to express your opinion on this issue, either positive or negative. This way I know if I am developing this for strict personal use or the wider audience.
Stefano.
— Reply to this email directly or view it on GitHub https://github.com/moneymanagerex/moneymanagerex/issues/336#issuecomment-77783606 .
@vomikan - I totally agree with release of 1.3.
Creating a new database structure will take some to develop, and will need to be released as V2.x.x together with database upgrading facilities if this is the new approach to be taken.
I think this is the correct approach (providing a full transaction history capability). From the comments that I have seen from users requesting the feature, I don't think the end users would be happy with the other proposed solution of just modifying the initial balance.
Users always request new features and we are the people to make decision on what and how to delivery. We should focus on existing and important thing instead of creating new feature (debt).
For interested users, continuing development for this has now been moved to https://github.com/stef145g/moneymanagerex/tree/stef145g_dev
Will the cash portion be handled in a (related) Checking account or in the Stock account? I see. There will be a separate checking account used.
It might be possible to release this in multiple steps. One could be just the stocks list/table. This could be used as a watchlist where prices can be tracked. Then the related transaction history can be added to it. We have a similar issue (https://github.com/moneymanagerex/android-money-manager-ex/issues/66).
@MisterY , I am not quite sure what you are asking here, but the Desktop version already has a "Watch list" as it were. Provided that the share symbol, is correctly entered as stock.exchange the button on the bottom right, will manually update all the stocks for this portfolio. (this account) There is no automatic periodic update feature as this would require the application to have this page constantly open, which would generally not happen.
Current implementation for Stocks works as follows: An account type of: Investment is created. This account is then listed in the navigation tree under the heading: Stocks
When the account is selected from the navigation tree, the Stock Investments panel is displayed. This is in effect can be used as a watch list which needs to be manually updated as required.
A Stock History option has also been provided and can be accessed by editing each company, and downloading the stock history. This can then be displayed in the Stock reports for the portfolio.
As an extra feature, I am providing the ability for each company within the portfolio to purchases or sell shares at different times, keeping a record of each transaction. A separate Shares account can be created, or existing accounts can be used to maintain share transactions stored in the existing CHECKINGACCOUNT_V1 table.
The portfolio view changes slightly. A single entry can now be used for share purchases. The number of shares is now the total of all the share purchases and/or sales. This allows the shares to show the original number, purchase price and the date of purchase. This gives a better representation for Stocks and Shares.
This is achieved by introducing a separate table which links the CHECKINGACCOUNT_V1 table record to a record in the new TRANSFERTRANS_V1 table. This new table then has a table type indicator which allows the record to be associated with another table and record within that table. The current associations are Assets, and Stocks. Extra fields have been provided for Stocks to store the extra information required for shares.
The Assets have been set up in a similar way to allow transfers between general accounts and Assets. This mechanism is flexible enough to allow the user to use the extra feature or continue using Assets as originally implemented.
I see. That makes sense as the history of trading will be available, as well as the list of all securities owned. One technical thing that is a bit confusing is why mix the Asset & Stock records when they have different fields, as you mention yourself? I believe that having them separate would make things simpler to understand and build upon further. Assets and Stocks are not related in any practical sense. That is if we see assets as a house, a car, etc. and stocks as the certificates of ownership of companies that are trated at stock exchanges.
My earlier question was more targeted to see your line of thinking so that we could start with a simple watchlist in the Android version, at first.
The actual implementation for Assets and Stocks is indeed different, at a dialog level, but the underlying code structure for the TRANSFERTRANS_V1 table is very similar.
This new table is generally a link table. Two tables could be used one for Stocks, and one for Assets, but this would have a lot of code duplication, because the Assets table would be a subset of the Stocks table. This link table could be then extended to possibly include other future table types.
Basically we are linking the existing multiple records of CHECKINGACCOUNT_V1 to a single record in either the Stocks table or the Assets table. The fact that I added extra fields for stocks, become possible future extensions to assets. I have also included a currency pointer the TRANSFERTRANS_V1 table for future use to provide currency information on a transaction basis. Not sure if this will become useful, but this was a past feature request for providing currency information on other transaction types.
New pull request issued: https://github.com/moneymanagerex/moneymanagerex/pull/462 Related issues addressed by this: https://github.com/moneymanagerex/moneymanagerex/issues/326, https://github.com/moneymanagerex/moneymanagerex/issues/361
Pull request closed. New solution required
Looking at other financial software packages, I'm only familiar with the way Quicken works and it covers many scenarios. I think they are hiding some layers on the surface (UI), while the base for investments are two accounts - one for cash transactions and one for portfolio. This is also pretty much what we have at the moment. Meaning that a similar approach could be adopted with very little pain.
Adding a linked cash account to an investment account covers the scenarios for transfers (https://github.com/moneymanagerex/moneymanagerex/issues/326). This cash account might be hidden from others, created by the user, but the transactions will still be visible in user accounts.
A portfolio table should be expanded so that the records there include a link to the purchase or a sale transaction in the cash account. These operations would be recorded as withdrawal/income in the cash account. The management of securities remains in the portfolio account. Calculation of the average price, etc. This is where things get a little bit complex with the specification which stocks to sell first from a batch of several purchases, stock splits, etc.
The concept should, of course, be expanded and we should try to work out some scenarios through this setup and see if it fits.
Some common scenarios are listed here. Please suggest more as you see fit. Also, have a look at the image below. It contains the concepts explained in this post.
Account identifier, type, notes:
These are cash transfers to/from investment account. We would create a regular Transfer transaction from MyBank to MyBroker. Both accounts are of Bank type. The bank account for investment cash (MyBroker) should have the name synchronized with the actual portfolio account of Investment type (MyPortfolio) or some other pointer to the cash account used for the portfolio. The investment cash accounts may or may not be hidden. They might not show up in the list/tree of regular bank accounts but should show in the selectors for transfer source/destination account.
Purchase of securities would include two linked transactions - cash and portfolio.
Sale of securities includes two or more linked transactions - one cash and one or more portfolio transactions, depending on Lots.
For display purposes, we would aggregate the sale records in MyPortfolio and show it as one investment operation even though there might be multiple records stored. (refer to the sale example in the image below)
For selling operation it is important to track the lots as different tax rates may be applied to short- vs long-term holdings.
According to this, a Stock Split operation would simply create a removal of existing stock portfolio for a symbol, and addition of new values (amount, price).
This does not include any cash transactions but only portfolio transactions. So, there is no cash amount being changed yet we still keep track of Lots. Relatively simple. We need to support both transfer to/from own accounts, as well as transfer to/from external accounts which do not exist in the database. Meaning we should also allow for addition/removal of stock records.
This is a specific Portfolio operation so that there is a link to a security symbol and the account in which it occurs. From this we can calculate the Yield of the holdings. There is also a cash transaction involved, that defines the amount and currency. The destination should be selectable, though. I have seen cases where income from certain funds purchased in one currency (EUR) gets deposited in another (USD) as this is how the fund pays the distributions. So, some tweaking should be allowed here in the UI.
For example, convertible shares mature and are converted to regular stock. This might be recorded as a simple sale of the old holdings and purchase of the new ones at a different price.
I hope this explains the concept graphically. The links to cash transaction, security symbol, and a lot can all be done with reference fields on Portfolio transactions.
All cash transactions go to MyBroker, while portfolio transactions are in MyPortfolio. For simplicity, the operations here refer to the same symbol. A lot of safety checking happens at the business logic level, like checking for available lots, the remaining stock amount in a lot, etc.
Portfolio (current Investment) account should contain the following information, based on the above:
... to be updated as the concept develops
I see that this solution is quite similar to the one proposed earlier by @stef145g. And the Assets could also be implemented using the same underlying tables. The only thing different would be the set of Operations allowed on those. The UI would handle Lots for stocks while the concept of Lots would not exist for Assets. Still not sure if it is worth mixing the two (stocks and assets) in the same table as that might lead to confusion. I see nothing wrong with having a totally separate table for this. They can, and probably will, have a life of their own so it is not worth restricting the potential future development in either case.
@MisterY , I like your analysis of this which is very similar to the analysis I originally did. The difference being the naming terminology.
My implementation required a connection to be made from the checkingaccount_v1 table to the stock_v1 table, to include the missing fields. This modifies the functionality of the stock_v1 table to represent a unified portfolio view, as this would contain all the share accounts for that portfolio.
Noticing the similarity for Assets, the same would be required for assets. The only problem here is: how to identify where the transfer is going to if you reference the transaction record from the checking_v1 table?
This table then required the distinction field to determine what table was to be referenced, since my new table is in essence, handles transfer transactions, and is an extension of the checkingaccount_v1 table. The following diagram displays the connections at a table field level.
All the other infrastructure is in place, which simplifies the backward compatibility issue. To handle shares, there is a need to create a Shares Account. (New account type). Similar to any other account. Accessing the shares account record would identify this as a foreign transaction, using the TOACCOUNT field. which would then identify the transfertrans_v1 record. This would then obtain the necessary extra fields for this transaction.
Using my implementation, here are some screen shots of a test stock/shares transactions.
This is a newly created database using a stock broker.
Here we gave added some bank accounts and share accounts. The share accounts have also been added to the Broker Jim portfolio
Extra transactions have been added through the broker page for the stock: AMP. Here we have added some share purchases, sales and dividends. Notice the transfer of money to ordinary bank accounts from the purchase and sale of shares, and using the proceeds of dividends for extra share purchases, reflected in the Stock (Broker Jim) screen.
This shows the transaction interface for shares. Further refinements may be required.
As an extra view is required for a share transaction history, an overly simplified view of the share transaction summary is shown. This requires a better implementation which I did not have time to complete at the time.
Note: This implementation is still available in my account. Unfortunately, now it cannot be merged, since the pull request was rejected at the time.
Hello, I am just testing MMEx, and i am very interested in stock/asset management. The solution shown here seems to be a good one to complete my needs (by the way, it's the only one). How could i test it ? Thanks
@Zorg2409, Since the original posted branch was out of date, it was removed. I have reposted the branch under a different name: stock_asset_transfers so you can review it.
This was originally set up to test the underlying infrastructure for this solution, with the user interface needing refinements. No real changes have been made since except for limited updates to suite my needs to match new features and fixes from the Master branch.
I will be interested to see what you think about this implementation in this basic form.
Thanks for answering Stef145g, as i am only a user i am not able to compile a new specific version, my aim was only to test your solution, can you send me (or tell me where i can download) an executable windows file ?
@Zorg2409, I have now created a pre-release: Test of Stock Asset Transfers where you should be able to get the executable files. Regards, Stefano
Many thanks Stefano, it seems to work, i'll see how and be back to you with my comments.
Nice WE and best regards
@stef145g, please have a look on the attached .pdf and tell me where i am wrong.
@Zorg2409, Setting up a stocks and shares account is a 2 stage process. First you need to set up the portfolio view, then you need to add the first entry to the shares account via the portfolio view. This will correctly set up the number of shares in the portfolio view, and also add this to the shares account view.
Hopefully the help file explains how this works. This is located in the help file under the section: Stocks and Shares
In your example, you have created
This means that you intend to use Stocks and Shares. Therefore you need to associate the first entry in the portfolio view to the Shares Account Bidon.
This will now show you the full picture for the Stock portfolio and the associated shares.
Adding or removing extra shares
Adding Transactions to the Shares Account
@stef145g, So nice !
_"Therefore you need to associate the first entry in the portfolio view to the Shares Account Bidon." _this was the trick i didn't catch, despite the fact i rod carefully the help file i didn't see this point and i was wondering how to set a correct first transaction !
Now it works and i have a question: for each transaction you can select an account, in my case Bidon or any other, if i choose a bank account the amount of the operation will be directly add to this account and that's all, if i choose Bidon i'll get a new entry in share account Bidon and then i can transfert money to a bank account from the share account Bidon, with this choice i have a history of each Bidon transaction in the share account Bidon. From your point of view, what is the best choice ?
I Think that with your implementation choices one can treat all the cases that can occur in stocks management: shares in/out, free shares add/substract, dividends, share splits, cash management, ... The concept can be applied for all the belongins.
In my case i have 50 diferents shares in 10 stock accounts along with 10 bank accounts this means 60 accounts and 60 lines in the account pop-up windows choice in the transaction window, isn't it too much ?
I"ll continue testing your solution.
Regards
When I originally worked out a solution for Assets and Shares, I was thinking of how to link the transactions of assets and/or shares to the regular accounts maintained in the transaction list (CHECKINGACCOUNT_V1). The user interface was initially developed to test out this process.
You will notice that Assets work in a very similar way, as explained in the technical description above. Being able to select any account gave me the flexibility to achieve simple asset transfer transactions.
Once I had the implementation in place, I realised that selecting any account was not practical when a large amount of transactions were involved because it becomes difficult to keep track of, what I call, foreign transactions in the transaction list. For this reason I introduced the new account types, Assets and Shares to contain and track these transactions. So to answer your first question: It is better to create an account for these transactions. For share management, this is becomes necessary to achieve the necessary functionality, even though it may seam a bit disjointed.
When it comes to shares, the navigation panel needs to be updated to remove the Share Accounts heading. This way all share accounts in a share portfolio would be shown as a sub-branch in the portfolio branch. Since this was only for testing, I didn't implement this as it became a bit more difficult to achieve, and for my needs it was OK like this.
I went further in testing your solution with no problem, it works nice for stocks and assets in the same way.
Definitively i think, from a user point of view, that it is a good implementation that can fullfill all the need for assets and stock management.
My today comments:
Do you think that your solution will be integrated in the main branch of MMEX ? in the next release ? later ? What would happen if i decide to continue with this test version and if your solution is not implemented in MMEX ?
@Zorg2409, at this stage I would suggest that you do NOT use this test version for your mainstream. Because of your interest, I am planning to update this feature which will modify the underlying table structure. Depending on the views of @guanlisheng and @vomikan will be the decider of weather it will be incorporated in MMEX or not. When I have updated the implementation, I will post a new test release.
@stef145g , thank you for your implication in giving an answer to my need, i'll wait with interest your new test release. By the way, the lost of original date and digits when editing stock transaction lead to a very difficult everyday use of the solution. I didn't found other problem.
@guanlisheng and @vomikan , I understand your technical needs and your responsability in MMex development, but strictly from a user point of view, a real stock management is actually a lack in MMex. Today we can only check securities of the original input date, despite the fact that the securities value can be easily checked on internet (this is great) their number can't be followed. Without a historic of stock transaction, one is not able to follow the evolution of its portfolio. As far as i know, the implementation proposed by Stef145g fullfills all the main needs of a stock portfolio management in a way easy to understand, easy to use and well integrated in the MMex main goals :Intuitive, simple, fast, clean.
@Zorg2409, thanks for the comments. We are all painfully aware of the missing bits in terms of Portfolio management. One thing to keep in mind, though, is that the solution needs to be sound from the start in order to avoid restructuring and data migrations at a later stage. What I mean here, is that all the scenarios from this comment should work well and the portfolio overview readily available with relatively simple SQL queries. A good way forward might be to release a test version with the Portfolio feature once the data migrations are in place so that interested parties can test it out thoroughly. What I still do not understand in the data schema is where the portfolio is actually located. There are two tables - stocks and transfertrans, and both of them contain share information. However, with the changes proposed to transactions table in #461, this might also get clarified further.
@MisterY, Thanks for your explanations. As i am interested in portfolio management, i can test, as an user, any solution that the MMex team will bring forward. Waiting for it, the stef145g solution is usable and useful for me.
The Asset Allocation procedure as described is all and fine if I want to review my Superannuation Investments. This is not what I want.
My scenario is this: I want to purchase shares directly from an organisation from a Shares Prospectus The I know the offer will be as a in packages of $1000, and the share price will range between x and y. This will be determined n the day of Share release.
Share Dividends are regarded as income and needs to be declared in tax returns. Even though I have reinvested the Dividend, this is still income, so I need to know how much I need to declare.
Now the above process is repeated over many companies. At the end of the year, I want to know:
How is Asset Allocation going to handle what I want to know?
And what's next now ? The subject is pending for a long time now and do not seems to go forward.
@stef145g I would add 2 items to your scenario:
@vomikan , would you mind to own this one since i heard you are using stock often.