Update: GeneralLedger et al

[EricForgy]

Hi guys 👋

I am not an accountant (not even close), but nonetheless, I feel like accounting is core to finance and should be baked into whatever we do here in some way from the beginning.

Along those lines, I just created a simple accounting package: GeneralLedgers.jl

As I noted here https://github.com/JuliaFinance/Roadmap/issues/7, all these packages are registered in the new JuliaFinance registry so you will need to add this registry to use them via:

pkg> registry add https://github.com/JuliaFinance/JuliaFinance.git

and then you should be able to add them as usual, e.g.

pkg> add GeneralLedgers

I included a very simple example function:

julia> using GeneralLedgers; const GL = GeneralLedgers;

julia> ledger, assets, liabilities, cash, payable, entry = GL.example(); ledger
NewCo
├─ Assets
│  └─ Cash: 0.0 USD
├─ Liabilities
│  └─ Accounts Payable: 0.0 USD
└─ Chart of Accounts:
   ├─ Accounts Payable: 0.0 USD
   └─ Cash: 0.0 USD

At its core, there is just one main Account type, but it uses (my understanding) of Holy traits to dispatch on five primary subaccount types:

1. GeneralLedger: Top-level Account. Has no parent. Has children. Journal entries cannot be posted directly to a GeneralLedger. Note: Since we use double-entry booking, the balance of the GeneralLedger should always be zero since debits = credits.
2. DebitGroup: Has a parent. Has children. Journal entries cannot be posted directly to DebitGroups, but for the purpose of aggregating children, it is treated as a debit account, i.e. a debit child increases the balance while a credit child decreases the balance.
3. CreditGroup: Has a parent. Has children. Journal entries cannot be posted directly to CreditGroups, but for the purpose of aggregating children, it is treated as a credit account, i.e. a debit child decreases the balance while a credit child increases the balance.
4. DebitAccount: Has a parent. Has no children. Journal entries can be posted directly to a DebitAccount. Debit entries increase the balance and credit entries decrease the balance.
5. CreditAccount: Has a parent. Has no children. Journal entries can be posted directly to a CreditAccount. Debit entries decrease the balance and credit entries increase the balance.

The example contains a journal Entry

julia> entry
Entry
├─ Debit: Cash
├─ Credit: Accounts Payable
└─ 10.0 USD

and we can post the Entry to the GeneralLedger as follows:

julia> GL.post!(entry); ledger
NewCo
├─ Assets
│  └─ Cash: 10.0 USD
├─ Liabilities
│  └─ Accounts Payable: 10.0 USD
└─ Chart of Accounts:
   ├─ Accounts Payable: 10.0 USD
   └─ Cash: 10.0 USD

Early drafts of this package had an internal Cash type defined. As we have discussed elsewhere, we need to take care and agree on some terminology. For the purposes of this package (and hopefully this organization),

Cash should be thought of as a Position in a Currency.

Thought in this way, Currency is a kind of FinancialInstrument so Cash is a special case of a more general concept of Positions of FinancialInstruments.

I did a bit of meditating over the weekend and came up with a hierarchy. Not set in stone and open for discussion, but I think we need something like this.

At the top are very simple and lean Countries and Currencies. These are kind of primordial types on top of which most others are built.

Sitting under these we have Markets. As I've described elsewhere, I see a Market as a kind of living provider of market quotes, e.g. yield curves, stock prices, implied volatilities, etc. ultimately to be used to price FinancialInstruments.

Sitting under Markets, we have Jurisdictions. A Jurisdiction could be a country, state / province, county, city / municipality, etc. Different Jurisdictions can have different laws / regulations, different tax treatments, different capital adequacy requirements, etc. These all have impacts on accounting.

Sitting under Jurisdictions. we have Entities. An Entity is basically an issuer of FinancialInstruments. These could be governement treasuries, central banks, municipalities, corporations, etc. This is also where we could keep info about counterparties such as credit ratings, etc.

Finally, sitting under Entities, we have FinancialInstruments. Based on conversations here, on Slack and on Discourse, I think most people will be interested in FinancialInstruments. That is the "quant" stuff, i.e. defining and pricing (possibly complex) financial instruments.

This is also where there is a large possibility to overlap with Miletus.jl since a FinancialInstrument is a contract. I suspect we'll need to do a lot of bikeshedding around FinancialInstruments, but it is my hope that we can have other packages spring up to extend Financialinstruments in such a way that it still plays nice with the underlying accounting stuff.

Next, we have Positions sitting under FinancialInstruments. Positions is a generalization of Cash to accommodate a position in either Currency or a FinancialInstrument.

At the bottom of the totem pole, we have GeneralLedgers - as described above - sitting under Positions. So the entire totem pole looks like

• Countries / Currencies
• Markets
• Jurisdictions
• Entities
• FinancialInstruments
• Positions
• GeneralLedgers

All these are still skeletal with the most flesh on GeneralLedgers, but I wanted to get something out there working so we can start an important discussion and map things out.

I hope this is helpful and I look forward to feedback / questions / suggestions / etc.

Take care! 😊

[JeffreySarnoff]

You should provide subclasses of assets and subclasses of liabilities. An option to buy a thing is an asset that is quite different from the asset that is ownership of that thing. A tax liability is distinct from others.

[EricForgy]

Hi Jeffrey,

Yes. Of course you are right 👍

An asset group would be an instance of a DebitGroup and an asset account would be an instance of a DebitAccount.

Similarly, a liability group would be an instance of a CreditGroup and a liability account would be an instance of a CreditAccount.

See the example I provided. "assets" is a DebitGroup, "cash" is a DebitAccount" under "assets", "liabilities" is a CreditGroup and "payable" is a CreditAccount under "liabilities".

Initially, I hard coded "Assets" and "Liabilities". The most common breakdown is "Assets", "Liabilities", "Revenue", "Expenses" and "Equity", but this is in no way universal, e.g. France is quite different. This design is flexible enough to handle any breakdown based on double-entry accounting.

A general ledger is structured like a tree where you only post entries to the leaves.

[EricForgy]

PS: I think it will help a lot to see an example of how a "GeneralLedger" can be used to build up "FinancialStatements" such as a "BalanceSheet" and "IncomeStatement". That is on the TODO list 😅

[alecloudenback]

@EricForgy - quite a set of commits over the last 24 hours! Nice work.

I will spend some more time with this and give this more thought, especially the overall structure. Before I forget, I wanted to jot down a few notes about the Ledger in particular:

• It can be useful to have different ledgers that you can combine. I've seen where a ledger will have a Primary ledger, which is mostly cash-like entries and then a separate ledger for the regime-specific entries, e.g. a GAAP ledger. The full GAAP ledger is thus the Primary + GAAP ledger.
  • Looking at the code, I think this can already essentially be accomplished given the tree structure. Just treat the first layer of children of accounts as these sub-ledgers.
• I might suggest a slightly different structure of the ledger, where all of the entries are stored and balances are only calculated and cached when called for.
  • Having the entries gives you a lot more to work with when drilling down into a ledger.
  • With an associated date, the income statement/balance sheet becomes a reducing exercise over the entries within the date range.
  • Downside is that this could get very memory intensive for large-scale systems. Perhaps have two forms of a ledger, e.g. a SimpleLedger which really just tracks the balances of different accounts and would suit most needs, and then a FullLedger which tracks all entries?

[alecloudenback]

On the general structure:

• The way that its defined above, isn't Currency just the financial instrument of an Entity that happens to be a country? Could the hierarchy be simplified if it were treated that way and have Countries/Currencies just a "batteries included" set of entities and financial instruments?

• Markets...

  • it seems intend to function similar to how Observables are intended to be used in the Simon Peyton Jones functional contracts sense. I would suggest expanding the universe of Markets to be intended to represent more than financial data - such as weather, insurance payment triggers, counterparty default, etc. Or make FinancialMarkets a subtype of Observables?
  • Does this need to be a part of the totem hierarchy? Could it be separated out more independently?

• Jurisdictions...

  • I agree that we should have a representation of different laws/regulations. In terms of the hierarchy, perhaps Jurisdictions would be better placed in between Entities and FinancialInstruments. I think that all FinancialInstruments would belong to a certain Jurisdiction, but Entities can be pan-judicial (ie a company issuing some contract in two different countries).
  • In terms of how a Jurisdiction would work, would it be a set of behaviors/rules that you could compose layers further down with? I.e. you could compose a certain Jurisdiction with a FinancialInstrument which might affect how its valued, or compose it with a GeneralLedger to get a set of accounting rules? How this would work in practice I'm not sure, but conceptually this might be a way to use them.

[EricForgy]

Wow! Thanks for the quick review and feedback 🙌😃

• It can be useful to have different ledgers that you can combine. I've seen where a ledger will have a Primary ledger, which is mostly cash-like entries and then a separate ledger for the regime-specific entries, e.g. a GAAP ledger. The full GAAP ledger is thus the Primary + GAAP ledger.

  • Looking at the code, I think this can already essentially be accomplished given the tree structure. Just treat the first layer of children of accounts as these sub-ledgers.

Yes. Not only that, since you also work in insurance, you might appreciate that I'd like to be able to look at the balance sheet under different bases. An insurer has at least three different balance sheets:

1. Accounting Balance Sheet - Used for financial reporting
2. Economic Balance Sheet - All items are market consistent. Used as a component for risk management.
3. Statutory Balance Sheet - Used for regulatory reporting

These three balance sheets often have sensitivities in opposing directions, which makes risk management for an insurer fun / challenging 😊 One example is Korea. The regulator in Korea restricts the liability duration, so no matter what is the economic duration, it will have a shorter statutory duration. This leads to situations where you need to balance being duration matched under a statutory basis with being duration matched under an economic basis. You can't have both. That is not even considering the various accounting bases, e.g. hold to maturity, available for sale, etc 😊

I didn't get much into my motivation, but my goal is to be able to support / simulate large multi-national, multi-currency corporations (likely insurers, but for these packages it shouldn't matter). I'd also like to look at one entity's balance sheet through the eyes of another jurisdiction, e.g. I'd like to look at a Malaysian insurer under Hong Kong's captial adequacy regime.

Initially, I bounced back and forth between calling the type SubAccount, SubLedger or AccountGroup, but the concept is the same. I settled on AccountGroup. Then, as I got a little deeper, I realized a Debit / Credit split made sense, so AccountGroup was split to DebitGroup and CreditGroup.

We can always consider name changes etc, but for the time being, a subledger could be thought of as an instance of DebitGroup. In your example, the GeneralLedger would have two DebitGroups called Primary and GAAP. These would roll up into GeneralLedger. Note: You can always convert a GeneralLedger to a DebitGroup by giving it a parent. So even if Primary and GAAP started out as independent GeneralLedgers, you could combine them by giving a common parent. Under the hood, there is just one type called Account and I use Holy traits via singleton types to dispatch on the other account types I mentioned above.

BUT... maybe this design is flawed. Maybe an Account should not even have a parent OR children. Maybe an Account should just be an Account with balances and transactions. Then we can construct a tree / report independent of the Account and add accounts to the structure. This way, the same account could contribute to multiple structures / reports.

I value your opinion on this. Nothing is set in stone for sure. I just thought it would be good to have some working code to help the conversation along 😊 If, at the end of the day, everything I did so far is thrown out, that is fine as long as it is replaced with something better 😊

I might suggest a slightly different structure of the ledger, where all of the entries are stored and balances are only calculated and cached when called for.

I agree 100%. I should clarify what I mean by DebitAccount and CreditAccount since the direction I am currently headed is probably nonstandard. I see these accounts as primordial. I might even have an account for an individual bond. So I would only be posting entries to accounts that have an inherent balance / price and not to groups of such things.

• Having the entries gives you a lot more to work with when drilling down into a ledger.

Yes. I agree 100%. All entries need to be stored. It is the right thing to do and required for audit purposes 😊

• With an associated date, the income statement/balance sheet becomes a reducing exercise over the entries within the date range.

💯👍

• Downside is that this could get very memory intensive for large-scale systems. Perhaps have two forms of a ledger, e.g. a SimpleLedger which really just tracks the balances of different accounts and would suit most needs, and then a FullLedger which tracks all entries?

Yes. Let's think about the right way to do this 👍 It is certainly a solvable problem.

The way I've thought of it is we can have a "current" ledger with all balances just representing their "current" amount (not "current" as in "current assets" 😅) and the history available elsewhere in memory and persisted somewhere. The footprint for the transactions is pretty small, so I am not too concerned about memory. We should be able to handle billions without hassle. As always, open to suggestions.

Another motivation, which is nonstandard, is that I want to be able to handle real-time reporting. When a policy is bought, its impact flows instantaneously to the balance sheet and shows up on a dashboard. That will be an important demo 😊

Real-time balance sheets pose interesting challenges. What do you do with depreciation? Depreciation is constantly changing. Do we need an entry every millisecond to update a depreciation account? The answer is "No" of course, so we need to think of ways to make this work and I have ideas. Happy to discuss 😊

[EricForgy]

• The way that its defined above, isn't Currency just the financial instrument of an Entity that happens to be a country? Could the hierarchy be simplified if it were treated that way and have Countries/Currencies just a "batteries included" set of entities and financial instruments?

Yes. Good point! You are correct of course. I didn't think of that 🤔 I might be inclined to consider Currency a FinancialInstrument issued by the Entity called "Central Bank" (because that's what it is 😅 ). Great point. Let me think about this. Value your thoughts 🙌

• it seems intend to function similar to how Observables are intended to be used in the Simon Peyton Jones functional contracts sense. I would suggest expanding the universe of Markets to be intended to represent more than financial data - such as weather, insurance payment triggers, counterparty default, etc. Or make FinancialMarkets a subtype of Observables?

Good point! And good idea 👍🤔 I should sit down and read that paper one of these days 😅 But yeah, the concept seems similar. Making Market an Observable also makes sense. Let's work it out 👍

• Does this need to be a part of the totem hierarchy? Could it be separated out more independently?

Sure. Anything is on the table. This looks nice: https://github.com/JuliaGizmos/Observables.jl

Maybe Markets.jl is just some sugar to construct some Observables. Maybe we called it FinancialObservables.jl or something. Open to ideas 👍

• I agree that we should have a representation of different laws/regulations. In terms of the hierarchy, perhaps Jurisdictions would be better placed in between Entities and FinancialInstruments. I think that all FinancialInstruments would belong to a certain Jurisdiction, but Entities can be pan-judicial (ie a company issuing some contract in two different countries).

Sure. Happy to consider this change. The argument makes sense. Maybe give me some time to think about it 👍

• In terms of how a Jurisdiction would work, would it be a set of behaviors/rules that you could compose layers further down with? I.e. you could compose a certain Jurisdiction with a FinancialInstrument which might affect how its valued, or compose it with a GeneralLedger to get a set of accounting rules? How this would work in practice I'm not sure, but conceptually this might be a way to use them.

I haven't worked this out yet. It is still just a placeholder so we can think it through, but, yeah, that is the idea.

[EricForgy]

PS: Thanks again for the quick feedback and suggestions. This is awesome 😊🙌

[EricForgy]

@alecloudenback, you brought up a lot of interesting points and suggestions. Thank you for that 👍

Countries and Currencies

One idea you floated was to change the totem pole and effectively subsume Currencies into FinancialInstruments and Countries into Entities. I did a quick Google search and this was the first result:

• DDD: Should 'country' be a Value Object or an Entity?

We should not make a decision based on that, but it brings up a good point.

In some cases, we want to think of a Country as an Entity, but not always. In some case, e.g. in an address, a Country is just a value. The latter is what I had in mind for Currencies so I am still kind of inclined to keep Countries as it is.

Similarly, in some cases, we want to think of Currency as a FinancialInstrument, but not always. Sometimes, we just want a value, e.g. a selection to specify how to display e-commerce prices.

So I am still inclined to keep Currencies and Countries as these primordial values / singleton types at the top of the totem pole and bring them in below when needed. But it is a good point that Currency is a FinancialInstrument issued by an Entity so we might reconsider at what level to bring them in.

Entities and Jurisdictions

You also raised a good point about the order of Entities and Jurisdictions on the totem pole.

I initially named Entities as LegalEntities. If you think of it this way, as LegalEntities, then all LegalEntities are incorporated in and subject to the laws of a Jurisdiction.

I dropped the Legal from the name because I was thinking of Countries and Individuals and wasn't sure if they are considered LegalEntities, but according to this, they are. So I think I'll change the name back and keep the current order of the totem pole. Does that sound ok?

[Edit: Here is a good Wikipedia article - Legal Entity]

Restucturing GeneralLedgers

I agree with you. More thought is required here. I am definitely open to restructuring and happy to dispose of my first attempt if we can come up with something better. Your points helped me realize my design was flawed and thinking about how to fix it now.

Thank you for your feedback 🙌

[taqtiqa-mark]

Hi, Thank you for stimulating the Finance domain in Julia, and for all the contributions you've made open.

Disclaimer: I'm not an accountant, but I've worked with them.

Reading the roadmap the ambitions are high - which is great.

In terms of accounting I wonder if it might not be useful to first focus on a General Journal. Perhaps even have the general Journal be the first class citizen, with parsers/readers to generate/populate a GL. If some convention-of-configuration approach is adopted you might even be able to generate the GL by parsing the GJ. When you encounter something that can't be handled then initiate a special journal, but not before you have an insoluble problem in the GL and parser.

I know journals have fallen out of favor in practice, but here maybe the can be useful. Specifically, with a GJ record/source of truth you could be quite aggressive in refactoring the accounts setup (GL), and replaying treatments.

Also with a GJ as a source of truth you could migrate GJ versions/formats and have a complete record of past in the new format.

Ack GJ size would be large (TB?) in some use cases, but we're at the point where that is not an obstacle/objection.

Final a GJ is a much simpler beast to define. The effort then goes into creating import utilities, example creating a interface utility to generate a journal able to be imported in to xero, etc. etc.

Ack this is speculation, but I'd appreciate any thoughts on gotha's I've overlooked, etc.

[EricForgy]

Oh hi @taqtiqa-mark 👋

It's really great to hear some thoughts and feedback. You and I are in the same boat. I'm not an accountant either, but I work with them 😊 I do also feel that accounting should be at the core of a JuliaFinance "ecosystem".

I think there may be a semantic issue though. What is the difference between a GJ and a GL? 😅

I've been working on GeneralLedgers.jl although I got distracted with some work on WebSockets.jl, Figures.jl and StochasticDiff.jl, but would like to come back to this soon.

I think of the GL as the primordial record of truth and all financial reports can be derived from the GL. If you are saying that is also the case of GJ, I wonder if we are talking about the same thing? 😅

Final a GJ is a much simpler beast to define. The effort then goes into creating import utilities, example creating a interface utility to generate a journal able to be imported in to xero, etc. etc.

I agree 💯😊 This was my goal with GeneralLedgers.jl, i.e. to eventually be able to import / export to / from popular accounting packages 😊🙏

I suspect we are thinking the same thing, but I will look into it 👍

Thank you again for your comments and welcome to JuliaFinance 😊

Edit: Found these while googling

• https://www.wallstreetmojo.com/general-journal-vs-general-ledger/
• https://www.accountingcoach.com/blog/general-ledger-general-journal
• https://www.investopedia.com/ask/answers/030915/whats-difference-between-general-ledger-and-general-journal.asp

[taqtiqa-mark]

What is the difference between a GJ and a GL

I think of the GJ as a log file, nothing more, just rows of data.
GJ -> GL. The GL is not a simple log file but normally has some 'structure'.

The idea would be that if we can get consensus and buy in, apps that trade would log their transactions using the JuliaFinance GJ conventions (JF-GJ) - the JF-GJ output would be little more than a pretty printer (KIS).
Then you could take that one GJ and post to GL's provided by xero, GNUCash, etc. etc. It might be the case that there'd need to be a JF-GJ -> xero-GJ utility, or JF-GJ->GNUCash-GJ Or it might be that we take as the JF-GJ format one that is compatible with say GNUCash, or some such candidate. GNUCash, xero, etc would provde Bal sheet, P&L etc

If we used convention of configuration it should be possible to get a JF-GJ that can be quite generic.

Sorry have to run.....

[EricForgy]

Thanks @taqtiqa-mark 👍

I did a little reading (see links above) on the difference between general journal and general ledger. I agree with you that we need a general journal and it would help with interfaces with other accounting packages.

I might summarize it this way:

The general journal is a chronological record of transactions (double entries).

The general ledger is a record of balances together with groups of similar accounts.

GeneralLedgers.jl already has an Entry type for double entries so, if I'm not mistaken, it seems like the general journal would simply be a chronological record of all entries 👍

[taqtiqa-mark]

... we need a general journal and it would help with interfaces with other accounting packages

I think so. I'm not convinced a GL is needed - at this point. The issues discussed above are not moot, although it did seem like heading toward designing a Financial Markets accounting package/chart of accounts, which my not be required in most cases.
The JF-GJ would need to contain some account info and it will make sense for any app, package, service to populate the JF-GJ with sensible default data - that likely is where the JF-GJ package would come in.

it seems like the general journal would simply be a chronological record of all entries

Pretty much coincides with my understanding - I'm not sure it even needs to be chronological, but that can be worked out.

I think the main points are (unordered):

• The JF-GJ is a transaction log that can be consumed by at least one accounting package (GNUCash, or some such)
• The default JF-GJ entries conform to a 'sane' chart of accounts
• The default JF-GJ entries can be ported to other accounting package requirements e.g. xero.

Hope that helps.

[EricForgy]

I think so. I'm not convinced a GL is needed - at this point.

A GL is needed if you want to build an accounting framework that accomodates financial models written in Julia natively (which is what I want to do) 😊

My goal is not to be able to plug into accounting software. I will build accounting software in Julia 🤓

Hope that helps.

For sure. Thanks 👍

[taqtiqa-mark]

My goal is not to be able to plug into accounting software. I will build accounting software in Julia

:bowing_man: