Continuation of the project

[alensiljak]

Hi all!

Since @dmitry-merzlyakov seems to be absent (hope everything is alright, in any case!), I am wondering if anyone is interested in keeping this project alive (bugfixes and updates) and building upon it. I think it would be a healthy part of the Ledger ecosystem and would like to work on building other parts on top of it, myself. Please ping me and we can arrange an org on GitHub to pool effort.

[dmitry-merzlyakov]

Hi,

Sorry for delayed response. I am actually here; if you do not have my response for several days or more - please, send a direct message to dmitry.merzlyakov@gmail.com; GitHub notifications might be not reliable sometimes.

Thank you for your interest to this project. I am open to any suggestions how to organize our collaboration and change further roadmap. My personal intentions were about fixing all upcoming bugs and making infrastructural changes in the build process to reflect new dotnet versions. People did not show big interest to other focuses, so I did not prioritize other enhancements so far. If you see something that might be added or te-prioritized in the roadmap - let's discuss.

So, I am here, sorry again for missing the new issue, I will work on it. If you have any thoughts, issues, enhancement requests - please, add them here or as tickets; I am glad to see interest to this stuff.

Thank you!

[alensiljak]

Hi, Dmitry! Glad to see your response. Thanks for the direct contact details! I've only today noticed the new branches that you've been working on a few months ago. By default I've only gotten the main branch locally so I was under the impression that the activity on the project stopped two years ago.

By no means was the intention to rush you into fixing the issue I reported a few days ago. I'd actually hope to be able to fix that myself with some assistance and explanation on the side. Anyway, I'll be in touch via email with additional background information. Here I'd just mention a couple of items - Rusty Ledger where I experimented with Rust implementation of Ledger, and https://cashier.alensiljak.eu.org/ (or https://cashier-II.alensiljak.eu.org/), where I'd like to include Ledger journal parsing. Long story short, after going through several technologies/platforms, I've tried doing this in .Net. Blazor in the later iterations seems like a decent enough target. Then I found your implementation of Ledger in .Net. I'd like to participate in the project to an extent, and try to include it with Cashier or built on that. I don't have concrete plans but would love to have a decent personal finance tool on my phone.

In any case, glad that you're around. I'll share some of my suggestions either as issues, or perhaps you'd want to enable Discussions and keep Issues to only items to NLedger in its current form. Or we could also form an NLedger org, similar to Rusty Ledger, and have interested people join in, if they wish so. Cheers!

[dmitry-merzlyakov]

@alensiljak Hi, as you asked to provide some generic background about the project principles, this is probably a good time to briefly review the current state of the project and consider next steps.

First of all, the main goals of this project are:

• Provide a dotnet utility (shared library and command line application) that is 100% similar to the original Ledger. This corresponds to all the features and behavioral aspects of input processing, so that people can see no difference and therefore trust the result obtained. The acceptance criteria for this purpose are passing all the Ledger tests that the Ledger project has.
• Constant receiving all new upcoming changes from the Ledger project. NLedger is like not a static port of a specific version of Ledger; it should reflect all recent bug fixes and feature changes. This goal defines the architectural concept: all core classes reflect the original design semantically as much as possible to expose C++ code to dotnet (with logical replacement source structures with basic features and techniques from dotnet word). This approach allows you to add new changes to Ledger with comparable little effort.
• Expand the basic functionality with new functions if they do not contradict the points above. This means that anything we want to add should build on the core classes, but not replace them. New features can have their own tests, but do not change existing tests (and therefore existing features).

It implies how I see the published artifacts produced by this project. Currently, they are:

• NuGet package representing NLedger library ready to be used by any consumers (usually dotnet applications that want to embed Ledger features into their own calculation process). Currently, there are two published NLedger packages (NLedger and NLedger.Extensibility.Python); they target .Net Framework and .Net Standard 2.0.
• Compiled binaries in a zip package that contains the library, the command line utility and additional things like Ledger testing framework and some other tools.
• MSI installer that contains the same content as zip archive. As I know chocolatey distributes it.
• Python package that can be installed on any Python environment and provide NLedger functionality by means of Python-centric API.

The main value of these published artifacts is that they went through a regular build process and passed all tests (for each target). Thus, they guarantee that all declared features work reliably and exactly conform the corresponding version of Ledger (without exceptions).

We can also consider dev tools (PowerShell scripts) as public artifacts; they let people download the git repository, configure environment and rebuild the project with minimal effort. It is a simple way to produce binaries for any target platform and OS. However, they are part of repository code, and it is up to people responsibility to validate produced binaries.

Essentially, these points define the acceptance criteria for the development process:

• The main criterion for accepting any changes is the ability to pass all Ledger tests; they should always be in good condition as they reflect the original features of the Ledger. Of course, all other unit tests must also be passed or corrected accordingly if new modifications change something in the existing implementation logic.
• Changes should not disrupt the existing build process as long as they confirm the consistency and stability of the artifacts being built. Specifically, the build checks the stability of command line utilities for all targets and consistency of the Python package. Of course, this is negotiable if new changes conflict with something outdated in the build process, but the build process should be eventually restored and kept in a valid state. This is probably a task for me since I am the owner of the build environment, at least for now.
• Changes must be tolerant of compatibility aspects, so decisions to break compatibility should be made thoughtfully (taking into account which dependency or dependent part is affected).
• It’s a common thing to cover new code as completely as possible with unit tests.

As for how development is currently organized:

• The "master" branch reflects the current public version of the library (currently 0.8.4). It is updated the moment we complete the development of the version, go through the full build process, publish all artifacts and add a GitHub release. No CI for Master.
• "next-dev" - a branch that accumulates ready-made functions for the new version. It has CI set up so developers can check if the build is in good condition.
• Other branches (usually synchronized with "next-dev") are placeholders for the actual development work. Updates may contain breaking or risky changes. This is ok for development, but all issues must be resolved before merging features or fixing bugs into "next-dev" (and pushing new changes through the CI process).

Currently, there is "next-dev-0.8.5" branch containing some bug fixes and "next-dev-0.8.5-net6" that contains a big piece of work to adopt existing development process for multitargets (net6/net8).

Thus, according to the described principles, there is the following vision of the current roadmap:

• Generalized periodic tasks:
  • Bug fixing. If a new bug is discovered, confirmed and fixed, the fix is added to the current accumulated development branch (for example, "next-dev-0.8.5"). If a fix is needed urgently, I can release a new version and move the rest of the development work to the next version.
  • Receive the latest changes from Ledger. I used to get all the new updates (code changes and tests) from Ledger at least once a year.
• Current tasks: in general, I consider the work to support multi-targeting to be the most important right now. The current build process depends on the specific targets and it must be solved to achieve three main goals: to enable easy development of applications that use the latest dotnet targets; maintain compatibility with projects that still need targets for older but supported platforms (primarily the dotnet 2.0 standard) and simplify the release of new NLedger versions by making it a completely automatic process.

Progress has been slow in this area but it may be forced. This seems to be my task because this work deals exclusively with the old build infrastructure. The good thing is that this likely the only task in the project that requires my direct contribution: any other changes to the dotnet code can be made by anyone as long as they follow the principles described above.

An interesting question arises as to what should be considered the most popular targets for NLedger right now. This is a question for many developers releasing shared libraries (e.g. https://community.devexpress.com/blogs/news/archive/2023/07/05/net-net-standard-2-0-deprecation-support-for-net-6-or-net-8 - lts-by-default-in-libraries-and-more-v23-2.aspx). I'm leaning towards the following concept:

• The NuGet package is multi-purpose (netstandard2.0, net6, potentially net8). This will satisfy existing projects that are still using the netstandard version and, also, new projects targeting net8.
• The command line application is multi-purpose (Net Framework 4.8 and net 6). This will maintain compatibility for users who still use the utility on "pure" Windows, and for those who want to use the binaries without recompilation on other OS. If a new version of dotnet becomes popular, the new target is added to the list of existing ones.

Of course, the above describes my personal vision, and I am open to discuss any aspects of this process. Any input is welcome, thank you! Also, if you want to move this comment to any another place - do it, I just added it here in continuation to your input.

[alensiljak]

This is great info, thank you! I'd suggest adding this as the Contributing doc, which I don't see at the moment, anyway. I've seen mention of the roadmap but am not sure where to find it. Is this on GitHub (milestones) or is/was there a different doc?

[dmitry-merzlyakov]

Hi, you can find the roadmap here - https://github.com/dmitry-merzlyakov/nledger/blob/next-dev-0.8.5-net6/roadmap.md (see it is a bit different from the version in master). However, this stuff probably needs to be generally re-visited. It is based on old ideas (5 or more years ago). Actually, at the moment, I see only two tasks that really need to be done: 1) the work related to the build process and multitargets; 2) get recent changes from the original Ledger (I did not do it for more than one year). If you have other ideas - we can create a ticket that accumulates all new ideas or requests.

I agree with you that contribution section needs to be added. The best time to do it is when a version is finalized, and we made final corrections in the doc. I worry I can forget this point - you can add a ticket for it so we will recall this point when, thanks.