rewrite in golang lolz =)

[rami3l]

I personally agree that rewriting the project sounds interesting. Just being curious, what's the potential benefit of using golang?

[icy]

@rami3l I don't really know. Maybe it helps to attract some developers. Now looking at Bash code: it's pretty hard to maintain and I don't think there are too many people who're using .bash. I think bash/shell scripting is the best tool for this kind of job anyway. The original idea is to support limited environment where .bash is not available generally.

I tried to rewrite the tool in dlang (it's on the branch nd), to see how far I can go with that. There are two problems, which I would see again in golang version (if any)

1. I don't know how to spawn an interactive session of the shell and at the same time, feed them with some script via stdin (the dlang version of the tool is still using bash scripts; the dlang code is all about argument parsing and it launches a subprocess)
2. Compiling on some special target: dlang support was quite limited from what I tried, and/or it took some hard way. Only could I build a binary from Linux system. Golang is better I think.

Btw, this is already a 10-year project. When I started I couldn't think I would go far like this. :))

[rami3l]

@icy If I were you, maybe I'll go for Python because:

1. Python is a popular (and quick-to-learn) scripting environment, and it's not very demanding as for different targets.
2. A scripting language suffices anyway for this kind of task, here we don't need to care much about speed.
3. subprocess (for running shell commands) and argparse (for argument parsing) are in the Python Standard Library. This might help with deploying the project (quick development, no compiling and stuff).

I'd like to experiment... Although I'm not quite sure of what you are trying to say here:

spawn an interactive session of the shell and at the same time, feed them with some script

[rami3l]

@icy A simple proof of concept has been made public here. Is this the correct behavior that you want (for example, python3 main.py -Syu just acts in almost the same way as the original wrapper)?

PS: I wrote the dispatch module exclusively for brew commands, but I think it's also possible to add support for other package managers if you just change that to bash xxx.sh, providing that you have the correct .sh file for each case. I also took a slightly different approach, implementing | grep xxx part in Python instead of calling grep. Not sure if it helps though. Also, maybe the parser module needs more work.

[icy]

@rami3l Thanks a lot for your contribution and your time. I really appreciate that.

I think Python or any higher level language can help a lot. However, we need to consider one of the main advantages of the current pacapt: It can run almost everywhere (yes, even on very old ubuntu and/or redhat and/or on some amazon AMI-packed machine).

I did try the Dlang-version, one of the reasons was it's a compiled language. I know there is someway to do the same in Python (PyInstaller -- which I use a lot in my work), but from my experience it's not very portable. Please note this is my little experience, as I'm not a heavy Python programmer.

Let's me write down some goals in my mind now, before we can go further:

• To have more developers for the project: The current shell scripts are not easy for many people
• To have better test support (there are a few smoke tests now, though)
• To expand the use of the script on all platforms if possible. The current limitation is that the tool can't be used on non-Bash system)

Do you think we would obtain the same thing (esp. the 3rd goal) when we rewrite the tool in Python and/or foobar?

[rami3l]

@icy I also have doubts concerning those aspects...

Testing

I'm not quite sure what tests should be integrated a wrapper like this (due to my lack of similar experience), but I'm sure that any "modern" language (which meets with your 1st point) has a proper way of doing tests far better than what we have now...

And we will do those tests on which platforms? If this project needs to be cross-compiled, then the problem might be more complex.

Portability

Your point reminds me of the fact that I don't know quite much about the potential use cases of this project.

Surely, if we want to maximize the portability, we should really choose to compile the project.

But when I came up with the idea of using Python, what I referred to was to run Python script with an interpreter, i.e to use Python as a scripting language, so that this project may run on any machine that is capable of running a Python 3 interpreter.

BTW: PyInstaller is for "freezing" a Python application, and a frozen app is something quite different in terms of portability.

So maybe I should ask, what exactly are the special cases in which, for example, the CPython interpreter is not available or at least not convenient to use? (From my experience, an example might be an OpenWRT router with limited flash storage. Installing bash is possible in this case, while Python might look a bit heavy.)

It is important to specify these special cases in order to make a good choice between languages.

PS: I am learning Golang right now, and I think it's a promising choice if we finally go for a compiled language. Of course, the portability must be considered before making the final decision...

[icy]

@rami3l Thanks for your input. That's great to discuss these things

Testing

We have unit tests that can help to modify code easily. Writing some tests in bash is not very comfortable.

We also need integration / smoke tests because we need to make sure the behavior of the tool is correct on the target machines. For this part the current way can be used (e.g, https://github.com/icy/pacapt/blob/ng/tests/dpkg.txt).

Portability

Yes I really want to use the tool on busybox-based environment, where bash (or even .sh) is not generally available. There are also many limited environments: router (OpenWrt), some Pi machines, some bsd machines, docker containers.

Golang can be built on many target machines as I know, and it's also easy to build on different targets (see for example https://www.digitalocean.com/community/tutorials/how-to-build-go-executables-for-multiple-platforms-on-ubuntu-16-04)

Fyi, there is already a C-based project https://github.com/emilengler/sysget . It's doing well it's just not pacman-liked though

[rami3l]

@icy Maybe we can start trying to refactor this project using Golang, and I think import "os/exec" might be useful. For not being shell-dependent, the full implementation must be done in Golang, just like what I tried to do in that Python repository. I'll try to see what I can do, but still, I'm relatively new to Golang, not knowing much about the toolchain stuff, and I believe I still have got a lot to learn. _

[rami3l]

@icy I have trouble translating my Python script in Golang in the parser part. I think it should be better to import a parser library, but I also noticed that the famous yay did that by hand... What do you think we should choose, with consideration of binary size, expandability, etc?

PS: Here I choose to import a library.

[icy]

@rami3l Thanks a lot for your initial work. Let me write down some important aspects of pacman..., which I tend to forget now. As far as I remember, if the parser can support multiple counting (number of appearances of any option).

I have a lot of time now, to spend a bit on pacman. Give me a little time to sell out my guitar :D

PS: TLDR;

In my dlang code, it's very easy to do that (https://github.com/icy/pacapt/blob/nd/dmain/internals.d#L467).

During the last few weeks, I discussed quite a lot with a team mate, to answer to the question, does that really make sense to write everything in golang? It's hard to answer. In fact, rewriting requires at lot more work so that tests and code-coverage can be good.

I don't know. Last week, I just wrote 300 lines of Bash code that cover most important features of this k8s/cd tool : https://github.com/fluxcd/flux , and it also can do a lot more useful things for my company (to be honest, I also need to write 2 jiffy tools in golang, one of them is the prometheus exporter; but they are optionally). If the problem is that we can't write some tests in Bash, why don't people try to solve that problem? Oh please!

[rami3l]

@icy Yep! Flag problems seem to have been solved, including flag counters.

As for language use, I still think a modern language will help, though it's important to help a bash-based project have a smooth transition. I wrote that "all should be written in golang in order to be shell-independent", but that isn't always important every single time (eg. for those systems that come with bash), especially in the transition progress.

Still, for experimental reasons, I'll rewrite at least homebrew support using golang, but I am actually trying to do just that in the dispatch module, let's see if we can just grab the scripts that we have now, maybe split them into smaller files, put them into a directory, and just call bash to do the job!

[rami3l]

@icy By the way, do you think that we can utilize GitHub Actions to simplify the testing?

[icy]

@rami3l Github Action is also fine, though it's a bit scary to use some 3rd party actions. For small projects, I don't think there is any huge difference between GithubActions and Travis/CircleCi. You may go with any tool you feel comfortable then ;)

[rami3l]

@icy I also referenced your Dlang version, I personally know nothing about Dlang, yet it seems like a transcription with the same global variable logic as the ng version, which is a little bit difficult for me to understand (my apologies @_@). Could you somehow guide me through the original project? In the lib directory (tree/ng/lib) I can see all the scripts targeting different package managers, so is there anything else that is also important to be aware of? Or should I just focus on how to dispatch and call the functions in lib? Thanks in advance!

[icy]

Hi @rami3l , sorry again with my belated response. And thank you a lot for spending your time on the project. I really appreciate that.

The dlang version was simply a direct port of the bash scripts, with some minor features. Let me write down some important things of the orginal pacapt script instead. There are a few important parts:

(1) understand user input

Scripts:

1. https://github.com/icy/pacapt/blob/ng/lib/zz_main.sh
2. translate_* methods in https://github.com/icy/pacapt/blob/ng/lib/00_core.sh

These contain the main argument parsing steps, and the goal is to have almost the same option sets as the ArchLinux pacman (see man pacman). It's this goal that adds quite a lot of logic and complexity to pacapt. This part when being written in Bash is hard to have unit-tests; rewriting it in any higher language would help a lot.

This part is heavily relying on bash-4 features, and makes it hard to be used on non-Bash environment (busybox, bourne shell, dash,...)

( Starting with another goal would be a lot easier (e.g, sysget), more neutral. In the very first days of pacapt, I used this page for transition https://wiki.archlinux.org/index.php/Pacman/Rosetta . But, it's the orginal idea of pacapt to reach that goal, isn't it?)

(2) understand the system target

Now when the script understand user input, it needs to translate them into the target's commands. Hence the next step is to understand the current environment: https://github.com/icy/pacapt/blob/ng/lib/00_core.sh . This script basically detects the type of the OS/package manager, and it's quite easy to rewrite them in golang/python Imho.

Nowadays pacapt can also support external package manager (e,g npm, ruby gems...) and there is another part https://github.com/icy/pacapt/blob/ng/lib/00_external.sh , which expects that the package manager is provided additionally (e.g, pacapt-gems, or pacapt gems,...)

(3) connect/invoke system package managers (and/or transform the output)

The remained parts in the lib/ directory (e,g . lib/dpkg.sh); most of them can be executed on non-Bash4 environment (that's because it is really depending on target machine, it's a machine requirements, not the script's requirements.) To help Arch users, some grep/awk/sed commands are used to make the output beautiful and/or clean.

Basically, in this step, you will issue the system commands and capture their output. Sometimes an important thing is to deal with interactive command: For example, when you run pip install foo from your terminal, it will start new interactive session where you may want to provide some answers/confirmation. If you want to use pacapt to execute pip install foo instead, it's also critical for pacapt to have the same behavior: kind of by-passing the stdin/stdout/stderr between user and system commands, and then you have a little chance to process anything.

As you can see in my dlang version, I just slightly modify the scripts under lib/ and then read all of them at build time (https://github.com/icy/pacapt/blob/f3ae5f8b1f8ec78590c1778cb77b5652e6b17faf/dmain/internals.d#L728). And when the program starts, it will generate temporary script (https://github.com/icy/pacapt/blob/f3ae5f8b1f8ec78590c1778cb77b5652e6b17faf/dmain/internals.d#L775) and then execute them https://github.com/icy/pacapt/blob/f3ae5f8b1f8ec78590c1778cb77b5652e6b17faf/dmain/internals.d#L342 . The dlang version has a pass-through option when you want to just skip all pacapt features (e.g, pacapt -v -- yum install is exactly yum install) -- that's a crazy thing you can ignore for now. To me, that way of invoking external scripts is efficient, unsafe, and yeah, it's still bash script so people may hate that ;)

pacapt has a lot of fixes and tricks, kind of 10-year bug fix, and it's quite a lot of complexity. It's very hard for me to have all of them in the details, and I'm getting lost when trying to make it simple. However, it's very natural and simple for me to write everything in Bash-like language, because it's quite easy to do these things in Bash: gluing commands (pipe), grepping, filtering,..., -- I really wish Bash can be improved instead ;)

Feel free to ask me any questions:100:

[rami3l]

@icy Thanks a lot for your reply! It turns out that even though I don't know much about the original project structure, I still broke my task into exactly these different pieces. So, I'd also like to share with you some details of my current work, so that we might come up with better ways to solve problems that might emerge...

1) Argument Parsing

This is not the difficult part for me. The argument parsing library I used is argparse, but in a way far from elegant. This library does not support flags as subcommands (only words, but so as many other argument parsers, so I chose just to stick with this one), so I managed to strip all the flags away from the package names, store the names directly for later, and let the parser deal with the flags only:

// stripTargets distinguishes between pacapt flags and package names.
func stripTargets(args []string) (cmd []string, keywords []string) { ... }

So if I want to add pacapt gems, I can just modify a bit that logic, and it will just work. I didn't add all the possibilities mentioned by README in the parser (yet), which will be done when we really have that many commands to implement, though.

2) Platform Detecting

This is where I got a bit confused. For a long time I just left this part blank and hardcoded the detection result to be "Homebrew":

// DetectPackManager detects the package manager in use
// TODO: Make this function REALLY detect package managers
func DetectPackManager(dryRun bool, noConfirm bool) PackManager {
    switch runtime.GOOS {
    case "windows":
        return &Chocolatey{dryRun, noConfirm}
    case "darwin":
        return &Homebrew{dryRun}
    default:
        return &Unknown{dryRun, noConfirm}
    }
}

Maybe now we can start to really do this part, but what's the most elegant way? What are the "special" cases to be aware of? Should we suppose that our users all have their package managers in their PATH, so that I can just use exec.Lookpath()?

3) Invoke Package Manager

As for the dispatch, I was suggested by a friend to use an interface (if it was in Python, maybe I'll just go for metaprogramming). I also created an Unknown instance of the interface, so that new package managers can quickly be cloned and modified out of it. The interface looks really ugly, but it gets things done (is there a better way?):

// PackManager represents a PACkage MANager
type PackManager interface {
    Q([]string) error
    Qc([]string) error
    Qe([]string) error
    ...
}

Also, after some research, I found out that it's not that worthwhile to stick with bash scripts. So instead, I offered the following functions to facilitate things...

// RunIfNotDry prints out the command if DryRun, else it runs the command.
func (hb *Homebrew) RunIfNotDry(cmd []string) (err error) { ... }

// CheckOutput runs the command and returns its output both to a string and to Stdout (ignored if DryRun).
func (hb *Homebrew) CheckOutput(cmd []string) (out string, err error) {
    ...
    p.Stdout = io.MultiWriter(os.Stdout, &outBuf)
    p.Stderr = io.MultiWriter(os.Stderr, &outBuf)
    ...
    out = outBuf.String()
    ...
}

Please note that the stdout/stderr is redirected (and tee'd if we want to check the output), so the output won't be a problem. If the command is interactive, maybe we just need to set the stdin similarly. And thus, our commands can run like this...

// Qu lists packages which have an update available.
func (hb *Homebrew) Qu(kw []string) (err error) {
    return hb.RunIfNotDry(append([]string{"brew", "outdated"}, kw...))
}

Finally, if really necessary, it's always possible to call bash -c.

4) Testing

I implemented a --dryrun option so that the program might work like this: (This also demonstrates the project's brew/cask support!)

$ go run main.go -S curl --dryrun
>> brew install curl
$ go run main.go -S gimp --dryrun
>> brew cask install gimp

... though I become quite unsure about how the testing can be done, especially automatically, and across platforms.

5) Homebrew Implementation: Extra Subcommands

I first saw the original homebrew_Rs() function, and then I realized that this part is more difficult than it seems, and there has been heated discussions, like here and here. So should I just call brew rmtree and tell the users to install this subcommand here, as we don't really have to maintain by ourselves the complicated logic that handles this operation?

Similarly, I think it's a good idea to invoke brew cu (this will do a user-friendly brew cask upgrade) when doing -Su.

Any advice is welcomed :)

[icy]

hi @rami3l , there are a lot of details. I will need to read that again. I will now have some quick feedback hoping that helps:

5) Homebrew

It's great if we can support them partially without other dependency. But if the subcommand provides safety and exactness we can just ask the user to install them.

4) Testing

We would go with some smoke tests. As I know, Travis has some macos support, we can add them if we need. For linux-based machines, I generate test scripts thanks to a ruby program: https://github.com/icy/pacapt/tree/ng/tests . The idea is simple: execute the command, and match them against some known pattern. This is the best strategy to make sure the program won't be broken at user runtime

3) System command invocation

Yes I also like the idea of using some system call instead. It would take a little effort to filter out the output (see https://github.com/icy/pacapt/blob/ng/lib/dpkg.sh )

[rami3l]

@icy It's nice to see your reply! It's the weekend again so I can continue working on this project.

The current structure of my project has made it rather easy to add basic support (i.e. the one-liner part) for a new package manager with some copy-paste work. However, as for the non-one-liner functions, I realized that my lack of knowledge about bash scripting has become a problem for me when it comes to translation without resorting to the bash -c workaround.

I am trying to make sense of bash, though it might be a bit slow, as I am just an amateur trying to learn something during my spare time @_@. It will be so nice if you wish to lend me a hand implementing some of the functions, like -Q, -Ql, -Qs in impl_dpkg.go. (Thanks anyway.)

Again, feel free to ask anything :)

[icy]

@rami3l I think you have a good framework to start now. You may want to have some tests, and start porting the bash things. That's quite a lot of work :100:

[rami3l]

@icy It's been a month and things are going quite well!

I think the best way to keep the momentum is to get happier while coding (I kind of hate the copy-paste approach in my original Golang implementation). So it took me quite a while to decide what to do, and I have finally finished rewriting the whole Golang code base in Rust, the language of my preference, and a language which handles multi-platform tests quite well AFAIK. (Another reason is that clap.rs is said to add pacman-style argument parsing support in the near future.)

I'm actually using this Rust version of wrapper on my Mac on a daily basis. Also, I came up with a way to do integration tests. I'll write them once I have the time.

Let's see how far I can go from here!

PS: The project has been renamed to pacaptr since Rust doesn't allow project names with a dash.

[icy]

Amazing and congratulation for the first success :100:

I don't know Rust much (At my work people are using Golang). I used to play with Dlang in my free time. However I know Rust is a better choice for many purposes. I'm happy that you've found the right tool and also get tool worked on your daily basis.

PS: I don't have much free time at the moment, it's hard for me to work on some hobby projects :(

[rami3l]

@icy I managed to build utilities to check stdout here, and you can see that I borrowed a bit of the test pattern from this project. (Thanks!)

Although I'm not quite sure, it looks like we have test cases for just a small part of package managers in this repo, so how's everything else guaranteed... or are they (?_?)

PS: It'll be so nice if you could add pacaptr to the Related Projects section of README.md :)

[icy]

@rami3l Thanks a lot, and congratulations. I've got some tough problems at my work and was not able to catch up with you.

Feel free to open a PR to mention your project there. I believe you don't need to rely on the original idea of pacapt, as that way you have more freedom. When your project is cool and stable enough, we can decide what to do with the current script then.

Regarding the tests, it's relying on contributors' works. Many of them have been contributing to the project, and as requiring tests in bash is not an easy task, tests are not enforced. I believe in the contributors , and it's also not very hard to spot the issues in the library functions -- most of them are very small . (The hardest part, in main or core, was heavily tested from my side)

Thank you very much , and bests

[icy]

Fixed in https://github.com/icy/pacapt/pull/131

[icy]

@rami3l

I'd like to get back to this ticket with some updates.

Now most parts in the library lib are ported to POSIX shell (starting with https://github.com/icy/pacapt/issues/149 and the 6-year old ticket: https://github.com/icy/pacapt/issues/62). There are few issues (lib/cave, homebrew: https://github.com/icy/pacapt/issues/175), but in general that's very great outcome -- I should have started the project in POSIX from the beginning instead.!

Your project is now mentioned a new section: https://github.com/icy/pacapt/blob/ng/README.md#similar-projects (There is also sysget which is very cool and minimal but I don't think it's flexible/extensible much like yours). I hope that's fine.

We have now more tests and also support some custom Docker image for testing purpose (https://github.com/icy/pacapt/tree/ng/tests). You may find any easy way to sync. them in your project. (kudo to @mondeja who has made a great effort to fix some issues, add new tests and operators.)

The project is using quite a bunch of different things https://github.com/icy/pacapt/blob/ng/ARCHITECTURE.md#history-languages . For learning purpose, it's just ... awesome :+1:

Regarding the future of the project, I'd suggest

• Add more and more tests (esp. having tests for homebrew)
• Making it stable and robust (correctly implementing some features on different targets is just hard)

As the general setup hasn't changed much, I hope you can find some easy way to update your project accordingly when applicable (tests, operation implementation).

Thanks.

[rami3l]

@icy Thanks for the update! In fact, it was your idea of remaking pacapt that made me a better developer in general.

What you've done is a really nice work, congrats! I still agree that technically, nothing beats sh in implementing something like this (especially not a compiled language such as D, Go or Rust).

At the same time, working on the Rust port is also an enriching experience for me: although I still haven't finished porting all your scripts, I brought your project to Windows, learned a lot about Rust, made a shiny interface, and even contributed to my upstreams. Most importantly, I finally have something written for myself to be used on a daily basis.

All of this started with this tiny issue, and this is just amazing! Thank you!

[icy]

Thanks for your sharing @rami3l . The tool and the idea may vary, but the open spirit would be the same and that I love to read from your message.

Shell is hard. I keep my eyes on this project https://github.com/oilshell/oil, but I've not seen any [work] place I can be ready for it. Give it more time. Maybe you want to help that project too.

What I am surprised from your Rust port is how it deals with the test DSL quickly/easily. I started with Dlang for a reason (it's quite friendly to work with new DSL in Dlang), and it's also easy to write new extensions for Bash (as you can see here https://github.com/icy/dusybox#a-bash-builtin-command). Is there sth we can do with Rust? I hope to have a little time to learn more :(

[rami3l]

@icy In terms of usage, how is a bash extension (dylib?) different from an executable binary?

[icy]

@icy In terms of usage, how is a bash extension (dylib?) different from an executable binary?

It'd be faster, and give some additional feature to Bash tool. One of the great examples is recutils (disclaimer: I haven't used them heavily tho.)

• library loading: https://www.gnu.org/software/recutils/manual/Bash-Builtins.html#Bash-Builtins
• how new feature readrec is added: https://www.gnu.org/software/recutils/manual/readrec.html#readrec

That said I've not used the feature heavily. It's because people tends to avoid bash in that situation. :dagger:

[icy]

I've tried the tests (the hello library from my dusybox)

$ make smoke-tests
:: Smoke-test bash_builtin_hello...
dz_hello is a shell builtin
Hello, world. It's Hello builtin command written in Dlang.
Hello, Foo bar
:: [dz_hello(loaded)] Invoke 1M times...

real    0m13.149s
user    0m7.751s
sys     0m5.242s
:: [echo] Invoke 1M times...

real    0m11.311s
user    0m6.147s
sys     0m4.980s
:: [/usr/bin/echo] Invoke 1M times...
# still waiting, =)))

[rami3l]

@icy If this dylib is exposing nothing more than a few "extern C" stuff, then I can't see anything stopping one from doing the same thing in Rust...

PS: Speaking of coreutils, we now have a cross-platform Rust rewrite of the GNU coreutils at https://github.com/uutils/coreutils.

[icy]

@icy If this dylib is exposing nothing more than a few "extern C" stuff, then I can't see anything stopping one from doing the same thing in Rust...

Got it. Thanks for the clarification.

PS: Speaking of coreutils, we now have a cross-platform Rust rewrite of the GNU coreutils at https://github.com/uutils/coreutils.

Great. I think I learnt about that from HackerNews, but I haven't tried that yet.

[icy]

@rami3l Would you mind recommending some book on system programming in Rust? Thanks.

[rami3l]

@icy You really don't need to buy any book. The official tutorial (The Rust Programming Language) is good enough to start your journey, and you can go to the doc site to read the Reference, the Rustonomicon, etc. when you want to know something more advanced.

[icy]

@icy You really don't need to buy any book. The official tutorial (The Rust Programming Language) is good enough to start your journey, and you can go to the doc site to read the Reference, the Rustonomicon, etc. when you want to know something more advanced.

I have a problem. I can't read anything from laptop or reader device (which I don't have.) Yes I do but that's not how I want to spend time with my hobby :dancers:

[rami3l]

@icy Lucky for you, the tutorial (a.k.a The Book) also comes with a printed version (and quite expensive in Europe, huh?), but please remember that Rust is a quickly evolving language, and the Book, as I remembered, is already a bit different from what I have read when I was learning Rust. Actually, now pacaptr is more of a notebook to me. I'm no longer using just what I need to build this project, but rather trying to fit whatever looks appropriate to me into it, so that I can remember the best practices of Rust.

[icy]

@icy Lucky for you, the tutorial (a.k.a The Book) also comes with a printed version (and quite expensive in Europe, huh?),

Oh I will try again. Yes in Europe books are expensive :dagger:

but please remember that Rust is a quickly evolving language, and the Book, as I remembered, is already a bit different from what I have read when I was learning Rust.

Yes I know. I already have some out-of-date books here, but it's fine. When I have new thing you can compare with old ones. With online resources it's a little to hard to do that (or I don't know how to do)

Basically I have a few weeks off soon, and I need to take a break with new things :P

[rami3l]

@icy I always forget to add architecture.md to my project, but I'd love to give you a comprehensive introduction if you want a guide through my project (i.e. my notebook) XD

[icy]

@icy I always forget to add architecture.md to my project, but I'd love to give you a comprehensive introduction if you want a guide through my project (i.e. my notebook) XD

Thanks for your suggestion. I'm happy to hear that. Please give sometime to write architecture.md , as I think it's better option for everyone (not simply me). But I definitely need to ask you many things once I will be gettin myself onboarded on some Rust code ;)