Add ability for dictu to load user modules

[briandowns]

Add ability for Dictu to load user modules

What's Changed:

Well, the time has come... :D The first take at a package manager. This change builds upon the change that moves the Dictu history.txt file to the user's home directory and adds an additional directory called "modules".

The "package manager" I wrote is located here.

Type of Change:

• [ ] Bug fix
• [x] New feature
• [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)

#

Housekeeping:

• [ ] Tests have been updated to reflect the changes done within this PR (if applicable).
• [ ] Documentation has been updated to reflect the changes done within this PR (if applicable).

#

Screenshots (If Applicable):

Below is the code that I tested this change with. The "slog/slog.du" file resides in ${HOME}/.dictu/modules/slog. I'm purposely import JSON here because it's also imported in the slog.du file and wanted to make sure it'd only be loaded once.

import JSON;
import System;

import "uuid.du";
from "slog/slog.du" import Log;

const log = Log();
log.info("yup...");

System.exit(0);

[Jason2605]

This is an interesting concept, and similar to how Python handles it where packages are essentially “global”. This can lead to troubles when you’re wanting different versions with different projects.

I think it may be interesting to think about how JS handles it for example where the .dictu_modules would actually be in the project repository instead so modules would be located there and be installed per project - thoughts?

As a side note, once modules have been imported they’re stored in a table so won’t get executed multiple times, so feel free to import where you see fit

[briandowns]

This is just POC. I have thought to a dictu_modules dir, or the old Go way of things using a vendor directory.

I was going to use the dictu path env var but wanted a simple example first. To show user mods versus system mods.

As for versioning, the script requires a config file basically and would look at that for versioning decisions. Would likely require different directory naming based on version also.

I'm happy to adjust in any direction you prefer. Just wanted to get this stuff going since I'm often using my structured logger, wanbli, and kahless amongst others.

Thanks for the consideration!

[Jason2605]

I was thinking for the change to the interpreter here though, do you think it makes sense to have the external modules directory at a project level (i.e in the path of the project rather than home)?

Yeah versioning is going to be a nasty issue for projects, it's just if we deal with it on a project level at least there should be no cross-contamination between projects or I guess even Dictu versions?

I'm happy to adjust in any direction you prefer. Just wanted to get this stuff going since I'm often using my structured logger, wanbli, and kahless amongst others.

Yeah for sure, definitely appreciate the work you're doing on this!

Also one last comment for the code itself, we probably need an additional check to ensure it also exists in the module directory

[briandowns]

If we go to a project specific approach, I don’t know if we need to make any interpreter changes. We can just import from the path itself. We would need to establish a new convention that “external” dependencies are placed in a specific directory. I'm partial to mods, pkg, deps. From there, I can adjust dictum to place dependencies there.

[Jason2605]

Yeah, it’s whether we want it to automatically search for this folder (like you’re doing already in home but within the designated folder name chosen) kinda like how JS does with node_modules for example - what do you think?

[briandowns]

I'll update this PR to look in a directory in the project and see how it goes. Would make import lines a little cleaner.

[Jason2605]

Nice one, appreciate it!

[briandowns]

Hmmm. This is going to end up being tricky. trying to figure out where each file is in relation to the project dir itself and how to get the files loaded from the modules directory.

[Jason2605]

I would just go with something simple to be honest. Make it relative from where the script is executed, so regardless of project directory the "vendor" import would always be top level (since we assume you're executing the script from the project dir root).

E.g

MyProject
    | Vendor
        | kahless
            | kahless.du
    | MyDirectory
        | run.du

run.du

 from "kahless/kahless.du" import Kahless;

[Jason2605]

Nice one, thank you!

[briandowns]

github.com/briandowns/dictum is up-to-date now with these changes. The next feature I'm going to be adding is the ability to pull from github. I've also started writing "dmi" the Dictu Module Index. All of these things are open to change in any direction. It's more to solve my problems but would love your feedback on all of it.

[Jason2605]

Awesome stuff!! Being able to pull from github will be massive, so hugely appreciated