Add jsonFromFile to vibe.d util code

[zhaopuming]

I found that a project needs to read a config file for its own use, so a default config.json is preferrable, or at least vibe.d could provide a jsonFromFile util function which is defined in dub's source code.

[etcimon]

I was suggesting something along those lines.

Each project would have a config.d file that defines the dependencies' settings objects. I feel this would better preserve the D programming style while making it scoped properly and simpler to implement, e.g.

/*
    Copy this in your config.d file to configure the package:
    public import vibe.web.userman;
    class UserManSettings {
        bool requireAccountValidation = true;
        bool useUserNames = true; // use a user name or the email address for identification?
        string databaseName = "test";
        string serviceName = "User database test";
        URL serviceUrl = "http://www.example.com/";
        string serviceEmail = "userdb@example.com";
        SMTPClientSettings mailSettings;

        this()
        {
            mailSettings = new SMTPClientSettings;
        }
    }

*/

In the new WebService defined in vibe.web.web, after you import your config, you can use mixin UserMan!UserManSettings, to add user management and automatically route it according to those settings, and so the behavior is well defined. If multiple WebServices are defined, nothing stops you from making a config2.d and importing that one for a different definition.

[s-ludwig]

I think that the ticket is really meant for run time (user) configuration. And indeed it's frequent enough that such a utility function in vibe.data.json makes sense. I'll add one.

---

But your suggestion sounds a lot like you mean compile time "traits" (in the past I though you meant normal run time settings classes, like they are used today). I agree that this would be a good idea for the REST and web interface generators. I'd use struct and enum, though (it's easier to use during CTFE and seems less like it should be used to instantiate run time objects):

struct MyRestInterfaceTraits {
    enum methodStyle = MethodStyle.lowerUnderscore;
    alias serializer = JsonStringSerializer;
    // ...
}

registerRestInterface!MyRestInterfaceTraits(new MyRestAPIImplementation);

instead of a separate struct, it could also be part of the declaration:

interface MyRestAPI {
    struct Traits {
        enum methodStyle = MethodStyle.lowerUnderscore;
        alias serializer = JsonStringSerializer;
        // ...
    }

    // ...
}

registerRestInterface(new MyRestAPIImplementation);

[etcimon]

The traits are pretty interesting but, in fact I was talking about settings that could be runtime too, like the ones from the user manager (UserMan).

I'm going to be building my web projects with Vibe.d from now on, and I want to partition them into extensions (sourceLibrary) that I would be made available for other Vibe.d developers to add to their vibe.web.web projects (user managers, blogs, remote api clients like oauth, openid, google apis, google maps, etc).

I think the current way of handling the settings is a little hard, I'd like them to expect a config.d file with the settings "class" there, they should have to copy the class to their project's config.d and modify the defaults so that they are instantiated that way. It's easy because then all the configs of the dependencies are in a single file. And, for the RegisterInterface part, allow to mixin the WebService methods that are packaged for them. In other words, I want to allow them to register it to their router through their WebService instance by making all my dependencies' WebService implementation into a template mixin, so it requires no changes to the original package to include them.

Think of it a bit like making easy installation possible like with joomla's extension directory here: http://extensions.joomla.org/ or wordpress plugin directory: http://wordpress.org/plugins/business-directory-plugin/ but for a vibe.d project with no web management panel included yet. It wouldn't have to be too difficult to create a management panel in the future that updates dependencies / re-compiles with new settings when a package's configurations are changed ;)

[s-ludwig]

Okay, but why not simply have this in the "config.d" file:

UserManSettings getUserManSettings()
{
    auto ret = new UserManSettings;
    ...
    return ret;
}

and then router.registerUserMan(getUserManSettings()); (or maybe using a mixin if it is required for some feature).

It could still be in one file, but doesn't produce template bloat or templates in general when they are not needed. The thing is that the overhead that is produced by making it compile-time can't be exploited anyway if run time configuration is still supported, so a non-template solution would IMO be much cleaner here.

In general I like to avoid templates and mixins when possible, at least for public APIs. Although they are nice and flexible, they don't come without costs of various kinds.

[etcimon]

I didn't know about any downsides of mixins. I'd opt for any solution that would be copy/paste into a well-defined file (config.d?). Something intuitive that makes it impossible for developers to wonder if we have to be choosing function names, return types, or what values should variables take...

User router.register.... wouldn't be possible unless I found a workaround because I intend on using WebService to make those sub packages.

I don't know if everyone has an IDE either, so having to guess the possible values or where to find the possible values is a downside. Being able to have comments would solve that issue, which means json is a bad candidate.

That's why I thought of just copying the struct settings into a common config.

I'd like to be able to do something like this:

class MyWebSite
{
    private SessionVar!(string, "someImportantVar") m_important;
    mixin UserMan!settings; /// notice here all the routes are registered when registering MyWebSite

    class SomethingElse {

        void getSomePage() {
            auto user = m_userman.user; /// accesses public SessionVar property
        }

    } somethingElse;
}

• edit: I'm already able to do this ;) I'd just want to consult before making it the norm. There has to be some acceptable way to instantiate and access public properties of dependencies, for many reasons, like having event triggers/callbacks as well for making plugins. e.g. a community builder dependency could be compiled with a profile statistics plugin using event triggers/callbacks, like a different library would do m_community.onProfileView ~= &handler.

[s-ludwig]

What about implementing opt-in support for JSON comments?

[s-ludwig]

Something like this: http://developer.android.com/reference/android/util/JsonReader.html#setLenient(boolean)

[etcimon]

Being able to recognize /* */ or // would make JSON a good candidate, we only need to decide where to put those files and how to call them .

[etcimon]

Something like this would be good because once a new library is added to config.json, its public members are available anywhere in a new project's WebService class through the identifier. E.g.

class MyService
{

    void getCustomProfile() {
        render!("CustomProfile.dt", m_userman.user);
    }
}

You can access the user object from the library's public methods because the config.json generated m_userman and made MyService as a nested class, so all the boilerplate was automatically written to make it available. Here's how:

CreateWebService!(MyService,"config.json")(router);

config.json:
{
    "userman": {
        "objectName": "UserMan"
        "identifier":   "m_userman",
        "useUserNames": false,
        "databaseName": "title",
        "serviceUrl":   "http://www.test.com/",
        ...
    },
    "vibenews": {
        "objectName":   "VibeNews",
        "identifier":   "m_vibenews",
        "dependencies": [{ 
                "name": "userman",
                "identifier": "m_userman"
        }],
        "title": "Example Forum",
        "hostName": "forum.example.org",
        ...
    },
    "myservice": {
        "objectName":   "MyService",

    }
}

// This class is auto-generated and executed in CreateWebService!
class WebServiceImpl(alias MyServiceMX) {
    UserMan m_userman;
    VibeNews m_vibenews;

    mixin MyServiceMX; // yes I know, we have to put the main WebService in a mixin template..

    MyService m_myservice;

    Json settings;
    this(Router router){
        m_userman = new UserMan;
        m_vibenews = new VibeNews(m_userman);
        m_myservice = new MyService(m_userman, m_vibenews);
        settings = jsonFromFile("config.json);
        m_userman.settings = Json["userman"];
        m_vibenews.settings = Json["vibenews"];
        m_myservice.settings = Json["myservice"];

        router.registerWebService(m_userman);
        router.registerWebService(m_vibenews);
        router.registerWebService(m_myservice);
    }
}

What do you think of this Sönke? I'd sure love to make it this simple to add components to a vibe.d project.

[s-ludwig]

For one, I'd like to keep the modularity that we have now and not introduce any reliance on run time files (let alone at a specific location). I think it should be possible to build something on top of the components instead of building the components on top of a configuration module. Ultimately it must be the choice of the final application how to manage its configuration. But DUB definitely shouldn't know anything about this - earlier when it was still tied to vibe.d that might have been a good idea, but now it's supposed to be a general tool and not cater specifically special use cases.

One idea could go like this:

auto settings = loadConfiguration!(UserManSettings, VibeNewsSettings, MyServiceSettings)("settings.json");
auto userman = new UserMan(settings.userMan);
auto vibenews = new VibeNews(userman, settings.vibeNews);
//...

So loadConfiguration would return a generated struct types that contains all the configurations deserialized from the given configuration file. Instead of JSON, I'd opt for SDL, which will also be usable for DUB package descriptions.

[etcimon]

That would be great, SDL is a good choice and yes I agree that building shouldn't rely on the way settings are implemented, the original purpose is to standardize a simple way settings are set for a dependency while preserving flexibility. What about the CreateWebService! part? I'd certainly like to see also a simple, standardized (not obligatory) way components are included in a project, like as a member of the main project's registered class. The idea would be to make public members of the instance available throughout the main project, so that any sub package can be used as an API, in a way that doesn't force users to read through the dependency projects to understand how they could achieve it. I'd like to build all my sub packages in a certain way so that installation instructions stay the same

I'm not sure auto vibenews = new VibeNews(dependency1, dependency2, ..., settings.vibeNews); would be the most simple way down the road if dozens of components are used throughout a dozen subpackages, which users should learn the dependencies of and send the appropriate instances of each to each of them. I know it's a concern for a problem bit far down the road but I'd rather have that tackled sooner than later

[etcimon]

Couldn't we have something around the lines of:

router.registerWebService!(settings, UserMan, VibeNews, MyService); *edit

Which takes care of passing the new instances properly to each object, maybe making them indirectly available publicly to other objects. That would make installations really simple in cases where there's 12 of those chained together to be resolved in some "spiderweb" sort of dependency.

e.g. mongodb is used by oauth, community, administration, blogs, .. oauth is used by users, facebookslider, google+counter users is used by community, administration, blogs, community is used by profilematcher, banners, search, ...

profilematcher might need to consult the community object's cached! object like this m_community.profile.find( { name: " .." }); (which consults users and dispatches through the "profile" db controller). If we want instances of the same objects to be re-used throughout the dependencies of the project is has to be registered manually.

Banners could have to setup a callback in the blogs to fetch an object on every view... etc

There can be many types of packages: plugins, modules, extensions, etc. which are adapted for components in a sort of spaghetti dependency scheme, relying on an API from each of those objects.

This could go on for ages, lists of 100s of components eventually. Registering them properly in a project could become cumbersome and repulsive if the dependencies must be resolved manually in the main project. I don't know if you see what I mean.

[zhaopuming]

+1 for comments in JSON, also it would be great if we could ignore the quotes in keys:

config.json :

{
    a : 1,  // keys can be unquoted, just like Javascript
    b : ["a", "b", "c"]
}

[s-ludwig]

BTW for configuration files, there is also sdlang-d, which I like very much for its syntax. Then there is also dyaml, which would also solve the quotes around identifiers issue - and yamkeys, which is a configuration library based on top of it (didn't take a closer look, though).

[zhaopuming]

Yes, I've heard of sdlang-d, I seen the discussion about yaml vs json, I want the loosely validated json because:

1. It's simple and useful enough. by adding comments and ignoring key quotes I'm all happy with it.
2. package.json uses it, so it's more formal. (I'd like to comment off some part or take a note in it too)

[etcimon]

I like sdlang-d better, it makes json look full of clutter and I like the ability to be much more type-specific while being less verbose when using it for configurations, it would also enable to tweak specific compile-time errors. I'd love to have it as a compile-time engine to eventually replace package.json and possibly to have an object settings parser directly in vibe.d. People should be forced to write their application settings through a specific interface. & finally, when you think about it, with the appropriate syntax highlighter, json blushes next to sdl .