Add opApply for UDAs, and lexicallize properly methods and functions

Dadoum commented 3 years ago

Description

UDAs could be way more powerful if we could manipulate the element they are attached to. It would permit to get types and methods and mixin them or even better implement them if they have no implementation.

This would require a better way to handle methods and functions as template types, since they currently require to use "(T...) if (T.length == 1)" and there is no way to get their body.

It could even allow to write code before and after, and allow Aspect-oriented programming:

Example (with terrible syntax, should find something better):

@tracedMethod void functionThatCanCauseProblem()
{
    writeln("Some dangerous operation");
}

struct tracedMethod
{
    void opApply(T)()
    {
        if (is(T U == A function(P) U)) // this will also ensure that T has a body; A would be the attributes, P tuple of params types, and U the original body
        {
            T = A function(P) // or A T(P) ?
            {
                writeln(T.stringof ~ " IN");
                auto ret = U(P);
                writeln(T.stringof ~ " OUT");
                return ret;
            }
        }
    }
}

What are rough milestones of this project?

Find a cleaner syntax to use methods as template arguments, and if possible find way to get their keyword separately like "static" or their body as "type".
Determine how we would implement an opApply(T)() for UDAs.
Create a DIP from what came out of the previous steps.

How does this project help the D community?

Improve meta-programming capabilities of D, add aspect-oriented programming, and simplify some mixins.

Example:

public class ImportedMethods {
    mixin importMethod(void function(), "init", "externalClassCtor");
}
// will become
public class ImportedMethods {
    @importMethod("externalClassCtor") void init();
}

Recommended skills

Knowledge about D parsing

maxhaton commented 3 years ago

https://github.com/dlang/DIPs/pull/196

Take a look at this.

UplinkCoder commented 3 years ago

core.reflect should actually help with this.

UplinkCoder commented 3 years ago

I am interested in the use-cases you have.

maxhaton commented 3 years ago

core.reflect should actually help with this.

I think core.reflect would be like using a sledgehammer to pet a dog in this particular usecase (if it could, nice, but writing a bench of reflection code just for a UDA seems like a lot of work).

My particular usecase was (amongst a few other things that I've forgotten): Using magic UDAs to write things like benchmarks and tests that effectively run themselves (and avoid having to walk the entire projects tree to be able to find things).

i.e. You can write something like this

@BenchmarkOverRange(/* size of dataset */ iota(1, 1000), generateRandomArray!int, validate)
void addOne(int[] data)
{
  foreach(ref elem; data)
    ++elem;
}
// no module level mixin required.

You can actually already do this with magic (and really expensive) templates - get parent, walk parent, find UDA - but they are unbelievably brittle and break in random places.

I still have an implementation of the DIP I linked somewhere. The implementation is dead simple, only issue is that walking upwards in the tree is actually quite hard since dmd doesn't really have a unified notion of something i.e. working out where you are in the tree involves testing a handful of things and walking each pointer far too many times - this is not as fast as one would like but the real issue is bugs where UDAs get attached to the wrong symbol or you end up walking all the way up to module scope by accident. Finding a kernel to test and trust was hard enough that I gave up.

Also it may be worth keeping in mind that a single UDA can be attached to multiple symbols.

Dadoum commented 3 years ago

Here is a snippet from a program I am writing:

extern(C++, example) struct AndroidHelloWorld {
    mixin AndroidClass!AndroidHelloWorld;
    @Import static void sayHello();
}

It loads class from an android library, but I need to specify the mixin to be able to find these methods. If something was called when I put Import, this would avoid the iterations I am making to seach these symbols. In addition, if I import something that is not in a class, I need to mixin all of them one by one.

About your DIP @maxhaton , I think it should not use __UDA_ATTACHED_TODECLS_\, but more be like:

struct override UDA(alias anyUDA) if (isCallable!anyUDA) { // make UDA accessible only for methods, override symbol (maybe "alias anyUDA = _UDA_DECL_" ?)
    static ReturnType!U opCall(Parameters!U params) // Problem here, how to include if anyUDA is an instance method or static one ?
    {
        writeln("Called " ~ __traits(identifier, U));
        return U(params);
    }
}

maxhaton commented 3 years ago

Here is a snippet from a program I am writing:
extern(C++, example) struct AndroidHelloWorld {
    mixin AndroidClass!AndroidHelloWorld;
    @Import static void sayHello();
}
It loads class from an android library, but I need to specify the mixin to be able to find these methods. If something was called when I put Import, this would avoid the iterations I am making to seach these symbols. In addition, if I import something that is not in a class, I need to mixin all of them one by one.

About your DIP @maxhaton , I think it should not use __UDA_ATTACHED_TO_DECLS__, but more be like:
struct override UDA(alias anyUDA) if (isCallable!anyUDA) { // make UDA accessible only for methods, override symbol (maybe "alias anyUDA = _UDA_DECL_" ?)
    static ReturnType!U opCall(Parameters!U params) // Problem here, how to include if anyUDA is an instance method or static one ?
    {
        writeln("Called " ~ __traits(identifier, U));
        return U(params);
    }
}

Why is my proposed solution not sufficient? Mine is much simpler for starters

Dadoum commented 3 years ago

Because it does not carry context applied element (like which overload, or is the UDA made for class, struct). It does just carry the name of the symbol, which can't be used easily to get by example the mangled name of it, or its location (in which module it is). But it does matter only in a some really specific cases like mine.

maxhaton commented 3 years ago

Because it does not carry context applied element (like which overload, or is the UDA made for class, struct). It does just carry the name of the symbol, which can't be used easily to get by example the mangled name of it, or its location (in which module it is). But it does matter only in a some really specific cases like mine.

Yes it does. In the following example, I have emulated the behaviour of the proposed DIP myself (i.e. the fully qualified name)

import std.traits;
template MagicUDA(string[] args)
{
    static foreach(x; args)
    {
        static if(is(typeof(mixin(x))))
        {
            pragma(msg, typeof(mixin(x)));
        } else 
        {
            pragma(msg, __traits(allMembers, mixin(x)));
        }
        pragma(msg, __traits(getLocation, mixin(x)));
    }
    enum magicUDA;
}
@MagicUDA!([fullyQualifiedName!func])
int func()
{
    return 0;
}
@MagicUDA!([fullyQualifiedName!Monkey])
struct Monkey
{
    int x;
    string legs;
}
void main()
{

}

https://run.dlang.io/is/V9oO30

Dadoum commented 3 years ago

Won't it be better to have it directly (and so getting the full qualified name from these) instead of having these as a string and having to mixin them before using them for advanced functions ?

import std.traits;
template MagicUDA(args...)
{
    static foreach(x; args)
    {
        static if(is(typeof(x)))
        {
            pragma(msg, typeof(x));
        } else 
        {
            pragma(msg, __traits(allMembers, x));
        }

    }
    enum magicUDA;
}
@MagicUDA!(func)
int func()
{
    return 0;
}
@MagicUDA!(Monkey)
struct Monkey
{
    int x;
    string legs;
}
void main()
{

}

(also if I prefer running them individually over an array (string[] or args...) it's because static foreach makes intellij-dlang and Mono-D parse code incorrectly)

dlang / project-ideas