Allow inline generation with custom placement

[cmeeren]

My project structure is generally like this:

- Domain.User.fs
- Domain.Order.fs
- ...

Each Domain* file contains first the types relating to the domain (e.g. the User type itself), then the modules with functions (e.g. the User module).

I want to use the lens generator. However, it seems like currently, Myriad can only output to either a separate file or the end of an existing file. That means that since the domain modules use lenses, I will have to double the amount of files in my project (e.g. Domain.User.Types.fs with Myriad output at the end and Domain.User.Functions.fs after that). Instead, I want the lenses generated in each domain file below the types but above the modules that use the lenses.

It would therefore be great if Myriad provided an option for me to specify where to insert the generated code.

For example:

[<Generator.Lenses("lens")>]
[<Generator.OutputInlineBetween("myriad-start-lenses", "myriad-end-lenses")>]
Type User = ...

// myriad-start-lenses

// myriad-end-lenses

Above, I manually add the comments myriad-start-lenses and myriad-end-lenses to the source file, and specify to Myriad using a new attribute OutputInlineBetween that specifies what Myriad searches for. (Myriad can fail if finding no or multiple matches.) Myriad then replaces everything between the matching lines with the generated output.

Ideally, Myriad should respect the indentation of the comments. For example:

module User =
  // myriad-start-lenses

  // myriad-end-lenses

Above, Myriad should insert the code at the same indentation level as the comments.

Ideally it should also be able to specify the same start/end comments for multiple generators, and Myriad then places the output for all of them in the specified location.

[7sharp9]

One of the problems with this is plugins currently use <MyriadInlineGeneration>true</MyriadInlineGeneration> which passes a command line option to the myriad sdk which modifies the way the output of the plugin is used. If an attribute is used then the plugin would have to be pre processed so that Myriad knew where to put the code.

Another problem is Fantomas is used to format the code so theres only the options that fantomas can accept to alter its output so Myriad would probably have to count the indentation in the start/end comments (What if they differ) and then write line by line the output with the indentation as prefix.

Reusing the start/end for multiple generators, this sounds the hardest, how would any plugin know if another plugin had already generated code in that start/end, Myriad would probably have to keep track of this information, or, filter plugins by generation type (newFile, inline, inlineStart/End) and process each type sequentially. In the case of the inlineStart/End you would be adding the contents of the file up to the start comment, appending all plugins for that start/end, then inclusively adding the code following the end block.

[7sharp9]

I'm going to go ahead and say I probably won't implement this. Ir seems to be a very niche case and would not add much to the overall scope of myriad, yet take a lot of effort to think about and implement.

[cmeeren]

Understandable. I think F# RFC FS-1023 Allow type providers to generate types from types is ideal for what I want, though unfortunately not immediately on the horizon.

[7sharp9]

Yes it’s been dormant for years now. I don’t rule out this feature, but I would probably require assistance.

Sent from my iPhone

On 15 May 2022, at 09:36, Christer van der Meeren @.***> wrote:

 Understandable. I think F# RFC FS-1023 Allow type providers to generate types from types is ideal for what I want, though unfortunately not immediately on the horizon.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.

[cmeeren]

From your perspective, is there a way to modify my request to make it more feasible to implement? For example, I'm fine with not using attributes. My goal is only to be able to generate lenses without doubling the amount of files in m codebase (and also without needing rec modules, which makes the editor/error experience worse and also doesn't protect you from bad cyclic dependencies). In other words, I'd like to stick with Domain.User.fs which contains (in order) types, lenses, and functions, instead of needing Domain.User.Types.fs containing types and lenses, and Domain.User.Functions.fs containing the functions. (That would also prevent me from using the private modifier on types that should only be internally accessible to the User module functions, since they are then defined in a separate file.)

[7sharp9]

So why can’t intrinsic extensions appear at the end, because of the added modules after, or because of the namespace they are in rather than a rec module etc? I’m mainly hesitant because I’m writing to users files, I don’t want to break things!

Sent from my iPhone

On 16 May 2022, at 06:21, Christer van der Meeren @.***> wrote:

 From your perspective, is there a way to modify my request to make it more feasible to implement? For example, I'm fine with not using attributes. My goal is to be able to generate lenses without doubling the amount of files in m codebase. In other words, I'd like to stick with Domain.User.fs which contains (in order) types, lenses, and functions, instead of needing Domain.User.Types.fs containing types and lenses, and Domain.User.Functions.fs containing the functions. (That would also prevent me from using the private modifier on types that should only be internally accessible to the User module functions, since they are then defined in a separate file.)

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.

[cmeeren]

My Domain.User.fs file is like this:

type User = { Id: UserId; Username: Username; FirstName: FirstName; ... }

module User =

  let create ...

  let setFirstName ...

So: The User type definition first, then a module with the same name that contains functions for whatever you can do with a User. It's these functions that use lenses. I currently have my lenses (manually) defined at the top of the User module.

If I am to use Myriad for generating lenses (which I'd like to, because lens definitions are completely mechanical), I need the lenses defined between the type and the module. Currently, the only ways to do that are 1) make the top-level namespace/module in Domain.User.fs recursive (and append the lenses at the end), which means Intellisense is less helpful and I'll lose out on one of the great benefits of F#, or 2) split the file into one file for the User type definition + Myriad-generated lenses at the end, and another file for the User module with all the functions, which doubles the amount of Domain.x.fs files (I have more than just User), and also means I can't as easily have type User = private { ... } in order to properly protect the internals/invariants of the User type, but still leave the fields visible to the functions in the User module (e.g. for the create function).

Does that answer your question?

[7sharp9]

Yep, leave it with me I'll have a think, probably a marker attribute specified in the generator might be better than comments.

Incidentally, Intrinsic type extensions added to TypeProviders also has a use in this area, thats why I initially worked on that RFC and compiler PR :-)

[7sharp9]

Also, if I could rely on compiler parsing of the type, they could be placed after the type definition is finished, sometimes F# range of the type in the AST reports the wrong range though [<Generate(InlineAfter)>] would of been perfect.

[7sharp9]

Just checked with ast output on a simple record and it looks like the range starts is correct so this may be an option, an attribute starting inline after the attributed type.

[7sharp9]

The problem with this is the lack of an end to the generated code so myriad knows what to cut out the next time it runs.

[cmeeren]

This may be a bad idea (or impossible), but how about decorating all the Myriad-generated members with a marker attribute? Then you can remove those (wherever they are) and insert the new ones at the start. If users happen to insert custom methods in between Myriad ones, those methods would just end up after the new Myriad ones.

In any case, I'm not really sure I'm personally that interested in methods on the actual type definition. I like to start by defining all my types at the top of the file, and then all the behavior comes later. This makes it easy for me to see just the type definitions, which generally tells you a lot in F#. If there were many methods in between the type definitions, getting that "overview" of a domain via the type definitions would be harder. (To be clear: Most of my domain files don't just have a single type; for example, Domain.Order.fs can have the main Order type, but also OrderLine and other child types, as well as "enum DUs" etc.)

[7sharp9]

The thing that makes this a little tricky is not all plugins have an input AST, so its more in the scope of what a plugin itself is doing, the next problem is the generation of actual code is outside of the plugin so as far as the plugin is concerned it gets some input, and it either returns an ast or some source text, that test is then inserted in either the output file or the input file at the myriad generated code header, so maybe the best mix of these ideas is your original one of using comments. I do like the generated code attribute though it just means reworking the CLI tool to integrate differently to the plugins. I mean thats fine, it's just finding free time to do all that that's all.

[7sharp9]

Myriad works as a library rather than as a framework so there's no such enforcing of how things are done, just lots of help to get there, so enforcing the addition of the codeine attribute could be problematic if I integrate code deletion based on that, to be honest, there are no users of inline that I know of beyond alpha/beta trials.

[7sharp9]

The flow of config goes from myriad to the plugin, so there's no communication from the plugin back to myriad, so the setting of where the generated code goes between has to be supplied at either the myriad.toml or fsproj file