Suggestion: Add more examples

[briochemc]

Great package! :) I would love to see more examples, in particular those that are mentioned in the Read.me:

This can be useful for attaching Bayesian priors or optional units to each field.

I tried something like that:

using Flatten, Tags, Unitful
@flattenable @units struct ParaUnit{T}
    l::T | u"m"   | true
    t::T | u"s"   | true
    c::T | u"mol" | false
end

But it gave me the error

ERROR: LoadError: UndefVarError: @units not defined

I saw one of the commits was to "remove units", so I am not sure if this was given up on, but I could use it!

(Edit: code formatting)

[rafaqz]

Yeah its all a bit vague still. Attaching other metadata tags is done using Tags.jl (which I will probably name-change to FieldMetadata.jl) so consult the readme over there and let me know if that doesn't explain it well enough.

But you either need to make your own tag @tag units nothing or import Tags: @units, units to use a shared tag, which is probably better. Then your example should work.

The commit about removing units was referencing that all the automatic unit stripping in Flatten.jl has moved to UnitlessFlatten.jl because not everyone will want units stripped every time.

[rafaqz]

Also generally you should just add units when you initially construct the type.

I only need optional units when I'm using Dual numbers and I need to wrap the fields in the Dual type first, and then apply the units afterwards, or when I'm using UnitlessFlatten.jl and I need units stripped everywhere, but want to reapply them to show up in an Interact interface or something. I'm not even sure this is a good way of doing it yet.

[briochemc]

I think we are dealing with similar problems/ideas, so I'll try to materialize some of the things I would like to have. First, I would like to have a default unit for each parameter. Second, I would like to have a printing unit (e.g., a is in u"m" by default, but I would like to print it as u"km" by default, although the SI unit u"m" should be used before stripping it whenever used in any calculation). I hope this makes some sense. Third, just like you, I would like to be able to optimize only some of the parameters and use DualNumbers but I will keep those DualNumbers-related questions for my next post 😄

So thanks to your reply, something like

using Flatten, Tags, Unitful, Parameters
import Tags: @units, units
@flattenable @units @with_kw struct Para{T}
    a::T = 1.0 | u"m"   | true
    b::T = 2.0 | u"s"   | true
    c::T = 3.0 | u"mol" | false
end
p = Para()
p = Para(3, 4, 5)

seems to work. Now I would like multiple add-ons: 1) That the constructor work with units, and raise an error if it cannot convert it. E.g., something like

    p = Para(a = 1u"km")

which should be the same as 
```julia
p = Para(a = 1000u"m")
```
And have
```julia
p = Para(a = 1u"s")
```
raise an error because the unit of `a` must be convertible to `u"m"`.
I am sure this is doable by overloading the constructor, but I don't really know how to do that... And I am getting hints that you already have done it or know how to 😄 

2) Have a printing unit. I am guessing I could do something like

    @tag printunits nothing
    @flattenable @units @printunits @with_kw struct Para2{T}
        a::T = 1.0 | u"km"   | u"m"   | true
        b::T = 2.0 | u"s"    | u"s"   | true
        c::T = 3.0 | u"mmol" | u"mol" | false
    end
    p = Para2()

where the printing unit could be used as the constructors unit as well as for, well, printing. That is, e.g., `p = Para2(a = 5)` will assume it is `5u"km"` and convert it to the default `5000u"m"` to assign the value to the field `a`. A printing function would also do the opposite, e.g., convert the default unit of `a` (which is `u"m"` in my example) to `u"km"` to show the user its value. Does that make sense? I do get a lot of such unit conversions in my models, and this could be useful to ensure I do not make unit mistakes! 

I'll stop here but I have a few more things in store to ask 😄 Please let me know if you would change anything in the code I put up here too, and if anything seems wrong or dumb 🙂

[rafaqz]

You could probably do some of 1 and 2 by swapping out Parameters.jl with my Defaults.jl package:

https://github.com/rafaqz/Defaults.jl

I override this function if I want to mess with the construction process.

I use something like this to add units during construction:

Defaults.get_default(t::Type) = begin 
    d = default(t) 
    u = units(t)
    add_units.(d, u)
end

add_units(::Nothing, u) = nothing
add_units(x, ::Nothing) = x
add_units(::Nothing, ::Nothing) = nothing
add_units(x::Number, u::Unitful.Units) = x * u
add_units(x::AbstractArray, u::Unitful.Units) = x .* u

Which means I can also easily build the types without units. But doing this in a package can be type pyracy, and I'm wondering how to formalise it a bit more.

For the printing units you can probably override show() or showcompact() for the type and set the units there. You could also just generally set upreffered in Unitful if you don't need field level control.

[briochemc]

I'm a bit lost... I already spent quite some time incorporating Parameters.jl to my functions 🙃 Does Defaults.jl clash with Parameters.jl? Also, if you have time, could you provide a bit more hand-holding to detail how to incorporate what you just presented here into one of the flattenable structs above? Could you also let me know if I can do something similar without replacing Parameters.jl?

[rafaqz]

They don't clash but using both would be a lot of duplication. Edit: they do clash because they are declaring the same constructors! I don't know which would win.

I wrote Defaults.jl because Parameters.jl doesn't seem to allow the kind of interactions you are describing easily, and because it's a large codebase that I don't understand. Defaults.jl is just a 40 line addon to Tags.jl that took me a few hours to write.

Basically all you have to do is use @default_kw instead of @with_kw and | instead of =. But you lose @unpack etc and the ability to chain keywords in the constructor, so there are tradeoffs. Maybe you can extend some methods in Parameters.jl to change the behaviour of the constructors as well, but it was easier for me to just to roll my own, and cut 700 lines of code from my dependencies at the same time. So I can't give you any help or examples of doing this using Parameters.jl as I specifically chose not to learn how to do that!

The example above is the entirety of the code I use to apply separated units tags in the constructor, and should work as-is on the struct in your example, constructing the Para2 struct with units.

Any keyword you pass in to the constructor will override the defaults generated in the get_defaults() function. If you want to change that behaviour, override default_kw(::Type{T}; kwargs...).

You should just be able to read the code in Defaults.jl and play around. Its only 46 lines and incredibly minimalist.

[rafaqz]

As an aside this is how I was using Flatten with Dual Numbers with DiffEqSensitivity. It's ugly because there are a few combinations DiffEq throws at you:

https://github.com/rafaqz/DynamicEnergyBudgets.jl/blob/master/src/sensitivity.jl

You would have to use retype() now instead of reconstruct().

I just realised I'm not even using Default.jl constructors there, and wondering why I even have units separate from default values at all when UnitlessFlatten.jl actually does all of it for you in a different way. It can still be usefull to reapply the units to the vector if you need them, but maybe not that useful.

[rafaqz]

To rewrite your example for Defaults.jl and my example above:

using Defaults, Flatten, Tags, Unitful
import Defaults: get_default
import Tags: @units, units
import Flatten: flattenable

Defaults.get_default(t::Type) = begin 
    d = default(t) 
    u = units(t)
    add_units.(d, u)
end

add_units(::Nothing, u) = nothing
add_units(x, ::Nothing) = x
add_units(::Nothing, ::Nothing) = nothing
add_units(x::Number, u::Unitful.Units) = x * u
add_units(x::AbstractArray, u::Unitful.Units) = x .* u

@tag printunits nothing
@flattenable @units @printunits @default_kw struct Para{T1,T2,T3}
    a::T1 | 1.0 | u"km"   | u"m"   | true
    b::T2 | 2.0 | u"s"    | u"s"   | true
    c::T3 | 3.0 | u"mmol" | u"mol" | false
end
p = Para()

So:

julia> get_default(Para)
(1.0 m, 2.0 s, 3.0 mol)

julia> Para()
Para{Quantity{Float64,Unitful.Dimensions{(Unitful.Dimension{:Length}(1//1),)},Unitful.FreeUnits{(Unitful.Unit{:Meter,Unitful.Dimensions{(Unitful.Dimension{:Length}(1//1),)}}(0, 1//1),),Unitful.Dimensions{(Unitful.Dimension{:Length}(1//1),)}}},Quantity{Float64,Unitful.Dimensions{(Unitful.Dimension{:Time}(1//1),)},Unitful.FreeUnits{(Unitful.Unit{:Second,Unitful.Dimensions{(Unitful.Dimension{:Time}(1//1),)}}(0, 1//1),),Unitful.Dimensions{(Unitful.Dimension{:Time}(1//1),)}}},Quantity{Float64,Unitful.Dimensions{(Unitful.Dimension{:Amount}(1//1),)},Unitful.FreeUnits{(Unitful.Unit{:Mole,Unitful.Dimensions{(Unitful.Dimension{:Amount}(1//1),)}}(0, 1//1),),Unitful.Dimensions{(Unitful.Dimension{:Amount}(1//1),)}}}}(1.0 m, 2.0 s, 3.0 mol)

[rafaqz]

Last comment!

You are bound to be a bit lost because this is all new, weird code. It would be very difficult to write without Julias multiple dispatch and macros so I have nothing to compare it to for a sanity check.

Some of it might be a really bad idea, and that should become clear over the next year or so. But it does seem to be very useful at the moment.

[briochemc]

Thanks for all these responses! I tried your example and I am still trying to figure some things out. I replaced my Parameters.jl usage with your Defaults.jl. First thing is I updated it to work with the renamed FieldMetadata.jl. I also wanted a unique type T, so I got rid of the part adding the units in the default constructor. I also overrid (proper english?) show and its compact form to show the values in the units I want. I don't know if that's useful to you but this is sort of what I have now:

using Pkg
Pkg.activate(".")

using Defaults, Flatten, FieldMetadata, Unitful
import Defaults: get_default
import FieldMetadata: @units, units
import Flatten: flattenable

@metadata printunits nothing
@flattenable @printunits @units @default_kw struct Para{T}
    a::T | 1.0 | u"m"    | u"km"   | true
    b::T | 2.0 | u"s"    | u"s"    | true
    c::T | 3.0 | u"mol"  | u"mmol" | false
end

function Base.show(io::IO, p::Para)
    println("Parameter values:")
    compact = get(io, :compact, false)
    for f in fieldnames(typeof(p))
        val = uconvert(printunits(p, f), getfield(p, f) * units(p, f))
        (~compact || flattenable(p,f)) && println("| $f = $val")
    end
end

[rafaqz]

Cool. Glad you got it working.

If you want to actually construct Para with those @units you will need Para{T1, T2, T3}.

[briochemc]

I think I prefer to leave them out and have them just as metadata for now, because Unitful does not work with LinearAlgebra, backslash, factorization, or sparse matrices for now... Should I close this issue? (I mean it diverged a bit from the original purpose 😄 )

[rafaqz]

Sure, that's a whole other set of problems that need fixing! Fortunately I rarely need those things so I can run everything with units attached.

I'll close this, but feel free to open issues on any of these packages, the feedback has been useful.