Tag parameter

[khaichiong]

Hi all,

I apologize if this is not the right place to post.

I have used ForwardDiff with great success in the past. Thanks for putting this together. However this time, I am using ForwardDiff.gradient on a function LLH that I have defined.

I ran into the following problem:

MethodError: no method matching LLH(::Array{Real,2}, ::Array{ForwardDiff.Dual{9,Real},1})
Closest candidates are:
  LLH(::Array{Real,2}, !Matched::Array{Real,1}) at...

My question is, what is the type ::Array{ForwardDiff.Dual{9,Real},1}, and how do I make my function accepts this type? It already accepts a vector of reals Array{Real,1}. In my search for answers, I think the "9" refers to a tag parameter.

Thanks!

[ChrisRackauckas]

Is there a reason not to just ducktype, or ::AbstractArray?

[khaichiong]

Wow thanks Chris, changing it to ::AbstractArray works!

I do wonder what is the reasoning behind this.

[ChrisRackauckas]

I do wonder what is the reasoning behind this.

What's the reasoning behind what?

1) The reasoning behind the tag is so that way perturbation confusion isn't a problem, letting you re-autodifferentiate autodifferentiated functions without confusing "which epsilon belongs to which derivative". 2) ForwardDiff needs to use its Dual type because operator overloading is how it's performing its forward-mode autodifferentiation 3) Every Array <: AbstractArray so changing the requirement to AbstractArray makes it always work. 4) Julia auto-specializes on function arguments, meaning that type assertions in function headers are just for dispatch and play no part in optimization. So you can leave them as loose as you want without a performance problem.

I think that covers whatever possible question there was?

[khaichiong]

Thanks for the quick reply.

I should make sure the function and its input are generic enough, i.e. using AbstractArray makes it always work.

I was just confused because the documentation says:

The target function must be written generically enough to accept numbers of type T<:Real as input (or arrays of these numbers)... The types of array inputs must be subtypes of AbstractArray.

My target function did accept arrays of Reals, but I am guessing it is not sufficient.

[ChrisRackauckas]

My target function did accept arrays of Reals, but I am guessing it is not sufficient.

f(x::Array{Real})

or

f{T<:Real}(x::Array{T})

? Note that Array{Float64} <: Array{Real} is false.

[ChrisRackauckas]

I think we're getting a little off topic here from ForwardDiff and onto Julia usage itself. If you want to discuss more, I recommend joining the chat channel:

https://gitter.im/JuliaLang/julia

or asking on the Discourse:

https://discourse.julialang.org/

[khaichiong]

The former f(x::Array{Real})

Yes you are right. I will close this thread, and bring my discussion to Gitter or Discourse.

Thanks again for your help.

[jrevels]

Just to clarify...

In my search for answers, I think the "9" refers to a tag parameter.

It appears you're using an older version of ForwardDiff, in which the dual number type is defined Dual{N,V} where N is the number of partial derivatives being tracked and V is the value type. In the most recent release of ForwardDiff (which implements the tagging system), the dual number type is Dual{T,V,N}, where N and V are the same, and T is the tag parameter.

My target function did accept arrays of Reals, but I am guessing it is not sufficient.

Yup, as Chris said, ForwardDiff won't be able to differentiate f(x::Array{Real}) because it can't pass in x::Array{<:Dual}, since Array{<:Dual} <: Array{Real} is false. While the documentation is technically correct, it's not as explicit/clear as it could be. We should change it to state that target function signatures should allow for x::AbstractArray{<:Real}.