Range adjustment via pre-processing.

[yossigil]

Input:

1. A function f(x|t), which for any value of the parameter t, the function is reduced to a simple PDF of x; f thus is a family of functions, e.g., log normal.
2. Bounds on t, e.g., t>0, t 0 <t < 2; whatever.

Output: A function g(x|t), which is the same as f, except that t is in the range -1,1, and such that t is “typically” around 0, for some definition of typically.

TRICK g(x|t) = f(x|h(t))!

Now find the correct h!

As to range adjustment: I believe that the right way to go about this, is to place restrictions on the input distribution. In other words, our universal estimator engine, is allowed to assume that the range of the parameter value is in the range [-1,1], or perhaps, slightly better in the range [0,1]. Further assume that there is a minimal "prior", in which the client somehow adjust the range so that the interesting is very approximately near 0.

The range reduction can be done automatically, here is some pseudo code in Java

Function<Double, Double> adjust(double min, double max, Function<Double, Double> f) {
   final double MAX == Double.POSITIVE_INFINITY; 
   final double MIN = Double.NEGATIVE_INFINITY;
   if (max <= min) return null; 
   if (min == MIN && max == MAX) return x-> f(full());  
   if (max == MAX) return x -> f.apply(from(min)); 
   if (min == MIN) return x -> f.apply(to(max)); 
}

Now for the different case.

• We rely on the tangent function, which in the range [-Pi/2,Pi/2] gives values in the range -infinity to +infinity. We start with the definition of function to map the range 0,1 to the entire real access,

  /* As x ranges in [0,1], the return values ranges between -infinity and +infinity; undefined when the X is outside this range. 
  math.PI2 is (I believe) half PI. */ 
  double blow(double x) { 
  return Math.tan(x/math.PI- math.PI2); 
  }

  Now we use this auxiliary function to map a parameter to the full infinite range.

  Function<Double, Double> full() {
  return x->blow(x);
  }

  Dealing with the case that the parameter has a lower bound, but not an upper bound, is a little more complex: we need to map the value MIN to -PI/2, and the value of infinity to PI/2. To do so, we map first the range [MIN,Infinity] to the range [0,1] and then apply the above function to produce values in the range [Infinity,Infinity] and then use the above entire range reducer.

  Function<Double, Double> from(double min) {
  return x -> { 
   double  LOW = arctan(min); 
   double HIGH = math.PI2
   double adjusted = LOW + x * (HIGH-LOW);
   // If x ranges between 0 and 1, then adjusted ranges between LOW and HIGH. 
      return adjusted; 
  };  
  }; 
  }

Finally, we have:

Function<Double, Double> from(double max) {
   return x -> { 
     double  LOW = -math.PI2; 
     double HIGH = arctan(max);
     double adjusted = LOW + x * (HIGH-LOW);
     // If x ranges between 0 and 1, then adjusted ranges between LOW and HIGH. 
        return adjusted; 
    };  
}; 
}

You can even make it simpler with the following:

/** A function to return another function, i.e.., adjuster of parameter from fixed range of 0 and 1, to original range that a function takes. */
Function<Double, Double> adjuster(double min, double max) { 
  final double LOW = (min == -INFINITY)? math.PI2 : arctan(min);
 final double HIGHT = (max == INFINITY)? math.PI2 : arctan(max);
  return x -> LOW)+ x * (HIGH-LOW);  
}; 

In fact, the above function could be used; you can always do it by hand, e.g., for log-normal, you can fix this and see what happens.

[yossi-cohen]

Ok, I've wrote the adjuster here (the short version).

def adjuster(min_param, max_param):
    def adjust(x):
        LOW = -math.pi/2 if min_param == -math.inf else math.atan(min_param)
        HIGH = math.pi/2 if max_param == math.inf else math.atan(max_param)
        if x < 0: x = 0 # undefined if x < 0
        if x > 1: x = 1 # undefined if x > 1
        return math.tan(LOW + x * (HIGH - LOW))
    return adjust

I use it like so (e.g: for log-normal):

adjust = adjuster(min_param=0, max_param=math.inf)
sample_lognormal(adjust(0.2), size=256)

Example mappings:

adjust(-1.0): 0.0000 adjust(0.0): 0.0000 adjust(0.2): 0.3249 adjust(0.5): 1.0000 adjust(0.92): 7.9158 adjust(1.0): 1.633123935319537e+16 adjust(2.0): 1.633123935319537e+16

I suppose this mapping should be useful for the universal_estimator...

[yossigil]

The adjustment function should satisfy the following crucial property:

When the input parameter ranges from its minimum value and its maximal value, the output parameter ranges monotonically from 0 to 1.

Not 100% sure, but I think that my original implementation did exactly this.

At any rate, your adjustment function is incorrect since it returns values outside the range.

[yossigil]

Method

We assume the bounds on the parameters 𝑑𝑖 are known (e.g., 0<𝑑1<2 , 𝑑2>=0 ).

1. Generate synthetic samples using 𝑓 ( 𝑥 | 𝑑1,...,𝑑𝑛 ) where 𝑑𝑖 are drawn from 𝑟𝑎𝑛𝑔𝑒(𝑑𝑖) .
2. Learn a DNN model from the synthetic data.
3. Predict the parameters 𝑑𝑖 on the input sample.

Parameter range adjustment

The estimator is allowed to assume that the range of the parameter values is (0,1). For that we use range adjustment:

Input:

1. A funation 𝑓 (𝑥|𝑑𝑖) as defined above
2. Bounds on the parameters 𝑑𝑖

Output: A function 𝑔 (𝑥|𝑡𝑖) , which is the same as 𝑓 , except that 𝑡𝑖 are in the range (0,1), and such that 𝑡 is “typically” around 0, for some definition of typically.

Let: ℎ(𝑑) be a mapping function from 𝑟𝑎𝑛𝑔𝑒(𝑑) to the range (0,1) ( ℎ is continuous within 𝑟𝑎𝑛𝑔𝑒(𝑑) ).

Thus, ℎ−1(𝑡) is a mapping function from the range (0,1) to 𝑟𝑎𝑛𝑔𝑒(𝑑) .

Let: 𝑔 ( 𝑥 | 𝑡𝑖 ) = 𝑓 ( 𝑥 | ℎ(𝑡𝑖) ) where ℎ(𝑡𝑖) is a inverse of the mapping of the bounds on the parameters 𝑑𝑖 .

Range adjustment using 𝑙𝑜𝑔𝑖𝑡(𝑥)

The logit is the inverse of the standard logistic function: 𝜎(𝑥) = 1 / (1 + 𝑒^−𝑥) for 𝑥 ∈ (−∞,∞)

The logit function is defined as: 𝑙𝑜𝑔𝑖𝑡(𝑥) = 𝜎^−1(𝑥) = 𝑙𝑛 ( 𝑥 / (1−𝑥) ) for 𝑥 ∈ (0,1)

We can use the logistic function and its logit inverse as follows:

Let: ℎ(𝑑) = 𝑙𝑜𝑔𝑖𝑡(𝑑) for 𝑑 ∈ (0,1)

Than inv( ℎ(𝑡) ) = 𝜎(𝑡) for 𝑡 ∈ (−∞,∞)

Define a function to return another function, i.e., an adjuster of a parameter from the range (0, 1) to the original range that a function takes.

def adjuster_logit(low, high):
    # adjuster is defined in (-INF, INF)
    # low: parameter lower bound
    # high: parameter higher bound
    # return: a function that maps it's parameter (x) to the original range (low, high).

    LOW = 0 if math.inf == low else logistic(low)
    HIGH = 1 if math.inf == high else logistic(high)

    def adjust(x):
        # adjust is defined in (0, 1)
        return logit(LOW + x * (HIGH - LOW))

    return adjust

e.g:

print(adjuster_logit(low=-2, high=10)(0))
print(adjuster_logit(low=-2, high=10)(1))

output:

-2.0 10.0

Range adjustment using 𝑎𝑟𝑐𝑡𝑎𝑛(𝑥)

def adjuster_arctan(low, high):
    # adjuster is defined in (-INF, INF)
    # low: parameter lower bound
    # high: parameter higher bound
    # return: a function defined in (0,1) that maps it's parameter (x) to the original range (low,high).

    LOW = -pi/2 if -INF == low else arctan(low)
    HIGH = pi/2 if INF == high else arctan(high)

    def adjust(x):
        # adjust is defined in (0, 1)
        if x < 0: return -INF
        if x > 1: return INF
        return tan(LOW + x * (HIGH - LOW))

    return adjust

e.g:

print(adjuster_arctan(low=-2, high=10)(0))
print(adjuster_arctan(low=-2, high=10)(1))

output:

-2.0 10.0