Sphere output precision limited to 15 significant digits

[toniorte]

Dear All,

I was wondering if there is any reason to keep the maximum precision of the printed outputs to DBL_DIG macro (15) instead of DBL_DECIMAL_DIG (17), being that just 15 digits don't allow round-trip conversion.

In my case it causes me trouble because I cannot restore dumps of databases done with pg_dump, since the dumps contain spolys which lost precision during the dump, and pg_sphere does not pass the check of certain polygons which were inserted with 17 digits (from caom2db software) but now have 15.

Regards.

[esabol]

[...] 15 digits don't allow round-trip conversion.

Oh, that's a good point.

[toniorte]

Dear All,

I have tried to create a binary backup using the pg_dumpbinary software, hoping that it would keep the original data with the needed precision, but the following error appears:

ERROR: no binary output function available for type public.spoly

Thus, going back to the quick solution for the text output, I guess that in fact the method _set_sphere_outputprecision(n) (which is limited to n<=DBL_DIG, i.e. 15) tries to imitate the PostgreSQL behavior of "SET extra_float_digits". Of course and by default, the PostgreSQL "extra_float_digits" its set to return 17 digits (since PG12 as @df7cb pointed out). And in fact, PostgreSQL documentation says that the maximum value is "useful for dumping float data that needs to be restored exactly."

Regards.

[df7cb]

The value changed from 15 to 17 in PG12.

[esabol]

@df7cb @vitcpp : So should it be changed to 17 on PG 12+?

[vitcpp]

Dear All,

Obviously, there should be a way to set 17 digits for output. But I'm not sure about DBL_DECIMAL_DIG. It seems to appear in C11. May be to utilize extra_float_digits? Let me, please, dig into this topic. I will come with some proposal later.

[df7cb]

Some other datatypes (like my postgresql-unit) use extra_float_digits for their custom float rendering on output. pgsphere should do the same.

Perhaps it is just a matter of using float8out_internal for printing the numbers, so something along that line.

[vitcpp]

I agree, we should unify this behaviour with postgresql. What I'm concerned is the backward compatibility.

I propose:

• Keep function set_sphere_output_precision for some time. Probably, issue some warning that this function is obsolete.
• By default, use postgresql settings (extra_float_digits).
• If set_sphere_output_precision is called, use the specified precision and the old behavuour.

If user decides to use postgresql extra_float_digits, set_sphere_output_precision shouldn't be called or called with -1, for example. It shouldn't affect old scripts with set_sphere_output_precision.

What do you think?

P.S. set_sphere_output_precision seems to be absent in the doc. Some clarification should be written in the doc.

[esabol]

That all makes sense to me, @vitcpp. Thanks!

[vitcpp]

The PR was merged. I will close the issue if no objections.

[vitcpp]

Closed as completed.