Coordinate Conversion: losing positional accuracy between coordinate formats (ArcMap)

[elinz]

This is an extremely important bug

The addin is not maintaining the same input coordinate positional accuracy in the output values for DD, DDM, and DMS.

For example, if I set the default map unit in the addin to "DD" and using the Map Point tool, add the location of Sevastopol as the input value. See screenshot below:

The addin sets the input value to 44.608559N 033.528627E; (6 decimal places lat/long). Which using the calculator at NGA to determine the length of a degree of latitude and longitude at 45 degrees, I calculate that storing a coordinate down to 6 decimal places will allow a location to be stored down below a 1/10 of a meter (i.e. 0.11 meters of latitude and 0.08 meters of longitude). See snapshot of my spreadsheet below:

However, the default formatting for DD, DDM, and DMS do not allow us to output the coordinate down to the same accuracy.

• The output for DD displays out to 2 decimal places, only gets us down to approx. 1,111 meters of latitude and approx. 788 meters of longitude (see screenshot above).

• The output for DDM displays the decimal minute out to 2 decimal places, which based on my calculations, gets us down to approx. 18 meters of latitude and approx. 13 meters of longitude.

• The output for DMS displays seconds out to a single decimal place, which based on my calculations, gets us down to approx. 3 meters of latitude and 2 meters of longitude.

The default configuration of the output coordinate formats should not store a less accuracy position then the user has entered. If the user wants to add custom formats and change the number of decimal places that's fine, but out of the box, the addin must calculate the same positional accuracy

Another example, would be the user entering the MGRS coordinate down to the meter, such as 36TWQ4194739604 (Sevastopol location specified above) and asking for DD, DDM, or DMS as the output. We are not providing the same coordinate back to the user.

[elinz]

Was testing with ArcGIS Pro 1.2

[elinz]

@jmhauck The default formatting for DD was not altered to handle 6 decimal places. It's still the original 2 decimal places:

[jmhauck]

@elinz I will need to check why it is automatically switching to "Custom" and why that is still showing the old format there. You should see the new default format when you select the example coordinate from the Formats list...in this case "70.49N 40.32W".

Would it also be better to just say "Default" in the formats list rather than showing an example there and another example below the Formats list...that don't even show the same coordinate?

[elinz]

@jmhauck Oh I see. I didn't even notice that the dialog was set to Custom when I opened it.

I like the idea of changing it to say "Default". It is confusing to show an example coordinate in the Formats list and then show a sample.

[elinz]

@jmhauck Yes, lets make the change so the default format is listed as "Default" instead of showing a coordinate pair in the format list.

[jmhauck]

@elinz ok, sounds good.

[jmhauck]

@elinz In terms of using the word Default for each. The code is currently using those example values as a unique key for looking up and storing the formats. Anything can be changed but this would require a fair bit of effort. As a possible alternative I switched them to be "Default DD" and "Default DDM" for example so they would retain the unique nature and avoid showing an additional coordinate value. Please let me know what you think and if we need to alter any further.

[elinz]

@jmhauck Ok. Understand the issue. The alternative of using "Default DD", "Default DDM" is a good alternative.

[elinz]

• Verified that the default DD format is set to display up to 6 significant digits.
• Verified that the default DDM format is set to display up to 4 significant digits.
• Verified that the default DMS form is set to display up to 2 significant digits.
• Performed the following test:
  1. Entered the MGRS coordinate:10TET4889274330 DD: 47.620528N 122.349328W DDM: 47° 37.2317'N 122° 20.9597'W DMS: 47° 37' 13.9"N 122° 20' 57.58"W GARS: 116MM35 MGRS: 10TET4889274330 USNG: 10TET4889274330 UTM: 10T 548892 5274330

2. Took the DD output (47.620528N 122.349328W) from #1 above and entered as input. The output is shown below: DD: 47.620528N 122.349328W DDM: 47° 37.2317'N 122° 20.9597'W DMS: 47° 37' 13.9"N 122° 20' 57.58"W GARS: 116MM35 MGRS: 10TET4889274330 USNG: 10TET4889274330 UTM: 10T 548891 5274329 <- 1 meter different, but believe this is just a rounding issue.

3. Took the DDM output (47° 37.2317'N 122° 20.9597'W) from #1 above and entered as input. The output is shown below: DD: 47.620528N 122.349328W DDM: 47° 37.2317'N 122° 20.9597'W DMS: 47° 37' 13.9"N 122° 20' 57.58"W GARS: 116MM35 MGRS: 10TET4889274330 USNG: 10TET4889274330 UTM: 10T 548891 5274330 <- 1 meter different, but believe this is just a rounding issue.

4. Took the DMS output (47 37 13.9N 122 20 57.58W) from #1 above and entered as input. The output is shown below: DD: 47.620528N 122.349328W DDM: 47° 37.2317'N 122° 20.9597'W DMS: 47° 37' 13.9"N 122° 20' 57.58"W GARS: 116MM35 MGRS: 10TET4889274330 USNG: 10TET4889274330 UTM: 10T 548891 5274329 <- 1 meter different, but believe this is just a rounding issue.

@jmhauck @topowright All the coordinates were the same except for UTM; I believe the 1 meter difference is do to a rounding issue. Can you please review? If you believe my assessment is true and you think the 1 meter difference is okay then I can close this issue.

[elinz]

@jmhauck Verified this is working on ArcMap addin build 10 Jan 2017 05:53 on ArcMap 10.3.1 final. Closing this issue.