Map UTM coordinates

[Jegelewicz]

See https://www.geoplaner.com/

[Nicole-Ridgwell-NMMNHS]

Ok, I think I see where you are coming from now.

[ebraker]

Chiming in that UCM would LOVE a UTM converter. When we migrated to Arctos we had 29K records with UTMs generated by the MapSTEDI project. Currently those are formatted with a standardized Locality_Remark, e.g., "LatLongRemarks: UTM Zone: 13, northing 4442980, easting 483435" so that one day we can pull them out and push them through a calculator.

[ccicero]

Ditto! We are currently dealing with ~1500 Spotted Owl blood samples, getting data ready to upload. Coords are all in UTMs.

[dustymc]

I think this can probably be closed.

• locality and/or collecting event attributes can be used for whatever you consider verbatim
• https://github.com/ArctosDB/arctos/issues/2660 will let us treat UTM in the same way we treat DMS etc.
• I've claimed collecting_event.verbatim_locality for "as-entered"

I'm not sure what's going on with postgis on production, I thought it just needed a scheduled reboot, maybe ping Chris @mkoo .

UTM Zone: 13, northing 4442980, easting 483435

That is unfortunately, as far as I can determine, insufficient information - see https://github.com/ArctosDB/arctos/issues/3415 and https://arctos.database.museum/info/ctDocumentation.cfm?table=ctutm_zone.

[Jegelewicz]

Dropping this here, because this is obviously something a lot of people are interested in.

Hi Mariel,

I'm afraid if anyone wants to know about the original coordinates for these records, they will have to look in the collection. I strongly encourage the development team to restore the functionality of the database to allow us to continue to use original coordinate formats (and get back to being able to use UTMs again). Luckily converting within degree formats can easily be done in a spreadsheet, not like UTMs that need an online converter. I can't imagine what was so beneficial from this change that justifies losing coordinate entry functionality.

Thanks for your help in getting this sorted out for me.

Andy

A lot more in the entire email thread Fwd_ bulk appending records.pdf

[dustymc]

Data would be most useful.

omits higher geography,

In next release.

Arctos only accepting decimal degrees format.

That is not and has never been true, and the recent issues with DD MM.mm format do not affect this tool.

verbatim coordinates are in a different format, we have to put the verbatim into the verbatim locality

Yes, that's always been the recommendation - https://handbook.arctosdb.org/documentation/collecting-event.html#verbatim-locality.

made a point of not having coordinates in verbatim.

I've been recommending some locality or event attribute for that situation - just needs an Issue.

because that bulkload collecting event template had only decimal degrees

An Issue can change that (or lead to documentation or whatever).

encourage the development team to restore the functionality of the database to allow us to continue to use original coordinate formats (and get back to being able to use UTMs again)

This is and always has been an administrative, not development, issue. I need tools, I don't have the ability to get them for myself or I'd have done so long ago. (Monday, I hope....)

[Jegelewicz]

This is and always has been an administrative, not development, issue. I need tools, I don't have the ability to get them for myself or I'd have done so long ago. (Monday, I hope....)

Yeah - but lots of users don't participate here and so don't know that...just wanted to keep all of the stuff somewhere I could find it if needed.

[Jegelewicz]

made a point of not having coordinates in verbatim.

I've been recommending some locality or event attribute for that situation - just needs an Issue.

2930 was that issue but what we decided there is not working because "as entered" does not accept UTM or degrees decimal minutes or PLSS. We can revive that issue if it is going to be needed. If the long awaited tools will handle these, then perhaps we need to wait for that?

[campmlc]

Here is data from the data file that lead to Andy's comment. - guid not included. <html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:x="urn:schemas-microsoft-com:office:excel" xmlns="http://www.w3.org/TR/REC-html40">

verbatim_date | verbatim_locality | coll_event_remarks | began_date | ended_date | locality_name | higher_geog | spec_locality | dec_lat | dec_long | minimum_elevation | maximum_elevation | orig_elev_units | min_depth | max_depth | depth_units | max_error_distance | max_error_units | datum | locality_remarks | georeference_source | georeference_protocol -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- 25 May 2018 | New Mexico: Eddy County; Guadalupe Mountains | 2018-05-25 | 2018-05-25 |   | North America, United States, New Mexico, Eddy County | Guadalupe Mountains | 32.03735 | 104.7825 | 2128 | 2128 | m |   |   |   | 15 | m | World Geodetic System 1984 | GPS (Transcribed) | not recorded

[Jegelewicz]

I think those headers don't line up with the data?

[Jegelewicz]

Also don't see any UTM or other verbatim coordinates?

[campmlc]

Try again - here are two records that the specimen event loader kicked out: <html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:x="urn:schemas-microsoft-com:office:excel" xmlns="http://www.w3.org/TR/REC-html40">

spec_locality | collecting_method | collecting_source | verbatim_date | verbatim_locality | began_date | ended_date | habitat | coll_event_remarks | locality_name | datum | max_error_distance | max_error_units | georeference_source | georeference_protocol | orig_lat_long_units | dec_lat | dec_long | lat_deg | long_deg | dec_lat_min | dec_long_min | lat_min | lat_sec | lat_dir | long_min | long_sec | long_dir | orig_elev_units | minimum_elevation | maximum_elevation -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- West side of Rio Grande 300 m S Montano Rd on Bosque trail system | wild | 2 October 2018 | New Mexico: Bernalillo County; West side of Rio Grande 300 m S Montano Rd on Bosque trail system | 2018-10-02 | 2018-10-02 | on trail |   |   | World Geodetic System 1984 | 100 | m | GPS (Transcribed) | not recorded | deg. min. sec. |   |   | 35 | 106 |   |   | 8 | 35.3 | N | 40 | 51.2 | W |   |   |   Guadalupe Mountains | wild | 25 May 2018 | New Mexico: Eddy County; Guadalupe Mountains | 2018-05-25 | 2018-05-25 | in narrow mixed conifer, maple, oak canyon | World Geodetic System 1984 | 15 | m | GPS (Transcribed) | not recorded | degrees dec. minutes |   |   | 32 | 104 | 2.241 | 46.95 |   |   | N |   |   | W | m | 2128 | 2128

[campmlc]

I'll forward the original file to @dusty via email. It's in the email string I forwarded earlier.

[dustymc]

users don't participate here and so don't know

Yes, that's come up a lot lately, I obviously have no idea how to resolve it but users being frustrated because I'm not giving them something I don't know they want isn't ideal....

Maybe we should again try confining the chatter to elsewhere and saving this for simple functional details-worked-out-elsewhere messaging, or set up some other repo for that, or ????????????

not working because "as entered" does not accept UTM

I think there's still some significant confusion. "As-entered" is "converted by Arctos" and that's it. If I can't convert it, then it doesn't belong there - it's nothing like "verbatim" and I regret the name. It should probably never have been exposed to users (it's only useful if a script breaks, sorta), the super awkward halfassed UTM-but-not-really workaround that's now polluted the data should never have been allowed, nondigitized datums can't use this concept as soon as I get postgis, etc., etc., etc. Ain't "verbatim" and never has been, despite the bad name. If you need verbatim, then you need a new concept.

"As-entered" will start accepting UTM maybe Monday, but it still wont' be "verbatim." (Something like 95% of the UTM data in Arctos doesn't have sufficient information to be converted, I suspect that's how it comes in; that new concept would be useful for the 'this is what the gave us before we guessed at the zone and datum' data.)

Thanks @campmlc - it's not in the email I received (and that's not my github handle).

[dustymc]

OK, got data - my first guess from there is always "Excel mangled something." Do you have the CSV with the errors?

[campmlc]

I'll ask - and sorry about the handle. That's what happens when I respond to github via gmail - yes, difficult to juggle so many communications platforms. But Github keeps much better track of our issues than gmail ever could.

[Jegelewicz]

I'm not giving them something I don't know they want

Don't take it personally - it is really THE COMMUNITY not giving them something they want. As long as stuff goes on in email threads, only the few people on the thread know what the problem is. I can complain about Excel until my face turns blue, but if I never post to Microsoft customer support - how will they ever know that there is a problem?

[Jegelewicz]

Maybe we should again try confining the chatter to elsewhere

It doesn't matter WHERE we do this IMO - some people will just not participate.

[campmlc]

But we can assist the community at large by posting concerns raised by email here. Not everyone is comfortable with github as a platform - honestly, it was designed for code, and the user interface takes some getting used to. In reference to the issues raised, here we need to post issues regarding the need for 1) true, verbatim coordinates - not "as entered", with all its confusing implications - this term should not be publicly visible, 2) the specimen event bulkloader should not have fields for coordinate formats that are going to throw an error when the file is loaded. What am I missing?

On Fri, Dec 3, 2021 at 10:44 AM Teresa Mayfield-Meyer < @.***> wrote:

• [EXTERNAL]*

I'm not giving them something I don't know they want

Don't take it personally - it is really THE COMMUNITY not giving them something they want. As long as stuff goes on in email threads, only the few people on the thread know what the problem is. I can complain about Excel until my face turns blue, but if I never post to Microsoft customer support - how will they ever know that there is a problem?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ArctosDB/arctos/issues/2309#issuecomment-985709399, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADQ7JBCDXVSMYWYT7HZ2RHLUPD6WZANCNFSM4JBA6OSQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

[Jegelewicz]

the specimen event bulkloader should not have fields for coordinate formats that are going to throw an error when the file is loaded. What am I missing?

Agree - the issue is the degrees decimal minutes - if we can't use that, then we need to take it out of the template. I've had to explain this to at least three people in the past week. People use this format and have for a long time and suddenly it doesn't work (although suddenly excluding it from the template will probably generate the same issues.)

[Jegelewicz]

But we can assist the community at large by posting concerns raised by email here.

Agree - this is why I post people's email all over Github along with whatever responses I send.

[dustymc]

Github keeps much better track of our issues than gmail

Absolutely, and anything else we've tried. I love github, but I can see how it could quickly become overwhelming too. I don't think "where" matters (although it probably does to someone), but "how." (And no, I don't know how to do better, but I still think I see a problem, maybe related to the sheer volume.)

this term should not be publicly visible

I'm hoping to file that issue next week or so, I'm glad to see some pre-agreement!

the specimen event bulkloader should not have fields for coordinate formats that are going to throw an error when the file is loaded.

That's what I'm confused about: that's really the case, it it/was temporarily the case, it's completely unrelated and not my problem, it's completely unrelated and totally my problem - I'm guessing without data, I don't have the bandwidth to really investigate right now, and there are 608 issues for this to get lost in.

degrees decimal minutes

Only in a different tool?? https://github.com/ArctosDB/arctos/issues/3916

[Jegelewicz]

@dustymc as far as I know you CANNOT load degrees decimal minutes anywhere (although you used to be able to), but it is still an option both in the code table and the templates. Bound to cause problems.

https://arctos.database.museum/info/ctDocumentation.cfm?table=ctlat_long_units#degrees_dec__minutes

The fact that the ONLY coordinates you can load when bulkloading a locality or event are decimal degrees does not allow for "as entered" in these situations - why are they any different than an entire catalog record?

[dustymc]

That may or may not be true now, I'm too overwhelmed to look at the moment, I will get what's in test (which handles DM.m) out early next week whether I have postgis (for UTM) or not.

bulkloading a locality

There's absolutely no place for anything other than DD.d coordinates in locality (and I plan to enhance that to "in WGS84" as soon as I get postgis).

or event

No technical reason that can't accept anything I can convert - Issue (if there's not one - or 10....).