Riverscapes / gcd

Geomorphic Change Detection For Windows
http://gcd.riverscapes.xyz
GNU General Public License v3.0
25 stars 5 forks source link

Release ArcGIS Pro Version of GCD Add-In #391

Open joewheaton opened 3 years ago

joewheaton commented 3 years ago

With ESRI's planned retirement of ArcGIS 10.X within the next year or two, we really need to look at providing an ArcPro release of GCD (if not RAVE). @philipbaileynar can you please scope the cost for both? I know they are not things we have funding for yet, but we need to have them ready to go if someone is willing to foot bill or we might even try and crowdfund it.

philipbaileynar commented 3 years ago

I will do some research. Last time I checked ArcPro uses a newer version of .Net that relies on a fundamentally different technology to build forms. There's an upgrade strategy, but it does mean that each GCD form (of which there are 40 or so) will need to get ported over and tested thoroughly. Weirdly this is the biggest risk/cost uncertainty and it has nothing to do with ArcPro itself. We do so little with Arc (literally add layers to the map and symbolize) that the Arc piece should be fairly straightforward.

In summary... some research is needed before I can scope.

joewheaton commented 3 years ago

Totally understand. Low priority until someone is willing to pay. Wanted to capture issue even if just the shelf. I will work on getting RAVE in year 2 budget for BLM.

jb10016 commented 2 years ago

I've been getting quite a few enquires relating to this topic down under and I do wonder if there might be an opportunity to look at co-funding some new development in this space ... something maybe we could collectively scope out if there is interest (and time!)

philipbaileynar commented 2 years ago

Curious if you have also seen much interest in QGIS?

Are there still some ArcGIS holdouts, or has the balance tipped to ArcPro?

jb10016 commented 2 years ago

I'd say the balance has switched to Pro already. Re Q, as it stands, NZ (via our Regional Councils) are well ensconced in ERSI ecosystems, so the demand is definitely for Pro I'm afraid. We've got people using the standalone and then visualizing in Pro, but there is a cry for seamless integration. I guess as datasets get bigger there is also the question of enabling the use of mosaic datasets too ...

fluviotect commented 2 years ago

I'm a proud holdout :-)

Philip's email is interesting in terms of how little processing actually gets done in an Arc environment (if I read that right).

James is right in terms of NZ council buy-in/dependency to Arc-based workflows. Crown entities are a bit of a mix with some very keen on open-source.But others very keen on ArcPro/FME workflows.

I used standalone GCD for some of my thesis processing with cartography in QGIS. For basic threshold number-crunching and pipelining, it was more straightforward for me to skip GCD and route through numpy using masked arrays.

I calculated I've lost almost a year of my life to dealing with, er, ESRI oddities over the last 23 years. There are enough stable python options and QGIS is so good now that I'm doing 99% of geoprocessing and cartography outside of an Arc environment over the last 15-18 months. Generally trying to do as much as possible within python.

While python has been historically performance limited for big rasters, there are at least two array-based acceleration packages worth having a look at:

Whatever you come up with, I'll be most interested if it has a Python API that does not have an arcpy dependency.

All of this said....I recognise that I am not the stereotypical GCD customer.

my $0.02

Best, Will

On Thu, Feb 17, 2022 at 10:07 AM James Brasington @.***> wrote:

I'd say the balance has switched to Pro already. Re Q, as it stands, NZ (via our Regional Councils) are well ensconced in ERSI ecosystems, so the demand is definitely for Pro I'm afraid. We've got people using the standalone and then visualizing in Pro, but there is a cry for seamless integration. I guess as datasets get bigger there is also the question of enabling the use of mosaic datasets too ...

— Reply to this email directly, view it on GitHub https://github.com/Riverscapes/gcd/issues/391#issuecomment-1042311771, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJ7FW2WW664RM2VLK2NHWSLU3QG2VANCNFSM4WK6E2QQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you are subscribed to this thread.Message ID: @.***>

jb10016 commented 2 years ago

I hear you Will and I'd hate to count the time I've lost to working around rather than with ESRI :-)

philipbaileynar commented 2 years ago

@joewheaton and I have been discussing several strategies. Presented in order of increasing cost...

1. Continue with .net

  1. Produce a new GCD standalone version based on Microsoft's new user interface technology (that is required for ArcPro). See my 20 Jan 2021 comment above.
  2. Wrap this new GCD version in an ArcPro version. The additional work is just the cartographic "add to map" functionality.
  3. Release a QGIS plugin that is nothing more than a project tree dockable window with the "add to map" functionality. You wouldn't be able to launch any GCD user interface features from inside QGIS, but you would have the cartography.

ps. Microsoft keeps delaying the launch of Maui that would enable GCD to run across platforms.

2. Open Source Desktop

  1. Rewrite core GCD in Python using NumPy
  2. Write an entirely new user interface in platform agnostic technology like Qt or Kivvy.

3. Cloud

GCD becomes nothing more than a hollow user interface shell. All rasters are uploaded into the cloud and processed there. @jb10016 this might relate to mosaics.

joewheaton commented 2 years ago

@jb10016 I lets work with @philipbaileynar to get some cost-estimates and see if we can shop this around. We have learned a lot over the years about good development strategies and made lots of mis-steps that were short-sided. One of the biggest short-sided decisions we made was never to implement anonymous user-tracking so we had a rough, anonymous sense of how often the tool was getting used and for what commands. Doing so would make it easier to make the case for funding some development. I know GCD gets used a lot still by the inquiries we continue to get.

My two cents on @philipbaileynar suggestions:

I think we need to think carefully about what next steps we might raise funds for. I am not doing much GCD research these days and have much bigger priorities that I am raising funds for. So I am not going to push on this stuff. If money lands, that's find, but I worry about what strategies are more future proof and or can be handed off to others. If researchers are going to use GCD for what its good at (housekeeping), then Python is the way to go. I don't like investing big in a stand-alone or add-in that is going to be difficult to update later. Anyway, we should keep stewing on this. The reality is, tools thrive and get used by lots of people when we invest lots into them and teach lots of folks how to use them. We have more than gotten our return on investment on GCD and it continues to work and be stable.

Enough of my stream of consciousness rambling.

joewheaton commented 2 years ago

I'd say the balance has switched to Pro already. Re Q, as it stands, NZ (via our Regional Councils) are well ensconced in ERSI ecosystems, so the demand is definitely for Pro I'm afraid. We've got people using the standalone and then visualizing in Pro, but there is a cry for seamless integration. I guess as datasets get bigger there is also the question of enabling the use of mosaic datasets too ...

I agree. People, Students and Organizations that were ArcGIS users and hooked have almost all already made switch to ArcPro. A tiny minority has taken that switch as an opportunity to go to QGIS (< 1%). Those that have left ESRI, all seem to have no regrets.

joewheaton commented 2 years ago

I'm a proud holdout :-) Philip's email is interesting in terms of how little processing actually gets done in an Arc environment (if I read that right).

There are zero ArcPy or Arc dependencies inside the GCD AddIn to ArcGIS. It all calls a C# core that works really well. The biggest development costs are on the UI side at this point anyway.

James is right in terms of NZ council buy-in/dependency to Arc-based workflows. Crown entities are a bit of a mix with some very keen on open-source.But others very keen on ArcPro/FME workflows. I used standalone GCD for some of my thesis processing with cartography in QGIS. For basic threshold number-crunching and pipelining, it was more straightforward for me to skip GCD and route through numpy using masked arrays.

Thank you @fluviotect! This is the thing that drives me NUTS. Our most important and best examples of power-users (@fluviotect, GCMRC, etc) are all NOT using GCD because there is not a command line or scripting interface. I know, there actually is, but we haven't ever shown anyone how to use it and IT doesn't do the most important part of GCD (housekeeping). It doesn't write GCD projects. Will does bring up another interesting point though @philipbaileynar. We could write an API for Python access to our compiled C# libraries. The GCD Core is super stable. If we exposed all the commands in a Python API, could we avoid having to rewrite it? Just curious?

I calculated I've lost almost a year of my life to dealing with, er, ESRI oddities over the last 23 years. There are enough stable python options and QGIS is so good now that I'm doing 99% of geoprocessing and cartography outside of an Arc environment over the last 15-18 months. Generally trying to do as much as possible within python. While python has been historically performance limited for big rasters, there are at least two array-based acceleration packages worth having a look at: - https://github.com/rapidsai/cudf - https://github.com/numba/numba

Whatever you come up with, I'll be most interested if it has a Python API that does not have an arcpy dependency. All of this said....I recognise that I am not the stereotypical GCD customer. my $0.02 Best, Will