Release ArcGIS Pro Version of GCD Add-In

[joewheaton]

With ESRI's planned retirement of ArcGIS 10.X within the next year or two, we really need to look at providing an ArcPro release of GCD (if not RAVE). @philipbaileynar can you please scope the cost for both? I know they are not things we have funding for yet, but we need to have them ready to go if someone is willing to foot bill or we might even try and crowdfund it.

[philipbaileynar]

I will do some research. Last time I checked ArcPro uses a newer version of .Net that relies on a fundamentally different technology to build forms. There's an upgrade strategy, but it does mean that each GCD form (of which there are 40 or so) will need to get ported over and tested thoroughly. Weirdly this is the biggest risk/cost uncertainty and it has nothing to do with ArcPro itself. We do so little with Arc (literally add layers to the map and symbolize) that the Arc piece should be fairly straightforward.

In summary... some research is needed before I can scope.

[joewheaton]

Totally understand. Low priority until someone is willing to pay. Wanted to capture issue even if just the shelf. I will work on getting RAVE in year 2 budget for BLM.

[jb10016]

I've been getting quite a few enquires relating to this topic down under and I do wonder if there might be an opportunity to look at co-funding some new development in this space ... something maybe we could collectively scope out if there is interest (and time!)

[philipbaileynar]

Curious if you have also seen much interest in QGIS?

Are there still some ArcGIS holdouts, or has the balance tipped to ArcPro?

[jb10016]

I'd say the balance has switched to Pro already. Re Q, as it stands, NZ (via our Regional Councils) are well ensconced in ERSI ecosystems, so the demand is definitely for Pro I'm afraid. We've got people using the standalone and then visualizing in Pro, but there is a cry for seamless integration. I guess as datasets get bigger there is also the question of enabling the use of mosaic datasets too ...

[fluviotect]

I'm a proud holdout :-)

Philip's email is interesting in terms of how little processing actually gets done in an Arc environment (if I read that right).

James is right in terms of NZ council buy-in/dependency to Arc-based workflows. Crown entities are a bit of a mix with some very keen on open-source.But others very keen on ArcPro/FME workflows.

I used standalone GCD for some of my thesis processing with cartography in QGIS. For basic threshold number-crunching and pipelining, it was more straightforward for me to skip GCD and route through numpy using masked arrays.

I calculated I've lost almost a year of my life to dealing with, er, ESRI oddities over the last 23 years. There are enough stable python options and QGIS is so good now that I'm doing 99% of geoprocessing and cartography outside of an Arc environment over the last 15-18 months. Generally trying to do as much as possible within python.

While python has been historically performance limited for big rasters, there are at least two array-based acceleration packages worth having a look at:

• https://github.com/rapidsai/cudf
• https://github.com/numba/numba

Whatever you come up with, I'll be most interested if it has a Python API that does not have an arcpy dependency.

All of this said....I recognise that I am not the stereotypical GCD customer.

my $0.02

Best, Will

On Thu, Feb 17, 2022 at 10:07 AM James Brasington @.***> wrote:

I'd say the balance has switched to Pro already. Re Q, as it stands, NZ (via our Regional Councils) are well ensconced in ERSI ecosystems, so the demand is definitely for Pro I'm afraid. We've got people using the standalone and then visualizing in Pro, but there is a cry for seamless integration. I guess as datasets get bigger there is also the question of enabling the use of mosaic datasets too ...

— Reply to this email directly, view it on GitHub https://github.com/Riverscapes/gcd/issues/391#issuecomment-1042311771, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJ7FW2WW664RM2VLK2NHWSLU3QG2VANCNFSM4WK6E2QQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you are subscribed to this thread.Message ID: @.***>

[jb10016]

I hear you Will and I'd hate to count the time I've lost to working around rather than with ESRI :-)

[philipbaileynar]

@joewheaton and I have been discussing several strategies. Presented in order of increasing cost...

1. Continue with .net

1. Produce a new GCD standalone version based on Microsoft's new user interface technology (that is required for ArcPro). See my 20 Jan 2021 comment above.
2. Wrap this new GCD version in an ArcPro version. The additional work is just the cartographic "add to map" functionality.
3. Release a QGIS plugin that is nothing more than a project tree dockable window with the "add to map" functionality. You wouldn't be able to launch any GCD user interface features from inside QGIS, but you would have the cartography.

ps. Microsoft keeps delaying the launch of Maui that would enable GCD to run across platforms.

2. Open Source Desktop

1. Rewrite core GCD in Python using NumPy
2. Write an entirely new user interface in platform agnostic technology like Qt or Kivvy.

3. Cloud

GCD becomes nothing more than a hollow user interface shell. All rasters are uploaded into the cloud and processed there. @jb10016 this might relate to mosaics.

[joewheaton]

@jb10016 I lets work with @philipbaileynar to get some cost-estimates and see if we can shop this around. We have learned a lot over the years about good development strategies and made lots of mis-steps that were short-sided. One of the biggest short-sided decisions we made was never to implement anonymous user-tracking so we had a rough, anonymous sense of how often the tool was getting used and for what commands. Doing so would make it easier to make the case for funding some development. I know GCD gets used a lot still by the inquiries we continue to get.

My two cents on @philipbaileynar suggestions:

• Producing an ArcPro version (i.e. 1.1 & 1.2) off existing core makes a ton of sense as that is where the ESRI user base has migrated and it won't take too much development or $$.
• I don't like 1.3 as suggested. If I'm in QGIS, I should be in QGIS and it will be weird to have to do analysis in stand-alone and add to map in QGIS. If that is all we're suggesting, we need to just figure out how to make GCD actually work with RAVE (both QRAVE and ArcRAVE). I know, I know, this "can't" be done for convoluted reasons at the moment, but that is a problem we can fix that is more future proof as all three flavors of RAVE have more stable funding in the long-term than GCD does.
• I don't think we can cross our fingers and wait for Maui and Microsoft to fix anything for us.
• I would forgo 1.3 and be happy to find funding for 2.1 and 2.2 concurrently such that we can make the same interface work in QGIS or Standalone. The reason I like this is for our power users (e.g. the @jb10016 and @fluviotec of the world) this makes accessible to them a command line version and python code they can use. We know Python is a total nightmare to deploy, but we side-step that by deploying from within QGIS instead of stand-alone. I feel like a Python GCD Core could and would get deployed in many more ways by many more people. We have invested big in a C++ and C# core and no one in the world but us knows how or has bothered to get their hands on them. Stability wise it has served us well, but it is the same old open-source reality - 99.9% of open source code doesn't get used. Still the right thing to do.
• I like 3 - a cloud based strategy (especially one based on a 2.1 Python core and made to be deployed from RiS and compatible with all flavors of RAVE. I know why GCD projects are hard to make work as riverscapes projects, but I don't accept that it cannot be done.

I think we need to think carefully about what next steps we might raise funds for. I am not doing much GCD research these days and have much bigger priorities that I am raising funds for. So I am not going to push on this stuff. If money lands, that's find, but I worry about what strategies are more future proof and or can be handed off to others. If researchers are going to use GCD for what its good at (housekeeping), then Python is the way to go. I don't like investing big in a stand-alone or add-in that is going to be difficult to update later. Anyway, we should keep stewing on this. The reality is, tools thrive and get used by lots of people when we invest lots into them and teach lots of folks how to use them. We have more than gotten our return on investment on GCD and it continues to work and be stable.

Enough of my stream of consciousness rambling.

[joewheaton]

I'd say the balance has switched to Pro already. Re Q, as it stands, NZ (via our Regional Councils) are well ensconced in ERSI ecosystems, so the demand is definitely for Pro I'm afraid. We've got people using the standalone and then visualizing in Pro, but there is a cry for seamless integration. I guess as datasets get bigger there is also the question of enabling the use of mosaic datasets too ...

I agree. People, Students and Organizations that were ArcGIS users and hooked have almost all already made switch to ArcPro. A tiny minority has taken that switch as an opportunity to go to QGIS (< 1%). Those that have left ESRI, all seem to have no regrets.

[joewheaton]

I'm a proud holdout :-) Philip's email is interesting in terms of how little processing actually gets done in an Arc environment (if I read that right).

There are zero ArcPy or Arc dependencies inside the GCD AddIn to ArcGIS. It all calls a C# core that works really well. The biggest development costs are on the UI side at this point anyway.

James is right in terms of NZ council buy-in/dependency to Arc-based workflows. Crown entities are a bit of a mix with some very keen on open-source.But others very keen on ArcPro/FME workflows. I used standalone GCD for some of my thesis processing with cartography in QGIS. For basic threshold number-crunching and pipelining, it was more straightforward for me to skip GCD and route through numpy using masked arrays.

Thank you @fluviotect! This is the thing that drives me NUTS. Our most important and best examples of power-users (@fluviotect, GCMRC, etc) are all NOT using GCD because there is not a command line or scripting interface. I know, there actually is, but we haven't ever shown anyone how to use it and IT doesn't do the most important part of GCD (housekeeping). It doesn't write GCD projects. Will does bring up another interesting point though @philipbaileynar. We could write an API for Python access to our compiled C# libraries. The GCD Core is super stable. If we exposed all the commands in a Python API, could we avoid having to rewrite it? Just curious?

I calculated I've lost almost a year of my life to dealing with, er, ESRI oddities over the last 23 years. There are enough stable python options and QGIS is so good now that I'm doing 99% of geoprocessing and cartography outside of an Arc environment over the last 15-18 months. Generally trying to do as much as possible within python. While python has been historically performance limited for big rasters, there are at least two array-based acceleration packages worth having a look at: - https://github.com/rapidsai/cudf - https://github.com/numba/numba

Whatever you come up with, I'll be most interested if it has a Python API that does not have an arcpy dependency. All of this said....I recognise that I am not the stereotypical GCD customer. my $0.02 Best, Will