We need to reinstate the NDVI point query tool

[timlinux]

This was removed in some refactoring last year. There is still the time series results alg in the code, but the UI interactions were removed. Can we please reinstate this functionality. @MLNoon if you could share some screenshots of the original tool, that would be great.

[MLNoon]

With Qv2, in Trends.Earth toolbar, we select the plot data icon. Then select AVHRR or MODIS, depending on time frame desired and resolution of dataset (example of AVHRR:) User selects an area of interest (I usually selected a point on the map when degradation results are preset and add that in the area). The submitted analysis returns a chart over the full time series of the dataset: OR User can right-click on graph to export chart info:

[alexbruy]

@MLNoon, @azvoleff how this tool should be exposed in the UI: in the same was as before by adding a button to the toolbar or in some other way?

[azvoleff]

I'd suggest a toolbar button in the main dock:

Then the user could:

• click the button
• select a region/or a point
• click ok
• get back the plot once the results are ready from our API (with a QgsTask querying our API for results).

[timlinux]

@alexbruy could you take a look at this over the next week or two, or transfer this to @gkahiu if you are not able to spend time on this?

[gkahiu]

Hi @azvoleff, am trying to run generate some plots but am not getting any results with the job getting removed when refreshing remote state. See below the payload being sent to the server and corresponding job created from the response:

{
   "year_initial":2000,
   "year_final":2015,
   "crosses_180th":false,
   "geojsons":"[{\"coordinates\": [[[33.89046838400023, -4.677504164999959], [41.8850191650001, -4.677504164999959], [41.8850191650001, 5.030375822607752], [33.89046838400023, 5.030375822607752], [33.89046838400023, -4.677504164999959]]], \"type\": \"Polygon\"}]",
   "crs":"GEOGCS[\"WGS 84\",DATUM[\"WGS_1984\",SPHEROID[\"WGS 84\",6378137,298.257223563,AUTHORITY[\"EPSG\",\"7030\"]],AUTHORITY[\"EPSG\",\"6326\"]],PRIMEM[\"Greenwich\",0,AUTHORITY[\"EPSG\",\"8901\"]],UNIT[\"degree\",0.0174532925199433,AUTHORITY[\"EPSG\",\"9122\"]],AUTHORITY[\"EPSG\",\"4326\"]]",
   "ndvi_gee_dataset":"users/geflanddegradation/toolbox_datasets/ndvi_avhrr_1982_2015",
   "task_name":"Test Info",
   "task_notes":"",
   "climate_gee_dataset":"None",
   "trajectory_method":"ndvi_trend"
}

Job(id = UUID('39e798bf-68cc-412c-a7f5-40ee0ab8fc27'), params = {
    'climate_gee_dataset': None,
    'crosses_180th': False,
    'crs': 'GEOGCS["WGS 84",DATUM["WGS_1984",SPHEROID["WGS 84",6378137,298.257223563,AUTHORITY["EPSG","7030"]],AUTHORITY["EPSG","6326"]],PRIMEM["Greenwich",0,AUTHORITY["EPSG","8901"]],UNIT["degree",0.0174532925199433,AUTHORITY["EPSG","9122"]],AUTHORITY["EPSG","4326"]]',
    'geojsons': '[{"coordinates": [[[33.89046838400023, -4.677504164999959], [41.8850191650001, -4.677504164999959], [41.8850191650001, 5.030375822607752], [33.89046838400023, 5.030375822607752], [33.89046838400023, -4.677504164999959]]], "type": "Polygon"}]',
    'ndvi_gee_dataset': 'users/geflanddegradation/toolbox_datasets/ndvi_avhrr_1982_2015',
    'trajectory_method': 'ndvi_trend',
    'year_final': 2015,
    'year_initial': 2000
}, progress = 0, 
start_date = datetime.datetime(2022, 6, 10, 17, 45, 15, 165078, 
tzinfo = datetime.timezone.utc), 
status = < JobStatus.PENDING: 'PENDING' > , 
local_context = JobLocalContext(base_dir = WindowsPath('D:/Frank/Kartoza/trends.earth/Data/trends_earth_data'), area_of_interest_name = 'kenya-all-regions'), 
results = EmptyResults(name = None, data_path = None, type = < ResultType.EMPTY_RESULTS: 'EmptyResults' > ), 
task_name = 'Test Info', 
task_notes = None, 
script = ExecutionScript(id = UUID('c0039c90-72c8-4955-b67c-1a4f92fb8e77'), 
run_mode = < AlgorithmRunMode.NOT_APPLICABLE: 0 > , 
name = '', 
slug = '', 
execution_callable = '', 
version = '', 
description = '', 
name_readable = '', 
additional_configuration = {}), 
end_date = datetime.datetime(2022, 6, 10, 17, 45, 15, 165083, tzinfo = datetime.timezone.utc), user_id = UUID('d09ca417-18e4-45db-a98e-490ac9712b4f'))

Is there something missing or a variable not set correctly especially for the payload?

[gkahiu]

Hi @azvoleff, further to earlier comment, I came across this here:

 if (
                        job.results is not None
                        and job.results.type == ResultType.TIME_SERIES_TABLE
                    ):
                        log(
                            f"Ignoring job {job.id!r} because it contains "
                            "timeseries results. Support for timeseries results "
                            "is not currently implemented"
                        )

Are time series jobs not supported? Kindly advise on how to go around this.

[gkahiu]

Am getting a failed status on the job:

Thank you for using Trends.Earth. The below task has FAILED. More details on this task are below:

Task name: Plot Test Kenya
Job: time-series 1_99_11
Task ID: ffec0515-00ae-41cb-8841-489c6c2a87a5
Start time: 2022-06-14 14:26:52.631363
End time: 2022-06-14 14:26:55.544339
Status: FAILED
For more information, and to view the results, view the datasets window in the QGIS plugin.

Thank you, The Trends.Earth Team

EDIT: Some additional information on the job:

...
"results": {
    "data_path": "",
    "type": "EmptyResults",
    "name": null
  },
  "task_notes": null,
  "progress": 0,
  "script": {
    "additional_configuration": {},
    "version": "",
    "slug": "",
    "execution_callable": "",
    "name": "",
    "id": "c0039c90-72c8-4955-b67c-1a4f92fb8e77",
    "run_mode": 0,
    "name_readable": "",
    "description": ""
  },
...

The script section is missing information (see above) such as name, slug, run_mode (in other scripts it shows remote but here it is set 0). Seems like these should be set in the server.

[azvoleff]

Hi @azvoleff, further to earlier comment, I came across this here:

 if (
                        job.results is not None
                        and job.results.type == ResultType.TIME_SERIES_TABLE
                    ):
                        log(
                            f"Ignoring job {job.id!r} because it contains "
                            "timeseries results. Support for timeseries results "
                            "is not currently implemented"
                        )

I think there are several things going on here. The above code is likely why the job was disappearing earlier after running it - the plugin was set to ignore these jobs given there was no functionality to display them yet. If you remove the above code the job should remain in the datasets list.

The other issue is that we made a change to our API last week, and all scripts ending in 1_99_11 should be run against the new version of the API (https://api2.trends.earth). Otherwise they will fail. The latest head of the develop repo should already be pointing to api2.trends.earth, but you'll need to make that change in your code if you don't yet have that change reflected in your branch. I just tested the time series script and it seems to be working fine on my end.

Last issue you noted was with the scripts list:

...
"results": {
    "data_path": "",
    "type": "EmptyResults",
    "name": null
  },
  "task_notes": null,
  "progress": 0,
  "script": {
    "additional_configuration": {},
    "version": "",
    "slug": "",
    "execution_callable": "",
    "name": "",
    "id": "c0039c90-72c8-4955-b67c-1a4f92fb8e77",
    "run_mode": 0,
    "name_readable": "",
    "description": ""
  },
...

The script section is missing information (see above) such as name, slug, run_mode (in other scripts it shows remote but here it is set 0). Seems like these should be set in the server.

I'm not sure why this isn't showing correctly - could be because ResultType.TIME_SERIES_TABLE is being ignored? Do these values show up properly when the code disabling ResultType.TIME_SERIES_TABLE is removed?

[gkahiu]

Thanks, the update works now. I've opened a PR, let me know if it works as expected.