Multiple imagery selection

[geohacker]

We will now start supporting Sentinel. There are a couple implications here:

1. User should be able to select the imagery they want to use
2. User should only see imagery available to their current AOI
3. User should only see models applicable to their selected imagery
4. For Sentinel, user should be able to select a timeframe. With a start and end date

Once the user selects imagery and timeframe, the map should update with the new mosaic. We should also use this to tell the inference API to use the tiles.

@LanesGood @vgeorge this requires some thinking of the design side. cc @ingalls @vincentsarago @batpad

[LanesGood]

Noting that there are already some mockups and design ideas and questions in https://github.com/developmentseed/pearl-frontend-internal/issues/491.

Ex:

[LanesGood]

Some followup questions here:

1. User should be able to select the imagery they want to use

Will the application permit a user to change the imagery source within a single project? AKA, can I run inference on an AOI with Sentinel imagery, and then (if imagery available) run inference on the same AOI but with NAIP imagery?

4. For Sentinel, user should be able to select a timeframe. With a start and end date

What is displayed on the interface? Does sentinel provide a single day's scene for the entire timeframe specified, or are there mosaics of multiple scenes over multiple capture days/times within that timeframe? Is there a limit/standard range of dates?

[geohacker]

Thank you @LanesGood!

Will the application permit a user to change the imagery source within a single project? AKA, can I run inference on an AOI with Sentinel imagery, and then (if imagery available) run inference on the same AOI but with NAIP imagery?

I think we may not support this. At the moment we set the imagery source when the project is created with the assumption that we may not actually change the source later on. It might be best to not change this assumption. The other consideration is that changing imagery will most likely need the user to change the model which may complicate the workflow. cc @ingalls

What is displayed on the interface? Does sentinel provide a single day's scene for the entire timeframe specified, or are there mosaics of multiple scenes over multiple capture days/times within that timeframe? Is there a limit/standard range of dates?

We are limited (in a good way) by the abilities of the Planetary Computer. The PC explorer does a good job of showing what the available options are https://planetarycomputer.microsoft.com/explore?c=-100.0038%2C21.2617&z=7.50&v=2&d=sentinel-2-l2a&m=cql%3A3168344d75a04a90d6a4c6204e033db5&r=Natural+color&s=false%3A%3A100%3A%3Atrue&sr=desc&ae=0

The most important one for us are acquired dates and cloud cover. We might want to limit the range to select quarterly and annual because daily imagery I don't think will be useful with the models we are building. cc @srmsoumya @vincentsarago

[LanesGood]

Thanks @geohacker One thing I'm still a bit confused by for the frontend date selection: what is the date range of a sentinel mosaic, and how is the data composed? For example, if a user sets 1/10/2020 - 1/10/2021 as their range:

1. Is each tile of the mosaic presented only the most recent image?
2. Is it clear to the user what timespan a tile may represent? Is it possible and/or useful to indicate this?
3. Is it possible to indicate which sentinel platform the tile is from?
4. Is this year-long date range of a mosaic even possible? Or is there a limit to the span of a mosaic?

[geohacker]

@LanesGood I'll try to expand on my thinking so far:

1. User creates a project. At the time of creating a project we can ask them to select between NAIP or Sentinel 2. For now, we can let them not change this. Though, the API doesn't have this restriction. Imagery source can be changed at the AOI level.
2. When Sentinel is selected, the user gets a month picker. The picker will enforce that the user select at least 3 months. Start month - End month. So that this is at least a quarter. The biggest reason for doing this to avoid running the model on a sparse mosaic.
3. When the user selects the month range, we will query the planetary computer API and create a mosaic search id (this is also now the unique mosaic ID we will use). This is stored in the PEARL database as an imagery source. We can automatically name this as Sentinel-2 April - June 2020 or something.
4. Once the imagery source and mosaic is selected, this will be visible on the frontend map
5. The next step is for the user to draw the AOI. And then run inference, retraining etc.
6. Any point in this workflow, the user will be able to change the mosaic by searching for a different quarter, for the current AOI. This will just create a new inference, retraining etc.

Does this make sense?

To answer your questions above:

Is each tile of the mosaic presented only the most recent image?

A mosaic is a bunch of GeoTIFFs stitched together. For example, if you check this planetary computer explore link, it pulls Sentinel 2 imagery in a region in Mexico. The list shows all individual scenes. We can use the PC API to generate a mosaic URL from this, based on a search ID. Which will look like for example https://planetarycomputer.microsoft.com/api/data/v1/mosaic/tiles/82ebdc445544365e45be4db6d22536ec/{z}/{x}/{y}?assets=B04&assets=B03&assets=B02&color_formula=Gamma+RGB+3.2+Saturation+0.8+Sigmoidal+RGB+25+0.35&collection=sentinel-2-l2a. Internally, the API grabs the individual geotiffs, stitches them together as a mosaic, and then tiles them. So each tile will be in the time frame but there is no guarantee that two adjacent tiles are from the same day/time. There's a way to set priority I think which we'll figure out.

Is it clear to the user what timespan a tile may represent? Is it possible and/or useful to indicate this?

May be, I'm not sure. But since we work with 3 months at a time, I think this would be ok. It will be very complex for us to do this. So for the first pass, I think we can not worry.

Is it possible to indicate which sentinel platform the tile is from?

We'll only use Sentinel 2

Is this year-long date range of a mosaic even possible? Or is there a limit to the span of a mosaic?

Multiyear mosaics are possible (NAIP for example). But for the current use-case we'll stick to 3 months as the change analysis is quarterly.

@ingalls @vgeorge @srmsoumya @vincentsarago please add questions and thoughts you all have. Thank you!

[LanesGood]

Thanks again for the comments @geohacker - as mentioned on slack I think this is a good basis for documentation.

The workflow proposed during today's stand up is as follows:

1. On project start, set name of project (as existing)
2. User first navigates to desired location (likely need generic mapbox/bing satellite map, or latest sentinel, loaded for global coverage)
3. User draws AOI
4. Based on AOI, user is presented with available imagery source selection
5. User selects timespan. Will be able to select presets, or choose 3 month span from arbitrary start date, likely using date picker UI
6. Based on AOI and imagery source, user selects model
7. Run model

As discussed:

• checkpoints can be saved and run on multiple AOIs and timespans, as on existing
• Updated UI will include timespan information on each AOI
• Updated UI can collapse/deemphasize imagery source and model after selection as these will be fixed
• Updated UI can include a label with timespan on the AOI bbox on the map for greater indication to user

There are likely more implications for timespans in the share/export/project page flow still to come

[LanesGood]

I've created a Figma prototype for this flow.

Some questions regarding these wireframes and the flow:

• does it make sense to ask users to first create the AOI before selecting imagery source and model, given that the later two are dependent on the first?
• alternatively, should imagery and model be selectable first, and then a "permitted" bounds area be added to indicate to users where imagery exists for the model?
  • For imagery, this could be as simple as "zoom in to see imagery"

The main changes on this include the selection icons/layout for Imagery and Model, given that we won't permit those to be changed once the model has been run for the first itme.

Some remaining decisions to make include:

• should the selectable options in the Predict tab be re-ordered to reflect what is modifiable vs. fixed after first model run? see suggested changed screen below (timespan, AOI and checkpoint below the fixed items)
• 

Other questions to enhance the workflow:

• If a user runs a model on an AOI and then changes the timespan and runs the model again, do we list the AOI in the list of available AOI's again, but with the different timespan associated? (I assume yes)

[vgeorge]

AOI first vs Mosaic/Model first

Considering only the user experience perspective I believe it makes more sense to allow drawing an AOI before selecting a mosaic/model.

In most use cases the user should already have a target area that they want to run predictions. Making the user to browse through mosaics and models to find one that includes the area seems more involved than inferring the suitable mosaics/models from an AOI.

But we might need to change the endpoint GET /mosaic to include the bounds in the response, otherwise, the client will need to do a request to GET /mosaic/:mosaic for each mosaic bounds.

The endpoint GET /model already returns the bounds for each model, but it is paginated. We might need to consider a bounds filter for both endpoints that return the elements that cover the area or to have a parameter that disables pagination.

cc @geohacker

Other UI questions

@LanesGood not sure if you already have a decision, but I believe it makes sense to keep the fixed parts separated. I also agree to list AOIs with different timestamps on separate lines, but I don't have a strong opinion on this.

[geohacker]

But we might need to change the endpoint GET /mosaic to include the bounds in the response

Yea we can do that

@vgeorge while implementing this, I'd approach is from an MVP standpoint. So if we are going to hardcode somethings specific to sentinel I think that's ok on the frontend. For example, the bbox. This is both to avoid making too many changes on the API that we might want to change after testing the approach and to save time on implementation.

[vgeorge]

@geohacker and I had a chat today to revisit this, here are my notes:

• Desired workflow: User draws AOI, selects Sentinel as imagery source, opens mosaic selector modal, picks an existing mosaic in the list or clicks a button to search for a new one. This will open another modal that displays a map, a date picker, and a few other filters (like cloud cover). Once the user finishes the search, the mosaic is applied. The new mosaic isn't created in the API at this point, but when the user requests a prediction. This will send a request to /mosaic endpoint with the parameters before requesting an instance
• All Sentinel mosaics will have a fixed time range of 90 days (or 3 months?)
• The user will be able to search for mosaics only when Sentinel is selected as the imagery source, NAIP will still be hard-coded

Please let me know if you have thoughts or questions.

[LanesGood]

@vgeorge can you provide feedback on the below screens? We discussed that mosaic selection and creation would take place in a modal. To permit both actions, I've added tabs to the modal for "Preset Mosaic" vs "Create New Mosaic."

Preset Mosaic

• Ideally, we could show a preview of each mosaic over the currently selected AOI (or even viewport). This may be a stretch goal

Create New Mosaic

• I suggest that a tabbed modal is a more discoverable workflow than a nested modal
• Clicking "Start date" would display a calendar date picker. This could be the native browser date picker for now.
• End date is automatically updated based on selected start date. End date input is disabled and can not be selected; an info icon displays the tooltip text "End date automatically set to 90 days following start date" or similar text
• User may set mosaic cloud cover percentage (may be a stretch goal)
• Map displays preview of selected mosaic date range and cloud cover over selected AOI/viewport (may be a stretch goal)
• Clicking "Create mosaic" applies the mosaic and closes the modal - there's no need to go back to the "preset" tab. However, opening mosaic tab in the future should show the user-created mosaic in the list of presets

[geohacker]

@LanesGood this looks great to me. I think the preview of current viewport on the presets might be tricky. We may want to just use the thumbnail Planetary Computer returns as default.

For reference, this is how we create new mosaics manually https://github.com/developmentseed/pearl-backend/blob/project-ts/helpers/create_sentinel_mosaic.py#L10-L21. The cloud cover is by default set to 50% in the request. cc @vgeorge

[vgeorge]

@LanesGood this workflow looks really good. Regarding the PC thumbnails, it seems they are only available for imagery sources, not for mosaics. If we want to display thumbnails for mosaics, we need to show a static map with the mosaic as the base layer on each card.

[LanesGood]

We've implemented this ticket. Continued revision and work is detailed in #85