Update sidebar filter categories, new branch

[matamadio]

Changes to the sidebar filters:

• add hazard type/process
  • issue: if any of these are selected, all general exposure datasets will be automatically filtered out
• add license type
• add geo scale
• add "global" category among individual countries for whenever data is global
• drop resource type
• drop organisation

The filters reflect some of the core attributes of v0.2 - specific attribute names will be added here when template rdl-0.2.md will be ready. Suggesting to create a new branch rdl-0.2 for all proposed updates. Currently, the JKAN is deployed from branch rdl-0.1 into GH pages (https://github.com/GFDRR/rdl-jkan/settings), however in GH pages there seems to be a mixup of empty raw source content and deployed content (e.g. dataset folder). Please revise / clean deploy.

[lydiascarf]

everything's in rdl-0.2 so this should be good to close! i recommend making rdl-0.2 the default branch and deleting the gh-pages branch since none of the GH actions workflows use that branch and there's nothing really in it. it was a pleasure working together! let me know if i can be of service in the future

[matamadio]

Few more final changes to implement:

WEBSITE:

• Limit items in sidebar filters to 5 each
• Countries filter in alphabetical order.
  • Fix long countries names being split into different items, e.g. "The democratic republic of the" + "Congo"; "United republic of" + "Tanzania".
• Add "Hazard" and "Project" as filters.
  • Try to keep Exposure dataset shown independently of Hazard filtering
• Exposure category and Metric Quantity Kind fail to show multiple values on page
• Fix linking of hzd, exp and vln in the loss metadata
• Add "(meters)" to "Spatial resolution" label: Spatial resolution (meters)

IMPORT SCRIPT:

• Fails to translate hazard dataset where disaster id is specified, e.g. this and this.
• If resource download_url is null, then use access_url

[lydiascarf]

@matamadio thanks for summarizing these for me! i'll post an update around noon

[lydiascarf]

@matamadio how would you like the disaster ids to render? there's a scheme and an ID. should both be displayed? e.g. 2005-0370-TJK (EM-DAT International Disaster Database) for the "Central Asia flood scenarios" dataset

Also, the new hazard sidebar filter is based on the particular hazard's ID, not the hazard type or some other property, right?

[matamadio]

@matamadio how would you like the disaster ids to render? there's a scheme and an ID. should both be displayed? e.g. 2005-0370-TJK (EM-DAT International Disaster Database) for the "Central Asia flood scenarios" dataset

Yes that is fine for disaster, e.g. Disaster ID: 2005-0370-TJK (EM-DAT International Disaster Database)

Also, the new hazard sidebar filter is based on the particular hazard's ID, not the hazard type or some other property, right?

We want to use "hazard_type" as filter (flood, drought, etc).

[lydiascarf]

@matamadio just drawing your attention to some broken looking files:

• three datasets have some stuff that conforms to other models but not dataset models, e.g. this one, this one, and this one; they're getting read as datasets without a title, causing errors that could be confusing in the future; how should i resolve this in each case?
• also, this one has semi-colon delimited categories, which is breaking the display; it should be comma-delimited. i'll fix this one myself since it's clear what needs doing! but i thought i'd call it out for visibility
• i'm also seeing newlines added to the project field on all of the datasets with a project of the following:

  SFRARR - Strengthening Financial Resilience and Accelerating Risk Reduction
  in Central Asia program. (https://www.gfdrr.org/en/program/SFRARR-Central-Asia)

  this newline thing doesn't seem to be a problem yet, but i may have to look into it while working on the projects filter if it leads to weird behaviors in the sidebar

i snuck in a few other tweaks and fixes, but here's what i've completed so far this morning from your list:

• Add "(meters)" to "Spatial resolution" label: Spatial resolution (meters)
• Countries filter options should be in alphabetical order (i did this for all of the filters)
• Limit items in sidebar filters to 5 each
• Script fails to translate hazard dataset where disaster id is specified
• Script: If resource download_url is null, then use access_url
• Exposure category and Metric Quantity Kind fail to show multiple values on page

feel free to test and look through the diff on any of my commits!

[matamadio]

Thanks for the update!

three datasets have some stuff that conforms to other models but not dataset models, e.g. this one, this one, and this one; they're getting read as datasets without a title, causing errors that could be confusing in the future; how should i resolve this in each case?

What do you mean they do not conform on the model? They follow the same schema template, they have a title attribute. Previously the script managed to convert those, so it's not clear to me where the problem lies.

i'm also seeing newlines added to the project field on all of the datasets with a project of the following:

SFRARR - Strengthening Financial Resilience and Accelerating Risk Reduction in Central Asia program. (https://www.gfdrr.org/en/program/SFRARR-Central-Asia)

Yeah, I'm fixing this one in the jsons making it shorter, so it doesn't add a line break.

Countries filter options should be in alphabetical order (i did this for all of the filters)

Actually for the Geo scale it was good to follow the zoom logic from global > regional > national > sub-national

Script fails to translate hazard dataset where disaster id is specified

Script is now failing :( with this error:

  File "C:\Users\matti\Documents\GitHub\rdl-jkan-0.2\import\rdl2jkan.py", line 115
    props_to_summarize["disaster_identifiers"].append(f"{di.get("id")}; {di.get("scheme")}")
                                                                 ^^
SyntaxError: f-string: unmatched '('

[lydiascarf]

@matamadio

• for the datasets that don't conform, look to the bottom and you'll see there's an extra entry in each under datasets, e.g. here
• thanks for taking care of the line break issue! i'll ignore that for now
• i'll also revert the sorting for geo scale
• that f-string syntax error is interesting! i'm not seeing it on my machine, but i imagine it has to do with the nested " marks; i just switched to single quotes and pushed up; please let me know if that helps!

[lydiascarf]

@matamadio countries filter is fixed! i think i also got the following issue done:

Fix linking of hzd, exp and vln in the loss metadata

can you confirm that the only issue here was that on datasets like this one, the vulnerability field was actually linking to the related exposure? if so, that should be fixed as of my last push!

[matamadio]

Brilliant work, thanks!

• The script is now working!
• The name of countries are correct!
• The interlink is working in all the loss pages!
• Line break removed on json side!
• Scale is correct!
• I'm going to fix those unconformed data!

[lydiascarf]

@matamadio you should officially have everything but the hazard type filter! i have that almost done but i have some other work to get to now for another client. if i get a second i'll try to finish that up for you over the weekend! if we don't chat again beforehand, i hope your workshop goes great :)

[lydiascarf]

done! check out the PR linked above. please ignore the GitHub Action failing, that's just some half set-up linting. i can get that running smoothly and fix all the errors in a couple weeks if you decide to do another scope together. that would really help with stability in the future!