standardizing weather description

[joons5834]

KMA

- 하늘상태(SKY) 코드 : 맑음(1), 구름많음(3), 흐림(4)
- 강수형태(PTY) 코드 : (초단기) 없음(0), 비(1), 비/눈(2), 눈(3), 빗방울(5), 빗방울눈날림(6), 눈날림(7) 
                      (단기) 없음(0), 비(1), 비/눈(2), 눈(3), 소나기(4) 

OpenWeather https://openweathermap.org/weather-conditions

NWS Natural language desc + METAR/TAF code

Yr Unknown

[joons5834]

KMA SKY, PTY -> OpenWeather code

• 하늘상태(SKY) 코드 : 맑음(1) -> 800 , 구름많음(3)->803 흐림(4) -> 804
• 강수형태(PTY) 코드 : (초단기) 없음(0)->None, 비(1) ->5xx, 비/눈(2)->616, 눈(3)->600-602, 빗방울(5)->3xx, 빗방울눈날림(6)->615, 눈날림(7)->600 (단기) 없음(0), 비(1)->515, 비/눈(2)->616, 눈(3)->600-602, 소나기(4) ->520-531

cannot pinpoint OpenWeather code since KMA has larger granularity.

[joons5834]

OpenWeather -> KMA code multiple codes -> 1 KMA code for SKY and PTY respectively.

Group 2xx: Thunderstorm -> LGT(0.2~100KA/㎢, only on KMA ultra short forecast) Group 3xx: Drizzle -> 빗방울(5) Group 5xx: Rain -> 비(1) Group 6xx: Snow -> 눈(3) Group 5xx: Rain -> 비(1) Group 7xx: Atmosphere Group 800: Clear -> 맑음(1) 802-803: 구름 많음(3) 803-804: 흐림(4)

loss of finer granularity except Thunderstorm

[joons5834]

Possible solutions

• split weather description to several columns

  • complicated string parsing

• choose KMA-style in favor of coarse desc. + numeric fields(eg. 'rain', 80mm) rather than finer description with sub desc. (eg. 'light/medium/heavy rain')

  • evaluating accuracy of sub desc. is not practical and error-prone.
  • categorizing sub desc. is error-prone
  • description not covered by KMA code does not belong to MVP

• use raw weather description as-is without standardization

  • cannot filter/aggregate on weather description: no apples-to-apples comparison
  • used only for evaluating forecast performance -> cannot evaluate forecast performance if observation service != forecast service is true
  • avoids interpolation/loss of data granularity during conversion
  • a simple and easy solution

• use METAR/TAF code instead of KMA code