github-actions[bot]
commented
2 years ago Intent Cross-Validation Results (5 folds)
| class | support | f1-score | confused_with |
|---|---|---|---|
| macro avg | 2517 | 0.7392969443040304 | N/A |
| weighted avg | 2517 | 0.8044163022910361 | N/A |
| faq | 643 | 0.7808946171341925 | estimate_emissions(23), inquire-ask_clarification-offsets(21) |
| inform | 616 | 0.9413680781758957 | faq(14), estimate_emissions(8) |
| affirm | 255 | 0.8919449901768173 | faq(7), express_positive-emo(5) |
| inquire-ask_clarification-offsets | 124 | 0.7309236947791165 | faq(22), inform_notunderstanding(4) |
| estimate_emissions | 73 | 0.6071428571428571 | faq(11), inquire-ask_clarification-offsets(3) |
| deny | 69 | 0.7164179104477613 | inform_notunderstanding(4), faq(4) |
| greet | 63 | 0.8615384615384615 | faq(4), thank(2) |
| insult | 63 | 0.7333333333333333 | faq(5), inform_notunderstanding(3) |
| why | 59 | 0.7428571428571429 | faq(7), inquire-ask_clarification-offsets(6) |
| inform_notunderstanding | 58 | 0.4615384615384615 | faq(19), deny(3) |
| farewell | 57 | 0.7747747747747747 | faq(7), deny(3) |
| thank | 54 | 0.8928571428571429 | faq(1), affirm(1) |
| express_positive-emo | 48 | 0.7083333333333334 | SCENARIO(2), deny(2) |
| vulgar | 46 | 0.6388888888888888 | faq(13), insult(7) |
| express_surprise | 43 | 0.6666666666666666 | faq(10), express_positive-emo(2) |
| express_uncertainty | 43 | 0.7341772151898734 | faq(7), affirm(2) |
| inquire-ask_clarification | 38 | 0.6486486486486486 | faq(8), why(3) |
| buy_offsets | 35 | 0.6562500000000000 | faq(8), affirm(2) |
| how_calculated | 29 | 0.8363636363636363 | faq(5), estimate_emissions(1) |
| deny_flying | 28 | 0.5937499999999999 | faq(2), deny(2) |
| express_negative-emo | 25 | 0.6222222222222222 | faq(3), inform_notunderstanding(2) |
| restart | 18 | 0.8648648648648649 | affirm(1), faq(1) |
| meta_inform_problem_bad-link | 12 | 0.9600000000000001 | N/A |
| SCENARIO | 10 | 0.6666666666666665 | faq(1), express_positive-emo(1) |
| help | 8 | 0.7500000000000000 | faq(1), inquire-ask_clarification-offsets(1) |
Entity Cross-Validation Results (5 folds)
| entity | support | f1-score | precision | recall |
|---|---|---|---|---|
| micro avg | 926 | 0.79567567567567580 | 0.7965367965367965 | 0.7948164146868251 |
| macro avg | 926 | 0.68244951237341060 | 0.7548112671820053 | 0.6466578336409559 |
| weighted avg | 926 | 0.79334634425227450 | 0.7947887275586288 | 0.7948164146868251 |
| city | 384 | 0.87741935483870970 | 0.8695652173913043 | 0.8854166666666666 |
| city.to | 182 | 0.76719576719576720 | 0.7397959183673469 | 0.7967032967032966 |
| city.from | 149 | 0.68918918918918930 | 0.6938775510204082 | 0.6845637583892618 |
| travel_flight_class | 95 | 0.87830687830687850 | 0.8829787234042553 | 0.8736842105263158 |
| iata | 76 | 0.67605633802816900 | 0.7272727272727273 | 0.6315789473684210 |
| iata.to | 19 | 0.57142857142857140 | 0.6250000000000000 | 0.5263157894736842 |
| iata.from | 16 | 0.42857142857142855 | 0.5000000000000000 | 0.3750000000000000 |
| number | 5 | 0.57142857142857150 | 1.0000000000000000 | 0.4000000000000000 |
The Climatiq API estimates CO2 given a distance or IATA codes. To make requests, we need an API key.
An important part of using the API is choosing the emission factors to be used. An emission factor is a coefficient used to compute the CO2 estimate from parameters like distance or flight class. Emission factors differ depending on the distance: E.g., long-haul flight planes fly higher, where the real impact of emitted CO2 is also higher. Different emission factors exist based on different methodologies, e.g. in the UK the emission factors also account for a plane's contrails, for the different greenhouse gases emitted, and upweight the estimate in a particular way for long-haul flights. Institutions in other countries produce different numbers as they use slightly different approaches.
In this PR, I chose the UK emission factors (but it's easy to switch to ones from a different source). They seem to be quite conservative, i.e. they account for emission effects that get ignored in many other emission factors. The UK factors also distinguish between domestic and international flights in a particular way that varies between countries and methodologies -- in this case,
domesticmeans any flight within the UK, i.e. up to ~760 km.Besides choosing the emission factors, this PR also adds the API key into the action server Docker image so that requests can be made when the action server is deployed (the actual API key is stored as a secret on the
carbon-botrepo).In terms of testing that everything works (around the API), I've checked this locally but I admit that we might want to set up a unit test for the action server.