Enhancement: FWI direct download from EFFIS and GeoJSON conversion

pyronear / pyro-risks

Data science for wildfire risk forecasting and monitoring

https://pyronear.github.io/pyro-risks

Apache License 2.0

25 stars 8 forks source link

Enhancement: FWI direct download from EFFIS and GeoJSON conversion #77

Closed GHCamille closed 7 months ago

GHCamille commented 11 months ago

This pull request introduces several enhancements to the existing codebase:

Ability to write a JSON file to a s3 locally (using localstack)
FWI downloading through requests package
GeoJSON formatting so that it can be used in Pyronear platform

These changes aims at providing a risk MVP for the platform. There are still some grey areas for me, please MP me after reviewing so that we can go over it !

Please review the changes and provide feedback or approvals as necessary. Thank you!

juldpnt commented 10 months ago

Hello @GHCamille , thanks for this huge PR ! In fwi_helpers.py I don't really understand how fwi_category works:

 categories = [
            (58, 6),
            (145, 1),
            (192, 5),
            (210, 2),
            (231, 4),
        ]

Do you have a link where it is stated how categories are related to their values ? As value(2) > value(4) but value(2) < value(6), it is quite odd to me ?

Acruve15 commented 10 months ago

[QUESTION] Should we keep the current CICD? Seems deprecated (@jsakv)

jsakv commented 10 months ago

[QUESTION] Should we keep the current CICD? Seems deprecated (@jsakv)

Yes @Acruve15 the CI/CD is deprecated, multiple directories and modules are also deprecated and some unit tests are calling external API endpoints that are deprecated.

What we can do is to:

Add the changes we want to make within the scope of this PR (FWI download and conversion) and merge the PR once we are ok with the code and ignore the CI/CD for now
Open a second PR for the refactoring/cleaning of the repository. I have started deleting deprecated files and directories and updating the CI/CD on another branch, once the PR is merged, I can rebase and update the branch raise a PR for review.

Let me know if works for you!