APSIM downloads database clean-up

[hol353]

I have just tried to work with the APSIM downloads database to generate a downloads map by country. Problems that I encountered:

• The csv version of the database has several rows that are badly formatted e.g. the country is in the wrong column. Probably because the user has entered a comma in one of the fields.
• The country names in the dropdown on the registration page on www.apsim.info are not standard. The Python library pycountry doesn't recognise some of them. Ultimately we need to get from country name to country code so that they are easy to map. Perhaps we should maintain a db table for mapping country names to codes.
• The APSIM Next Generation upgrade form uses an edit box for entry of country name. This needs to be a mandatory dropdown field to stop users entering non-standard country names.
• The database should then store country codes rather than country names. A conversion of the existing country names to country code will be required.
• Currently the database has products for each version of APSIM classic (e.g. APSIM7.10, APSIM7.9...) and for APSIM Next Generation (e.g. APSIM Next Generation 2019.05.05.1453). It is important to capture the version number but we should do that in a separate field in the database. Then the products would become 'APSIM' and 'APSIM Next Generation'. This makes it easier to plot the number of APSIM and APSIM Next Generation downloads over time.

[hol430]

Why do we collect state/city/street address info from the registration form? Can this be removed altogether?

[hol353]

@sarahcleary Do we still need to collect address information from users when they download APSIM. We never use the information for anything.

[sarahcleary]

We need country and email address. We may need additional information when setting up the click-wrap commercial license.

[hol353]

Yes I agree we need country and email address but what about street address, city etc? Can we remove them?

[sarahcleary]

OK to remove

[hol430]

Do we want some way to distinguish between upgrades (via apsimx upgrade form) and registrations (via web form)?

[zur003]

I would think we'd want to be able to distinguish new registrations from upgrades. It would make it easier to estimate the actual number of distinct users.

[sarahcleary]

Agreed - if possible

[hol430]

We can currently download the registrations database in .csv format via a public-facing web service. Should this .csv file omit sensitive information such as name/email?

[sarahcleary]

yes

[hol430]

Ok I've addressed all of the issues raised in this thread and have cleaned up all of the country entries in the database. Anyone using an old version of apsim will still be using a text entry for country and so will still be able to submit a non-standard country name. To work around this, I've changed the registrations web service to ignore any requests with an invalid country.

I've also finished up that python script you sent me to generate a map of downloads by country. Source code is here.