GovReady / odesurvey

Center for Open Data Enterprise website and Open Data Impact Survey
http://www.opendataenterprise.org
2 stars 2 forks source link

Make "provide details" field required for "other" selection in "What are the most relevant types of data your organization uses" question. #238

Open gregelin opened 9 years ago

gregelin commented 9 years ago

When user indicates "Other" for survey "What are the most relevant types of data your organization uses?", the survey must require the "provide details to be filled out". It also must check the other is filled out before the form is submitted, so the value can be used in generating the top three country responses.

Can users leave "provide details" blank for data use type other?

There is an issue of complexity of triggering the overlay and generating the data use triples question. Current trigger is set to go off once the data type and number of countries fields are selected, regardless of order.

Example of data captured correctly and stored correctly:

2015-05-29 10:14:35; INFO; Array
[org_name] => Renewable Energy and Energy Efficiency Partnership (REEEP)
profile_id: HmPmVfzgVN
gregelin commented 9 years ago

@aa3181 I looked further into the storing of the data from the data impact use question when user selects "Other."

Currently the "provide details" field for "Other" in that question is not checked as a required field. Because the values of the data types question are used in the data use triples question which is generated dynamically, it is more complicated to indicate the field is required. A different mechanism must be used than the required check on the other field.

When "provide details" is left blank, "Other" is used as the value in arcgis_flatfile.

It is because that particular value is part of the fancy part of the page with the triples, that easiest path during initial development was to leave out creating a required value for Other's "provide detail" question.

audrey-a commented 9 years ago

I've edited the entries but am not sure if I've done it in the best way. --> indicates the change I made

Example of unedited entry: org_name: REEEP data_type: climate data_use_type: ["Energy","Environment","Other"]

Example of edited entry: org_name: Suzuki Engineering data_type: Sports --> changed to 'Other' data_use_type: blank --> ["Other (Sports)"]

audrey-a commented 9 years ago

@gregelin please let me know as I will need to do an upload by Sunday

please also confirm that changes are to be made in the arcgis flatfile only

gregelin commented 9 years ago

From log file of incoming data:

Array
(
    [org_name] => Renewable Energy and Energy Efficiency Partnership (REEEP)
    [org_description] => REEEP is a non-profit organization investing in clean energy markets in developing countries and running a comprehensive Open Knowledge Programme to support data, information and knowledge sharing in climate change 
    [org_type] => Nonprofit
    [org_url] => http://www.reeep.org
    [org_hq_city_all] => Austria
    [org_hq_city] => 
    [org_hq_st_prov] => 
    [org_hq_country] => AT
    [latitude] => 47.516231
    [longitude] => 14.550072
    [industry_id] => Energy
    [industry_other] => 
    [org_year_founded] => 2002
    [org_size_id] => 11-50
    [org_greatest_impact] => Environmental
    [org_greatest_impact_detail] => Supporting clean energy entrepreneurs in developing countries to ensure uptake of clean energy 
    [data_use_type] => Array
        (
            [0] => Energy
            [1] => Environment
            [2] => Other
        )

    [data_use_type_other] => Climate 
    [data_country_count] => 2 - 5
    [dataUseData-1] => Array
        (
            [src_country] => Array
                (
                    [src_country_locode] => GB
                    [type] => Array
                        (
                            [Energy] => Array
                                (
                                    [src_gov_level] => Array
                                        (
                                            [0] => National
                                        )

                                )

                            [Environment] => Array
                                (
                                    [src_gov_level] => Array
                                        (
                                            [0] => National
                                        )

                                )

                            [Climate ] => Array
                                (
                                    [src_gov_level] => Array
                                        (
                                            [0] => National
                                        )

                                )

                        )

                )

        )

    [dataUseData-2] => Array
        (
            [src_country] => Array
                (
                    [src_country_locode] => US
                    [type] => Array
                        (
                            [Energy] => Array
                                (
                                    [src_gov_level] => Array
                                        (
                                            [0] => National
                                        )

                                )

                            [Environment] => Array
                                (
                                    [src_gov_level] => Array
                                        (
                                            [0] => National
                                        )

                                )

                            [Climate ] => Array
                                (
                                    [src_gov_level] => Array
                                        (
                                            [0] => National
                                        )

                                )

                        )

                )

        )

    [dataUseData-3] => Array
        (
            [src_country] => Array
                (
                    [src_country_locode] => 
                )

        )

    [use_advocacy_desc] => 
    [use_prod_srvc] => True
    [use_prod_srvc_desc] => Using data (eg thesauri) for tools such as the climate tagger that supports tagging of unstructured content
    [use_org_opt_desc] => 
    [use_research_desc] => 
    [use_other_desc] => 
    [org_additional] => 
    [survey_contact_first] => Florian
    [survey_contact_last] => Bauer
    [survey_contact_title] => Operations & IT Director
    [survey_contact_email] => fb@reeep.org
    [survey_contact_phone] => 
    [org_profile_year] => 2015
    [org_profile_status] => submitted
    [org_profile_src] => survey
    [org_profile_category] => submitted survey
    [submit] => submit
)

From arcgis_flatfile:

{"createdAt":"2015-05-29",
"data_country_count":"2 - 5",
"data_src_country_locode":"US",
"data_src_country_name":"United States",
"data_src_gov_level":"National",
"data_type":"Climate ",
"data_use_type":["Energy","Environment","Other"],
"industry_id":"Energy",
"industry_other":"",
"latitude":47.516231,"longitude":14.550072,"no_org_url":false,"objectId":"GmnmD5ObPz",
"org_description":"REEEP is a non-profit organization investing in clean energy markets in developing countries and running a comprehensive Open Knowledge Programme to support data, information and knowledge sharing in climate change ",
"org_greatest_impact":"Environmental",
"org_greatest_impact_detail":"Supporting clean energy entrepreneurs in developing countries to ensure uptake of clean energy ",
"org_hq_city":"",
"org_hq_city_locode":null,"org_hq_country":"Austria",
"org_hq_country_income":"High income",
"org_hq_country_income_code":"HIC",
"org_hq_country_locode":"AT",
"org_hq_country_region":"Europe & Central Asia",
"org_hq_country_region_code":"ECA",
"org_hq_st_prov":"",
"org_name":"Renewable Energy and Energy Efficiency Partnership (REEEP)",
"org_open_corporates_id":null,"org_profile_category":"submitted survey",
"org_profile_src":"survey",
"org_profile_status":"publish",
"org_profile_year":2015,"org_size_id":"11-50",
"org_type":"Nonprofit",
"org_type_other":null,"org_url":"http:\/\/www.reeep.org",
"org_year_founded":2002,"profile_id":"HmPmVfzgVN",
"row_type":"data_use",
"updatedAt":"2015-07-26",
"use_advocacy":false,"use_advocacy_desc":"",
"use_org_opt":false,"use_other":false,"use_prod_srvc":true,"use_prod_srvc_desc":"Using data (eg thesauri) for tools such as the climate tagger that supports tagging of unstructured content",
"use_research":false,"date_created":"2015-05-29",
"date_modified":"2015-07-26",
"eligibility":"YY",
"use_org_opt_desc":null,"use_research_desc":null,"use_other_desc":null,"org_additional":null,"org_confidence":0},
audrey-a commented 9 years ago

thanks. please answer my questions?

On Aug 15, 2015, at 23:00, Greg Elin notifications@github.com wrote:

From log file of incoming data:

Array ( [org_name] => Renewable Energy and Energy Efficiency Partnership (REEEP) [org_description] => REEEP is a non-profit organization investing in clean energy markets in developing countries and running a comprehensive Open Knowledge Programme to support data, information and knowledge sharing in climate change [org_type] => Nonprofit [org_url] => http://www.reeep.org [org_hq_city_all] => Austria [org_hq_city] => [org_hq_st_prov] => [org_hq_country] => AT [latitude] => 47.516231 [longitude] => 14.550072 [industry_id] => Energy [industry_other] => [org_year_founded] => 2002 [org_size_id] => 11-50 [org_greatest_impact] => Environmental [org_greatest_impact_detail] => Supporting clean energy entrepreneurs in developing countries to ensure uptake of clean energy [data_use_type] => Array ( [0] => Energy [1] => Environment [2] => Other )

[data_use_type_other] => Climate 
[data_country_count] => 2 - 5
[dataUseData-1] => Array
    (
        [src_country] => Array
            (
                [src_country_locode] => GB
                [type] => Array
                    (
                        [Energy] => Array
                            (
                                [src_gov_level] => Array
                                    (
                                        [0] => National
                                    )

                            )

                        [Environment] => Array
                            (
                                [src_gov_level] => Array
                                    (
                                        [0] => National
                                    )

                            )

                        [Climate ] => Array
                            (
                                [src_gov_level] => Array
                                    (
                                        [0] => National
                                    )

                            )

                    )

            )

    )

[dataUseData-2] => Array
    (
        [src_country] => Array
            (
                [src_country_locode] => US
                [type] => Array
                    (
                        [Energy] => Array
                            (
                                [src_gov_level] => Array
                                    (
                                        [0] => National
                                    )

                            )

                        [Environment] => Array
                            (
                                [src_gov_level] => Array
                                    (
                                        [0] => National
                                    )

                            )

                        [Climate ] => Array
                            (
                                [src_gov_level] => Array
                                    (
                                        [0] => National
                                    )

                            )

                    )

            )

    )

[dataUseData-3] => Array
    (
        [src_country] => Array
            (
                [src_country_locode] => 
            )

    )

[use_advocacy_desc] => 
[use_prod_srvc] => True
[use_prod_srvc_desc] => Using data (eg thesauri) for tools such as the climate tagger that supports tagging of unstructured content
[use_org_opt_desc] => 
[use_research_desc] => 
[use_other_desc] => 
[org_additional] => 
[survey_contact_first] => Florian
[survey_contact_last] => Bauer
[survey_contact_title] => Operations & IT Director
[survey_contact_email] => fb@reeep.org
[survey_contact_phone] => 
[org_profile_year] => 2015
[org_profile_status] => submitted
[org_profile_src] => survey
[org_profile_category] => submitted survey
[submit] => submit

) From arcgis_flatfile:

{"createdAt":"2015-05-29", "data_country_count":"2 - 5", "data_src_country_locode":"US", "data_src_country_name":"United States", "data_src_gov_level":"National", "data_type":"Climate ", "data_use_type":["Energy","Environment","Other"], "industry_id":"Energy", "industry_other":"", "latitude":47.516231,"longitude":14.550072,"no_org_url":false,"objectId":"GmnmD5ObPz", "org_description":"REEEP is a non-profit organization investing in clean energy markets in developing countries and running a comprehensive Open Knowledge Programme to support data, information and knowledge sharing in climate change ", "org_greatest_impact":"Environmental", "org_greatest_impact_detail":"Supporting clean energy entrepreneurs in developing countries to ensure uptake of clean energy ", "org_hq_city":"", "org_hq_city_locode":null,"org_hq_country":"Austria", "org_hq_country_income":"High income", "org_hq_country_income_code":"HIC", "org_hq_country_locode":"AT", "org_hq_country_region":"Europe & Central Asia", "org_hq_country_region_code":"ECA", "org_hq_st_prov":"", "org_name":"Renewable Energy and Energy Efficiency Partnership (REEEP)", "org_open_corporates_id":null,"org_profile_category":"submitted survey", "org_profile_src":"survey", "org_profile_status":"publish", "org_profile_year":2015,"org_size_id":"11-50", "org_type":"Nonprofit", "org_type_other":null,"org_url":"http:\/\/www.reeep.org", "org_year_founded":2002,"profile_id":"HmPmVfzgVN", "row_type":"data_use", "updatedAt":"2015-07-26", "use_advocacy":false,"use_advocacy_desc":"", "use_org_opt":false,"use_other":false,"use_prod_srvc":true,"use_prod_srvc_desc":"Using data (eg thesauri) for tools such as the climate tagger that supports tagging of unstructured content", "use_research":false,"date_created":"2015-05-29", "date_modified":"2015-07-26", "eligibility":"YY", "use_org_opt_desc":null,"use_research_desc":null,"use_other_desc":null,"org_additional":null,"org_confidence":0}, — Reply to this email directly or view it on GitHub.

gregelin commented 9 years ago

@aa3181 I looked at your edits. We should discuss since there are some longer term implications.

I wrote a quick little script to identify records that had value for data_use that did not exist in data_use_type. This allowed me to see the records that had "Other"-related issues. You can access the spreadsheet here: https://docs.google.com/spreadsheets/d/1_Rxq8r0hdMkv1RMrx2Uamq7ZSK4-QdkJX-lUpWzQYEM/edit#gid=0

If the goal is to display "Other" as the common value (instead of the user entered for "Other" in the survey), I think I have fixed the records that needed to be changed.

But you can see in the spreadsheet that other discrepancies exist from variation in the original source inventory spreadsheet that we may want to normalize.