Closed gilliankerr closed 7 years ago
@gilliankerr in relation to Review and make recommendations on all OrgWise surveys #17, all the orgwise surveys look complicated because all the question groups are named the same. It's not clear which of the 7 domains the questions fall under and the questions are repeated with the same answers.
The questions should first be grouped by area then similar questions should be combined into single questions with subquestions like what we did for the SIS survey. For ease of navigation in lime, we need to define the subsections and name the groups appropriately.
For example, there are seven areas in the Organizational Capacity Assessment Tool:
Next there are currently 44 subsections, each with their own descriptions:
@gilliankerr can you please identify which area each subsection belongs to. I will prepare an excel file so you do this easily.
@traciwilson Could you identify these areas using the excel file that Mark will provide?
@traciwilson can you please review this list and send it back with the correct area assignments. section list.zip
@gilliankerr please see my comments below:
Can you correct most of them in csv and then re-upload the content? If you need to refer to the original content, here's the material - https://usaidlearninglab.org/library/organizational-capacity-assessment
can correct them faster on the web but i think we need to restructure the survey first.
The OCA should be split into seven surveys, one for each of 7 domains: governance, finance, administration, human resources, program management, project performance management, and sustainability. Each domain should have its own welcome page (with the nice image). For now, use the same content on each section that is now on the full survey. I will send individual content for each module in a few days. (If you have other suggestions for how to split them I would be interested.)
if this survey already has all the questions needed, all we need to do is add one more question at the beginning of the survey asking which areas to display. please see previous comment with regards to grouping the sections in the current survey. once we have identified which area each section belongs to we can control the display by area.
The questions should be numbered. Possibly each section should have a table of contents - is that possible? Does it look okay?
a table of contents may not be necessary for display after we restructure the questions and groups. but we definitely need to define the survey better in a word doc. this will be the basic for creating the questions groups and the questions themselves. i think the survey needs to be redone completely.
The intro section should have better fonts for the headings. E.g., in the following screenshot, the heading and subsection headings are underlined (which should never happen on the web unless they are hyperlinks) and the font seems too small. I will be providing content soon.
i think the survey would look better after the questions are restructured and survey is set to display question by question. it would also be easier to manage the questions after that.
These are excellent recommendations; I agree. I have asked Moiz for this spreadsheet - he generated these questions - and for his advice on how to do this. If he doesn't have it handy, perhaps @traciwilson can do this..? We will know Moiz's response by tomorrow morning.
@mmirasol Could you take this task? Traci is busy with the other 4 surveys, and you have a clear idea of what is required. You would be splitting up the questions into the correct 7 divisions by referring to the source document, adding Traci's introductory text on each survey, fixing typos. Is that okay? We need to complete it by Monday, and the interviews are starting Tuesday. Let me know if that's a problem.
@gilliankerr yes i will be happy to fix all the text and regroup all the questions.
I created_ a copy of the current survey and i can proceed with making changes to those. I see tracy made the same recommendations as i did the other regarding the groupings on the other 4 surveys.
Do you still wish to create 7 separate surveys or just use one main survey with an initial question asking which areas to show to control the logic?
@gilliankerr i started working on the text changes. my first few comments on this. please open the sheet attached and go to the Review tab to go thru my comments. Organizational Capacity Survey (293 Question & Thier Answers) - 20170603.01.xlsx
This is how we used to collaborate on survey specs. We have the specs on an easy to read document. This one is much easier to read than the lime export and named with a date and serial number to track how many times it has been changed and also to exchange comments on the document directly.
or perhaps it might be better to discuss the issues here directly. here are the first few.
@traciwilson
@traciwilson no action has been made on these yet. please confirm and advise back.
@mmirasol Please see updated spreadsheet - I made changes to the first 5 questions! Organizational.Capacity.Survey.293.Question.Thier.Answers.-.20170603.01_TW.xlsx
@traciwilson @gilliankerr changes applied.
@traciwilson
See below for reference.
The text in the spec document have them in multiple lines but these would look better as straight sentences like so.
@traciwilson questions with incomplete answers like these have been corrected as recommended above.
@mmirasol Great, thanks. If time permits, please change the instruction to "Please choose the answer that best describes your board" Also, if time permits, could you please change the first option to "…not been established or is not functional, has no written terms of reference or does not understand its functions, and plays only an advisory role". This helps clarify it a bit for the reader.
@gilliankerr @traciwilson changes applied.
@traciwilson This question has two options named "The accounting system is cash based".
@traciwilson shall i remove one?
@mmirasol Yes, please remove one. There will only be three answer options, for a score of 1, 3, and 4 respectively.
@traciwilson @gilliankerr changes applied.
@traciwilson This one has two options for "The accounting system is computerized" and the answers don't sound right because it's also asking "what software was used".
@mmirasol Please change the question to simply, "Is the accounting system manual or computerized?" and answer options to -manual (1 pt) -computerized (4 pts) I don't think it makes sense to have an option "manual or computerized".
@gilliankerr @traciwilson changes applied.
@traciwilson This question does not answer "when the chart of accounts was last updated" and the second answer is repeated 3 times.
@mmirasol Please remove the question about when the chart of accounts was last updated. Please remove 2 of the 3 identical answers. So there will be just two answers: -Not based on a standard chart of accounts that corresponds to the general ledger (1pt) -Based on a standard chart of accounts that corresponds to the general ledger (4pts)
@traciwilson @gilliankerr changes applied.
@traciwilson This one has duplicate options too.
@traciwilson can i remove the last two?
@mmirasol Please change the answer options to: -checks are not secured -checks are secured
@traciwilson @gilliankerr corrected.
@traciwilson The answers don't match this question.
@traciwilson @gilliankerr corrected.
@traciwilson The answers are not appropriate for this question.
@mmirasol Please change question to: "what are check-signing procedures? For example, does the organization ever make checks out to “cash” or “bearer”? Are blank checks ever pre-signed? Under what circumstances?" Please change answer options to: • Checks are pre-signed or made out to “cash” or “bearer” • Checks are not pre-signed nor made out to “cash” or “bearer”
@traciwilson @gilliankerr corrected.
@traciwilson Duplicate options for this question too.
@traciwilson can i remove the last two?
@mmirasol Yes, please.
@traciwilson @gilliankerr corrected.
@traciwilson The first two options for this question look the same to me. Please advise.
@mmirasol Yes, please remove one. The answer should read "Managers do not regularly review financial statements, and when they do, rarely take corrective actions based on the review"
@traciwilson @gilliankerr corrected.
@traciwilson The answers do not match this question.
@mmirasol Looking at the spreadsheet, it appears that the answer options for this question are the same as the two previous questions. Is this still the case? If so, we'll need to review. Please remove the variance reporting question, as there aren't any matching answer sets for it.
@traciwilson done.
As of this writing, i have correction the question text for 11/45 question groups. Have to pause temporarily to rest a bit. I'm also at 19/20 hours this week. @gilliankerr Please add 10 more hours for this week and I will resume working on the text corrections after dinner tonight.
In case you would like to review, the corrected OCA survey is the inactive one. I did not correct the group descriptions yet as the content provided by Tracy requires html formatting. Will be doing those on the front end.
@gilliankerr @traciwilson Also, instead of displaying all the questions in the same group in one page, I would recommend displaying the questions one by one and to show the group description before displaying the questions.
For example, show the text below before showing the first question for that group. Vision and Mission Objective: Assess the clarity of the organization’s statements of its purpose and values and how they have been shared and applied. Before completing this section, please refer to your organization’s: Vision statement; mission statement; and board, senior manager, and staff questionnaires or interviews
what do you think of this?
@mmirasol when you see errors or duplications in the questions, flag them for @traciwilson to review. Traci can go back to the original source and check for accuracy. You will have to post the surveys publicly so that Traci can see them, or set her up with appropriate user permissions.
hi @gilliankerr i flagged all the comments above for @traciwilson and i posted the source file in a comment below. all the issue posted here need review because they either don't match the question or simply don't make sense. the ones that were incomplete i corrected directly.
Other than errors and questions in the text, what other issues are outstanding? You and I can discuss them before I leave the house (in about 40 minutes); then I will be unavailable until evening.
@gilliankerr re: issues
@gilliankerr re: other recommendations
@mmirasol what is the best way to communicate changes/corrections to you after I review? When you see issues such as the text didn't come through in the limesurvey, and you have the full text in the source document, I would suggest just fixing on the spot, rather than creating an issue log for it. This removes several steps of work.
Also, if you'd like to set me up with user permissions, I can make changes or corrections directly as I review them. Thanks!
@traciwilson best way is for you to search for the question in each of the comments above in the source file. the question column already has a filter so you can search for those quickly and respond to each comment above like im doing for you now and put the corrections there. by doing this we don't have to send files back and forth and we will have everything on github. with regards to lime permissions, that won't work as i am replacing the survey on every import so only one person can work on text changes right now. i have only fixed about 25% of the survey so i will have to incorporate your corrections as soon as you post them on this task.
I set you up with user permissions on limesurvey ; let me know if you have any problems with it
hi @gilliankerr @traciwilson sorry for the wait. had to drive late last night. got ko'd.
all the issues in text that can be fixed are already fixed actually. the issues i have posted here are the ones with options that don't make sense. i posted the issues directly on this log so you didn't have to look at the source document but we can do that also so you have full reference.
@traciwilson here's the source file for reference. i initially thought of putting my comments there but thought it would be better to put them here directly so you can see the issues directly and we don't have to send files back and forth. Organizational Capacity Survey (293 Question & Thier Answers) - 20170603.01.xlsx
@gilliankerr traci and i won't be able to make lime changes simultaneously because we are replacing the survey on each import.
@traciwilson please search for the beginning part of the questions in excel. See example below. allows you to find the question with text "Does the organization have an external board of directors that is independent of management.." quickly.
Kindly edit my comments above and put in the corrections below them addressing them to me. I will go back up and check which ones you have responded to and update the same comment once the fix has been applied.
An example for the first issue above, @traciwilson There are questions that have the exact same answers as the one before them. Questions should match their answers and it looks like this should be one question split into two. @mmirasol your reply here. @traciwilson corrections applied.
@mmirasol Thanks so much for the explanation. I am reviewing the questions now and will post in each comment individually.
@gilliankerr @traciwilson i should be done with the first 3/7 areas within the next hour. I will importing these as separate surveys so tracy can test them and put in the correct descriptions for the groups.
@traciwilson i think the answers need rephrasing.
@mmirasol Answer options should be: • Notices required by law or donors are not posted • Notices required by law or donors are not usually posted • Notices required by law or donors are usually posted
@traciwilson @gilliankerr corrected.
@traciwilson @gilliankerr you can start making corrections to these surveys directly. the typos from the questions and answer texts have been removed. only the issues posted above remain. the into text for the survey and each section also need to be updated.
@gilliankerr just finished making changes to the first survey. kindly test @. Governance and legal structure http://logicaloutcomes.net/forms/index.php/admin/survey/sa/view/surveyid/235289 test link: http://logicaloutcomes.net/forms/index.php/235289?newtest=Y
@gilliankerr just finished making changes to the second survey. kindly test. Financial management and internal control systems http://logicaloutcomes.net/forms/index.php/admin/survey/sa/view/surveyid/345769 test link: http://logicaloutcomes.net/forms/index.php/345769?newtest=Y
@gilliankerr just finished making changes to the third survey. kindly test. Administration and procurement systems http://logicaloutcomes.net/forms/index.php/admin/survey/sa/view/surveyid/139821 test link: http://logicaloutcomes.net/forms/index.php/139821?newtest=Y
Human resource systems http://logicaloutcomes.net/forms/index.php/admin/survey/sa/view/surveyid/386626 test link: http://logicaloutcomes.net/forms/index.php/386626?newtest=Y
@gilliankerr just added the 4th survey.
@traciwilson please confirm if you were able to login to lime and see the surveys mentioned. i cannot check your login because @gilliankerr created it. but i can create a new one for you if needed. let me know if you need guidance in navigating lime and testing surveys. please add me on skype mirasol.mark
@mmirasol I just tested and yes, I'm able to log in and view the surveys above. I'm going offline now but I'll be back in about 12 hours. I think I've answered all the issues you posted above. Thanks for your help!
@traciwilson noted on your comments. thank you very much will apply them as soon as i've gone thru all the sections so i can apply all the changes in one go.
@traciwilson these two questions have the same answers.
@mmirasol Let's combine the two questions, so the question should read: Are there written policies on job descriptions and delegations of authority that are complete and appropriate? Do job descriptions include position titles and clearly specify roles and responsibilities, required qualifications and skills, and reporting?
@traciwilson @gilliankerr corrected.
thanks @traciwilson can you start putting in the survey intros and section intros?
@mmirasol I've finished adding the intros for the first 3 OCA surveys. Now I am adding the new intros to the non-OCA surveys (community capacity, managers, teams, and partnerships).
@traciwilson these two questions should probably be combined too.
@traciwilson @gilliankerr combined.
@traciwilson i think these questions should be combined.
@traciwilson @gilliankerr combined.
hi @traciwilson great job with the intros. can you help me with:
ex. https://sites.google.com/view/orgwise/understanding-your-results/governance-and-legal-structure?authuser=0 which says,
Understanding your results In future phases, OrgWise will be able to score your results and provide a summary. For now, you look at your own responses and save them in pdf by clicking the link below:
Print your results - do not include this text, instead turn on the survey setting for "Participants may print answers".
Following is some additional information about this module and how to use the results. Achieving your mission and vision The results of this survey can be compared against benchmark scores of strong capacity. Each question is potentially scored from 1 to 4: low capacity (1), basic capacity (2), moderate capacity (3), and strong capacity (4). For any area where your organization had a low score, review the question answers to identify the next level of capacity that you can aim for. This will assist with prioritization and action planning.
Once the items above are done, we're done with the first 3 surveys. We will do the same with the last 4.
@traciwilson The options don't sound right for this question.
Perhaps, these should be: Salaries and wages are consistently late, inaccurate or incomplete Salaries and wages are usually late, inaccurate or incomplete Salaries and wages are usually on time, accurate and complete Salaries and wages are consistently on time, accurate and complete
@traciwilson @gilliankerr corrected.
@traciwilson The answers don't sound right for this question.
@traciwilson @gilliankerr corrected.
@traciwilson the answers don't answer the questions being asked.
@traciwilson @gilliankerr converted to open-text question.
@traciwilson even this one doesn't sound right.
will put in, Employee benefits are not documented in a policy manual or handbook given to all employees Employee benefits are documented in a policy manual or handbook given to all employees
@traciwilson @gilliankerr corrected.
@traciwilson These two questions should be combined too.
@traciwilson @gilliankerr combined.
@traciwilson I think these two questions should be combined.
also this one,
@traciwilson @gilliankerr combined.
These questions should be combined too.
@traciwilson @gilliankerr combined.
@traciwilson @gilliankerr these questions do not match their answers. Please advise.
@traciwilson as discussed, changed question text to: Does the organization refer clients to other appropriate service providers? Does the organization have adequate physical space to meet with individual clients and groups?
and added new open text questions for: What are the organizations strengths in stakeholder involvement? How does it build on stakeholder involvement? What are the organizations weaknesses in stakeholder involvement? How can it improve stakeholder involvement?
@gilliankerr these questions would look better as likert scale, will convert them after i split the last 3.
@gilliankerr @traciwilson converted to likert scale.
Hi @traciwilson please put in the intros and end page settings for these surveys,
Program management http://logicaloutcomes.net/forms/index.php/admin/survey/sa/view/surveyid/223922 test link: http://logicaloutcomes.net/forms/index.php/223922?newtest=Y
Project performance management http://logicaloutcomes.net/forms/index.php/admin/survey/sa/view/surveyid/193817 test link: http://logicaloutcomes.net/forms/index.php/193817?newtest=Y
@mmirasol I have completed the intros and end page settings for these two, as well as organizational management and sustainability.
@traciwilson thank you.
Here is a list of questions that I've removed from Program management module, Donor compliance subsection:
Deleted: Has the organization received a direct award from USAID?
Modified: Has the organization received a direct award from donors with complex compliance requirements?. Modified answers to remove mention of USAID.
Modified: Does the head office consistently comply with complex requirements of donors? - removed USAID from question and answers.
Modified: Does the organization have a good understanding of any relevant cost principles? [was USG cost principles]
(additional USAID-specific questions were changed to relevance = 0 so they would not appear in the survey)
@gilliankerr here are links of our 11 modules.
Effective managers http://logicaloutcomes.net/forms/index.php/154199?newtest=Y
Effective teams http://logicaloutcomes.net/forms/index.php/837344?newtest=Y
Working in partnership with other organizations http://logicaloutcomes.net/forms/index.php/484266?newtest=Y
Building community capacity http://logicaloutcomes.net/forms/index.php/577829?newtest=Y
Governance and legal structure http://logicaloutcomes.net/forms/index.php/235289?newtest=Y
Financial management and internal control systems http://logicaloutcomes.net/forms/index.php/345769?newtest=Y
Administration and procurement systems http://logicaloutcomes.net/forms/index.php/154199?newtest=Y
Human resource systems http://logicaloutcomes.net/forms/index.php/386626?newtest=Y
Program management http://logicaloutcomes.net/forms/index.php/223922?newtest=Y
Project performance management http://logicaloutcomes.net/forms/index.php/193817?newtest=Y
Organizational management and sustainability http://logicaloutcomes.net/forms/index.php/114118?newtest=Y
[ ] There are many obvious typos in the Organizational Capacity Assessment (OCA): http://logicaloutcomes.net/forms/index.php/489512 Can you correct most of them in csv and then re-upload the content? If you need to refer to the original content, here's the material - https://usaidlearninglab.org/library/organizational-capacity-assessment
[ ] The OCA should be split into seven surveys, one for each of 7 domains: governance, finance, administration, human resources, program management, project performance management, and sustainability. Each domain should have its own welcome page (with the nice image). For now, use the same content on each section that is now on the full survey. I will send individual content for each module in a few days. (If you have other suggestions for how to split them I would be interested.)
[ ] The questions should be numbered. Possibly each section should have a table of contents - is that possible? Does it look okay?
[ ] The intro section should have better fonts for the headings. E.g., in the following screenshot, the heading and subsection headings are underlined (which should never happen on the web unless they are hyperlinks) and the font seems too small. I will be providing content soon.
Your design comments are very welcome.