search

data-lessons / librarycarpentry

Materials for Library Carpentry development
22 stars 8 forks source link

Pre- and post-workshop questionnaires #20

Closed jezcope closed 6 years ago

jezcope commented 7 years ago

Should we think about customising these to be more applicable to library & information professionals?

weaverbel commented 7 years ago

@jezcope @drjwbaker I am in favour of that. I think something short and simple that also addresses the specific tools in the planned workshop is worth doing.

drjwbaker commented 7 years ago

@jt14den asked after one of these recently.

drjwbaker commented 7 years ago

We didn't do this. This is something we should work on though. @jt14den: can we adapt from what you did in Pittsburgh pre-workshop?

jt14den commented 7 years ago

Here it is: https://goo.gl/7ODbcc. Have at it. I used the "nothing, little, lots, lots and lots" categories found in @drjwbaker's original Library Carpentry materials. I also liked the way @weaverbel gathered bios and desired outcomes from learners prior to the Portland workshop, so incorporated that in the survey.

We might want to ask Maneesha how the workflow works with SW/D carpentry. Once a workshop is requested there, the workshop name shows up in a drop down on both the pre-assessment and post-assessment surveys. Having one off surveys will become difficult to manage very quickly.

drjwbaker commented 7 years ago

Belatedly, thanks @jt14den

We could just have one big gdoc where one of the questions is 'Which LC are you attending?'. Then we have two options:

1) Have a multiple choice response to this question. +ive: we get consistent data. -ive: we have the manage the sheet, make sure new LC events are added to the multiple choice options. 2) Have a free text response to this question. +ive: less overhead in terms of managing the form. -ive: lack of controlled vocab for response.

I'm in favour of 2) as it should be obvious - for now - which LC people are referring to even if they offer inconsistent data.

My idea is to add a line of this to the Instructor Notes for each lesson. I can take this as an action once we agree - in principle - a process.

Thoughts?

weaverbel commented 7 years ago

Software and Data Carpentry have pre and post workshop questionnaires on Survey Monkey. Designated people have rights to add new workshops to the list and then to download and share results. More structured and less at risk than Google Docs, I'd have thought @jt14den @drjwbaker ?

drjwbaker commented 7 years ago

Don't know to be honest. Both are commercial services et cetera. I'm happy either way.

tracykteal commented 7 years ago

Having one-off surveys is really challenging and also doesn't let you compare responses across workshops. We've spent quite a bit of time designing and setting up to analyze surveys with our Deputy Director of Assessment. I think it would make sense to align efforts. Being able to do effective assessment is important in important in evaluating outcomes and seeing what could be improved, and is also something important in grants or other funding opportunities.

drjwbaker commented 7 years ago

@tracykteal Agreed. Aligning is better where possible. Are there any dependencies for us aligning? For example, do we need to be fully hooked into AMY? If so, we're going to need an interim solution until/if that happens.

tracykteal commented 7 years ago

No connection with AMY is needed. It's done through Survey Monkey. So, it does use our Survey Monkey account. Maybe @weaverbel can help us coordinate here.

drjwbaker commented 7 years ago

@tracykteal Huh. I have just presumed AMY was all powerful on all Carpentries admin. Let's make sure we use the same workflow then.

@weaverbel I can take this if you give me an insight into what the Carpentries pre-/post- look like.

jduckles commented 7 years ago

For SWC, our workshop pre/post surveys are linked to here: https://software-carpentry.org/assessment/

DCs are here: http://www.datacarpentry.org/assessment/

weaverbel commented 7 years ago

Thanks @jduckles. @jezcope I think this might be a good topic for our face to face, post-sprint catch up which I am now trying to organise. Expect an invitation !

tracykteal commented 7 years ago

@weaverbel good idea both for a community catch up call and to include this in the discussion. All our assessment data is online. This is Data Carpentry's http://www.datacarpentry.org/assessment/ An issue is likely around infrastructure and if you also want to use Survey Monkey.

drjwbaker commented 7 years ago

@weaverbel This is currently assigned to me. Are you taking this forward? If so, shall I assign to you? (not trying to shirk, just figure out who is leading on what!)

jt14den commented 7 years ago

@drjwbaker, We have a presentation for RDA that includes assessment for lib carp, so we hope to nudge this along some. We need to connect up with Dr. Jordan.

weaverbel commented 7 years ago

@drjwbaker @jt14den @tracykteal I am going to be on the move from next Wednesday morning (going to US to run workshops, then attend a two-day meeting) so am happy to assign this to someone else (James? Tim? Juliane?) unless it can wait till I am home? I am back here 16 Sept.

drjwbaker commented 7 years ago

@jt14den Dr Jordan?

weaverbel commented 7 years ago

This is Kari Jordan from Data Carpentry - she has just published a whole lot of info about assessment. Start here: http://www.datacarpentry.org/blog/reflections-on-assessment/

Belinda Weaver Community Development Lead Software and Data Carpentry e: bweaver@carpentries.org | p: +61 408 841 882 | t: @cloudaus

On Sat, Sep 2, 2017 at 1:05 AM, James Baker notifications@github.com wrote:

@jt14den https://github.com/jt14den Dr Jordan?

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/data-lessons/librarycarpentry/issues/20#issuecomment-326605062, or mute the thread https://github.com/notifications/unsubscribe-auth/ACp3mEItmESGfeWdrdnjl4z3UIQ9Lciuks5seB1QgaJpZM4NLlLh .

drjwbaker commented 7 years ago

Right. Ta.

pitviper6 commented 7 years ago

For the presentation, how about as a first step we look at the SWC and DC surveys, and evaluate them for use in an LC version (what questions are pertinent to the LC model/mission, what would need to be added)? @jt14den, does that sound like a good start?

tracykteal commented 7 years ago

That sounds great! I'll put you in touch with Kari Jordan, our assessment person, who just got through updating the Data Carpentry survey, and get you set up on SurveyMonkey. I'll send an email for introduction.

pitviper6 commented 7 years ago

Thanks!

danmichaelo commented 7 years ago

Just drafted a post-workshop evaluation for the Padova workshop:

https://goo.gl/forms/xDPNabN9izCKRhYl1

Based on https://software-carpentry.org/assessment/ , but I made it a lot shorter, since I feel like there's too many questions in the SWC survey. Did I miss some important aspect?

drjwbaker commented 7 years ago

@danmichaelo Thanks for this!

pitviper6 commented 7 years ago

I threw one together for the UCLA workshop - I went with a more lesson-based approach, which may be clunkier since workshops can vary in the lessons taught:

https://docs.google.com/forms/d/1dWawu3CNdl_1wxjKObM_kTPAGNAbIogXQrcj_YzWvpQ/edit?usp=sharing

Belinda had sent out a pre-workshop assessment, so we already had asked them about their pre-workshop comfort levels.