Openness Indicator for OER services

[trugwaldsaenger]

Story: As a user I want to recognize at a glance how open the materials are, which are provided by a special service, so that I can restrict my search to the level of openess appropriate to my personal needs.

Acceptance Criteria:

• use of a colour based "traffic lights" system shown prominently on the service profile page
• proposal for colour scheme:
  • green = cc by, cc by-sa
  • yello = cc by-nc, cc by-sa-nc
  • orange = cc by-nd, cc by-nc-nd
  • red = no license information
• per default only green and yellow services will be displayed on the map (in the search result list)

[trugwaldsaenger]

This is a rather complex story, It's not easy to define the different levels of openess and its probably even more difficult to put it into praxis. Quite some questions, still have to be answered: Especially it seems to unclear, when a repository is e.g. "green"?

• If all included contents are CC BY or CC BY-SA?
• if some contents are CC BY or CC BY-SA?
• if a significant number of contens is CC BY or CC BY-SA?
• if it has a policy which says that contents have to be CC BY or CC BY-SA?

We should look for a solution which is practicable. Probably nobody will have the time to count documents and calculate...

[trugwaldsaenger]

@acka47 @drrobertfarrow Would you agree, that we need something like this and that we should develop it within phase II of the project?

[trugwaldsaenger]

Hi Adrian, could you please comment on this? Thanks a lot!

[acka47]

Besides the question already addressed whether and how the content ins CC-licensed there are a lot more aspects of openness with regard to a repository or "referatory", e.g.:

• Is it open to everyone or only for registered users?
• Is it open for everybody to contribute?
• Is it build using free software?
• Are the contents offered in an open file format?

However, as we need to start with an actionable openness indicator, I agree to focus on content licensing in the first run. I suggest the following scheme:

• Green: all resources are licensed under an open (as to the open definition, excluding NC) license.
• Yellow: Some or all resources are licensed under CC-BY, CC-BY-SA, CC-BY-NC, CC-BY-NC-SA
• Red: All resources are licensed with ND-license or have no license indication at all.
• No colour: licensing status is unclear.

[trugwaldsaenger]

Thanks @acka47 for your comment! I like the scheme you are proposing, since it seems pretty handy! Are there any examples available, which we could look at?

[drrobertfarrow]

I think it's a nice idea but it's difficult to measure 'openness'. I agree that it makes sense to focus on the licensing as @acka47 suggests and I think the proposed scheme is reasonable.

The people I would suggest talking to about putting together information about repositories would be Javier Atenas and Leo Havemann, who put together the repository map at http://www.zeemaps.com/view?group=562530

[trugwaldsaenger]

@buschfeld: I guess this is quite an interesting design challenge. I can imagin at least two options:

1. having concentric circles. When red only the inner circle is marked red, when green, all the circles are green.
2. having something like a thermometer Though we will concentrate only on the licenses first, we should keep in mind, that we might extend the indicator to factual openess (registration?, open formats?) in the future. So the indicator design should be scalable to add another dimension in the future.

[drrobertfarrow]

I am a little unsure about a 'traffic light' system that could show 'red' for a resource that simply does not allow derivatives. It's still an open resource if people can access it so the 'red' could give the wrong impression. I actually think the original proposal makes more sense...

proposal for colour scheme:

• green = cc by, cc by-sa
• yellow = cc by-nc, cc by-sa-nc
• orange = cc by-nd, cc by-nc-nd
• red = no license information

I'm not sure I agree that anything below yellow should be excluded - that would exclude from the representation all OA materials that do not allow remix. But they are still OER!

I wonder whether this is an area where we might benefit from inviting opinion from the wider OER community as to what information they would find most useful and then design the visualization from there. This would leave us less vulnerable to the charge that we are imposing some particular idea of openness.

[trugwaldsaenger]

Hi @drrobertfarrow , thanks very much for you comment - I think it is a good idea to include the community on this! We should try to write a blog post next week.

I also agree that marking ND licensed material as red might look a bit offensive. Nevertheless I would argue that these are no OER according to both the Hewlett and the UNESCO definition of OER. Maybe we could use some red, which does not look so agressive...

[trugwaldsaenger]

Another metaphor which is quite common in the OER World is a door, which can be closed, partly open or open...

[literarymachine]

I like the idea of the door! But why are you closing this issue, are we done?

[trugwaldsaenger]

Ups, this was accidentally - we are talking about OPENNESS here, so reopened!

[trugwaldsaenger]

Github uses also red and green circles of open/close distinctions...

[literarymachine]

Github uses also red and green circles of open/close distinctions...

True, but this is actually more of a workflow indicator, similar to what @buschfeld meant earlier.

[trugwaldsaenger]

@pgogy recommended us to have a look at Open Attribute: http://openattribute.com/ It`s a nice tool which helps to become aware of CC licensed content and giving attribution. But it seems to be different from our approach, since we aim at giving a simple visualization which indicates to users how open a complete repository is.

[trugwaldsaenger]

The portal of the Siemensstiftung https://medienportal.siemens-stiftung.org/portal/main.php?todo=metadata_search&crits[licence]=1&crits[medialang]=4949 is an interesting example. It contains primarily proprietary licensed materials. In order to access these, you have to register. But it also contains a pool of CC BY-SA licensed materials. Using @acka47 scheme described above I guess it should be yellow: "Yellow: Some or all resources are licensed under CC-BY, CC-BY-SA, CC-BY-NC, CC-BY-NC-SA" since not all materials in the repository are open licensed. Nevertheless I would argue that it should be green, because the part referring to OER is perfectly open. The indicator should not "punish" an service for not providing open materials exclusively.

[trugwaldsaenger]

The BCOER Librarian (http://bccampus.ca/2015/07/02/bc-librarians-collaborate-to-address-growth-and-navigation-of-open-education-resources/) developed an OERRepository Rubric, which provides indicators of OER repositories (OERR).

http://open.bccampus.ca/files/2014/07/OERR-Rubric.pdf

Its quite interesting to look at. The question of registration is dealt with in the section "User-friendliness". The section "Licensing and Permission" deals with similar questions like our discussion here. They propose a three level system. But is seems to be less strict, since you can get the highest ranking even for CC BY-NC licensed materials.

[trugwaldsaenger]

Maybe we should take an user-perspective, when defining our levels: green: (As a user) I`m allowed to use all contents available in the repository as long as I make sure that I give attribution properly. yellow: In case that there is no explicit policy I must check every single item if I can use it for my purpose. It also might be necessary for me to register. red: I´m not allowed to revise and remix (=ND), also I might not even redistribute the included material (=no license).

[trugwaldsaenger]

[trugwaldsaenger]

Mmmh, maybe the red should be orange, since ressources, which are not open at all will not be in the repository...

[acka47]

We decided at the meeting today to add the needed data manually to the services, i.e. some property like "openness" with a value from0, 1, 2/"open", "partly open", "non-open". We will have to:

• adjust the JSON schema
• add the information to the service descriptions

Will open issues for these tasks.

[pgogy]

On 2015-09-02 08:43, Adrian Pohl wrote:

We decided at the meeting today to add the needed data manually to the services, i.e. some property like "openness" with a value from0, 1, 2 or "open", "partly open", "non-open". We will have to:

• adjust the JSON schema
• add the information to the service descriptions

Will open issues for these tasks.

Reply to this email directly or view it on GitHub [1].

*

Links:

[1] https://github.com/hbz/oerworldmap/issues/237#issuecomment-137062134

I think the response to Open / Non-Open is / will lead to opinions from certain OER people. It might be useful to crowdsource how you define the value?

Pgogy Webstuff pgogywebstuff.com

[trugwaldsaenger]

@pgogy We are definetly going to write a blogpost on this question! I`m curious about the response...

[pgogy]

Make it obvious that it is open to being changed, or allow other schemas

to be used?

Pgogy Webstuff pgogywebstuff.com

[trugwaldsaenger]

I`m not completely sure if I understand your question right. We will make clear that we are looking for feedback. If anybody will propose another schema we will be happy to look at it and see if it is better than our actual scheme ("open", "partly open", "hardly open"). What exactly do you mean by "allow other schemas to be used"?

[pgogy]

Take the pic above (with the licenses on) Perhaps allow for the licenses to be checkboxes and if I untick one of them, The map colours change and so on

[drrobertfarrow]

My thinking here was that we identify, say, six to ten criteria of openness. E.g.

• Is it licensed?
• CC-BY?
• ND?
• SA?
• Freely available online? (Open Access)
• etc.

(I'm not saying that these need to be the criteria.)

These can be identified with a Yes/No response and so we end up with a score out of 6 and this indicates a measure of openness. This could be presented as a number, or as an indicator which is more darkly shaded according to value. If users want to know why a recourse got a particular score they should be able to see the breakdown.

This allows us to include resources which, say, are not openly licensed at all but available freely online.

It also means that we provide meaningful information on the 'openness' of a resource but don't slip into what could be considered judgements about the rightness or wrongness of a particular licensing option.

We could invite opinion on what the important indicators should be (maybe even offer a poll) - it would then be protected from some potential criticism.

[pgogy]

Relate it to Tim Berners-Lee's five stars?

[literarymachine]

Examining the data model of the International Rights Statements Working Group, I believe that this approach can be adopted for our openness indicator. We could model our openness scheme as a series of skos:collection and then add concrete licenses to those collections:

@prefix skos: <http://www.w3.org/2004/02/skos/core#> .

<open> a skos:Collection ;
  skos:prefLabel "Licenses for OER that are considered completely open."@en ;
  skos:member <https://creativecommons.org/licenses/by/3.0> ;
  skos:member <https://creativecommons.org/licenses/by-sa/3.0/> .

<partly-open> a skos:Collection ;
  skos:prefLabel "Licenses for OER that are considered partly open"@en ;
  skos:member <https://creativecommons.org/licenses/by-nc/3.0/> ;
  skos:member <https://creativecommons.org/licenses/by-nc-sa/3.0/> .

<non-open> a skos:Collection ;
  skos:prefLabel "Licenses for OER that are considered closed"@en ;
  skos:member <https://creativecommons.org/licenses/by-nd/3.0/> .

I am unsure if there are any semantic problems when considering a CC license document a skos:Concept.

[acka47]

Examining the data model of the International Rights Statements Working Group, I believe that this approach can be adopted for our openness indicator.

I had a similar thought. We could also replace the license information by using the rightsstatement vocabulary. That would free us from creating a controlled vocabulary of licenses, see #358 .

[trugwaldsaenger]

Sound interesting. More Information can be found here: https://docs.google.com/document/d/1x10JsIfi8Y74pgJJEAqMtyO5iYp0p6DO5DrOZK-5umY/edit We should discuss this!

[pgogy]

http://sparc.arl.org/news/sparc-launches-open-access-evaluation-tool

[literarymachine]

Thanks Pat! As for the UI used to show the openness of OER repositories, the scale to the left and right of the Open Access Spectrum referenced in that post is interesting:

[trugwaldsaenger]

@drrobertfarrow contributed this link: http://sparceurope.org/how-open-is-our-research-a-checklist-for-institutions/

[acka47]

At #OERde16 there was a barcamp session offered by @renepickhardt named "Rate this OER" that revolved around very similar questions. See the results (German) in the etherpad at http://pad.o-e-r.de/p/oercamp16-Atelier-1030h

[trugwaldsaenger]

The flyer "Some rights reserved" includes something similar to our initial ideas:

[renepickhardt]

true but there should be much more dimensions included. e.g. is the data available in an open format. is there an API one can query. Are there data remixing and editing tools on the web site and so on (:

the pdf is a nice flyer to hand to co-workers (:

best Rene

On Thu, Mar 10, 2016 at 8:06 PM, Jan Neumann notifications@github.com wrote:

The flyer "Some rights reserved http://www.pittstate.edu/dotAsset/f9c1f516-225a-40f2-9274-c9b7e4bf29e0.pdf" includes something similar to our initial ideas: [image: gauge of restrictiveness] https://cloud.githubusercontent.com/assets/6639965/13681059/a22fd1d8-e6fb-11e5-82e2-f59014dac7e9.JPG

— Reply to this email directly or view it on GitHub https://github.com/hbz/oerworldmap/issues/237#issuecomment-195003777.

www.rene-pickhardt.de http://www.beijing-china-blog.com/

Skype: rene.pickhardt

mobile: +49 (0)176 5762 3618 office: +49 (0) 261 / 287 2765 fax: +49 (0) 261 / 287 100 2765

[trugwaldsaenger]

@renepickhardt: You are absolutely right about this. I just wanted to document a similar approach. Our own model can be found here: https://docs.google.com/spreadsheets/d/1qjDUi9GvibnveOqj5Oo6ijx9m6FvukIxkGA1-ckXUJg/edit#gid=1082787439

I compared it to the notes of your barcamp session (http://pad.o-e-r.de/p/oercamp16-Atelier-1030h). I do not understand all point, but all in all it seems as if we are thinking very much in the same direction! I especially like that you point out that the licensing has to be easy to discover (e.g importance of usability). There is only limited value, if you use very open licenses, if the user has to search them.

But your approach seems to be slightly different, since you were looking for individual OER ("rate-this-oer"). We are looking at services (e.g. repositories or collections). This can make things a bit more complicated, since many repositories include resources with different licenses....

[trugwaldsaenger]

I just refined the model a bit and included an example: https://docs.google.com/spreadsheets/d/1qjDUi9GvibnveOqj5Oo6ijx9m6FvukIxkGA1-ckXUJg/edit#gid=1082787439 I´m asking myself if the importence of licenses is taken into account adequatly. Lets have a look at this theoretical example: An repository includes only a few contents which are licensed CC BY-NC and no content with more open licenses. But it fulfills all other requirements. This case it would achieve a score of 87,5 which is pretty high.

[trugwaldsaenger]

We should have a look at Leonhard Dobuschs et al Offenheitsindex: http://momentum-kongress.org/cms/uploads/PAPER_Dobusch_Palmetshofer_Offenheit-ranken-Der-digitale-Offenheitsindex.pdf

[acka47]

I just got aware of th ALMS framework by David Wiley (via http://www.hoou.de/p/2016/04/28/entwicklung-einer-offenen-technischen-infrastruktur-fuer-hoou-lernarrangements-an-der-tuhh/). We should definitely take this into account for the opennness indicator.

[trugwaldsaenger]

Good point, thanks for sharing! Let`s do a very short test, comparing ALMS framework with the actual version of the Openess Indicator (OI):

1. Access to Editing Tools: Is the open content published in a format that can only be revised or remixed using tools that are extremely expensive (e.g., 3DS MAX )? Is the open content published in an exotic format that can only be revised or remixed using tools that run on an obscure or discontinued platform (e.g., OS/2)? Is the open content published in a format that can be revised or remixed using tools that are freely available and run on all major platforms (e.g., OpenOffice)?

The OI does not look at the tools, which are used to remix open licensed content. Right now we distinguish between Open Contents which "enable remix" and those which "impend remix". This is similar, but not identical.

1. Level of Expertise Required: Is the open content published in a format that requires a significant amount technical expertise to revise or remix (e.g., Blender)? Is the open content published in a format that requires a minimum level of technical expertise to revise or remix (e.g., Word)?

This is a very interesting point as well. In the OI we arguably do not focus on remix so much. We might stress this aspect more in the next version. Nevertheless, I`m not sure if it will be possible to add this point for practical reasons.

1. Meaningfully Editable: Is the open content published in a manner that makes its content essentially impossible to revise or remix (e.g., a scanned image of a handwritten document)? Is the open content published in a manner making its content easy to revise or remix (e.g., a text file)?

This looks closely related to 1 & 2. If I do have tools to remix available and if editing does not require much expertise, than a content is "meaningfully editabe".

Self-Sourced: It the format preferred for consuming the open content the same format preferred for revising or remixing the open content (e.g., HTML)? Is the format preferred for consuming the open content different from the format preferred for revising or remixing the open content (e.g. Flash FLA vs SWF)?

This seems to address the question, if a content is available in different formats. We have not included this in the OI as well.

[drrobertfarrow]

At the team meeting in Milton Keynes we returned to this discussion briefly. The idea of using radar charts to represent the openness indicator was mentioned. Radar charts allow you to compare data across several vectors.

https://en.wikipedia.org/wiki/Radar_chart

[trugwaldsaenger]

I have created a Version 0.3 of the Openness indicator: https://docs.google.com/spreadsheets/d/1qjDUi9GvibnveOqj5Oo6ijx9m6FvukIxkGA1-ckXUJg/edit#gid=1400185386 @acka47: Could you please have a look at it? I think especially collum F needs refinemend. Attention: I did not copy the comments from Version 02 to Version 03!

[acka47]

Re. open formats: For the sake of making this actionable, I'd rather refrain from separating open formats that impede remix vs. open formats that don't. Reasons:

• This is – as always – rather a question of nuances and preferences than of "yes" and "no". While docx might be considered more open than pdf by lots of people other might consider plain text to be more open than docx or odt because I can open and edit it with any editor.
• To determine whether a format impedes remic you not only have to look at the format but at its use. Photos ad .png file are a good fit while scanned texts as png might be seen to impede remix.

My proposal: Make this column one-dimensional by asking "Open format: yes/no" and referring to an openly editable list on the web like https://en.wikipedia.org/wiki/Category:Open_formats for answering the question.

[karindr]

See also: The Continuum of Openness http://www.oel.edu.au/resource/