search

todogroup / osposurvey

Open Source Programs (OSPO) Survey
https://todogroup.org
Creative Commons Attribution Share Alike 4.0 International
71 stars 26 forks source link

Questions / Responses to include in 2023 Survey #115

Closed anajsana closed 1 year ago

anajsana commented 1 year ago

Opening this issue to share a compilation of questions and answers derived from community input gathered in previous community calls and open discussions, as well as reflecting current OSPO trends. The questionnaire template that is being reviewed can be found here.

Questionnaire Review

To a certain degree, any organization calling itself an OSPO (or similar open source initiative) likely indicates that the organization has reached a maturity stage and critical mass, where OSPOs share key characteristics:

change "Employees" to "workforce", "workers", "staff" or "personnel" to be inclusive with the formation of OSPOs in the public sector

Q.9 - How many employees are part of your open source program or initiative? (select one)

Include (either physical or virtual)

Q.10 Where is the open source program or initiative located within the organization? If the effort is informal, answer based on who the primary organizers report to. (select one)

Answers are mainly focused on the corporate world. Since OSPOs are being formed in other sectors (universities, governments), below is a suggestion to add new fields (in bold).

Q.11 Does your open source program have an InnerSource team or strategy to drive open source culture within the organization? (select one)

Include or implement principles: Does your open source program have an InnerSource team, strategy, or implement principles to drive open source culture within the organization? (select one)

Q.16 What are the areas where your organization has most benefited from the open source program or initiative? (select all that apply)

Q.17 What are the ways your open source program or initiative quantifies success? (select all that apply)

Update the possible answers to include those that the new study on the value of the OSPO KPIs and measurements mentions:

Q.19 How business-critical is your open source program or initiative to the success of your engineering or product teams? (select one)

remove "business" to be inclusive. change "engineering or product teams" to "organizational goals" to cover non-corporate goals that OSPOs in other sectors may face.

Q.20 Has the open source program or initiative had a positive impact on your organization's software practices? (select one)

Reframe to include other scenarios (not only software):

NEW- Q.20 Has the open source program or initiative had a positive impact on the following scopes?

Q.24 Where will the open source program or initiative be located within the organization? If the effort is informal, answer based on who the primary organizers will report to. (select one)

Duplicate answers from Q.10:

Value of Companies and Foundations

Drop questions from 32 to 40 (both included)

License compliance and security

Include a new question:

[NEW] Does your OSPO or similar open source initiative directly address open source security issues?

[NEW] Add a new section called "Open Source and OSPO Sustainability inside the organization".

Explain what we mean by sustainability in this context:

[NEW] Does your OSPO or similar open source initiative work on OSPO Sustainability (maintaining the initiative in the long term)?

[NEW] If answered extremely or very likely, how does your OSPO communicate the work done to different organizational teams (legal, communications, HR, etc.) and their own internal team (if any)? What best practices do you follow? Are there any antipatterns (bad practices) that you would like to share?

[NEW] If answered slightly or not at all, what prevents your OSPO from working on this?

[NEW] Does your OSPO or similar open source initiative work on Open Source Sustainability (giving back to the open source community and going beyond: helping open source projects become sustainable by identifying community and project health risks and providing funding, contributions from your organization, infrastructure, etc.)? (mark all that apply)

[NEW] If answered not at all, what prevents your OSPO from working on this?

LawrenceHecht commented 1 year ago

1) Changing "employees" to workers" in one instance. In two others, changed it to "staff". 2) Q9, can be changed to "How many workers are part of your formal and/or virtual open source program or initiative?" 3) Q10 and Q24 -- what's the difference between "Department responsible for digital services", "IT" and "Computing Services"? I'm adding Faculty Services 4) Q11, done. 5) I will finish up commenting/responding soon.

LawrenceHecht commented 1 year ago

Perhaps we add a question that is asked of everyone about what they think are the areas where OSPOs in general are having the greatest positive impact on society. Also, as a side note, these 2 sets of answer choices are highly correlated, so we might be able to cut them from the survey if we are looking for space:

  1. (Faster time to market with new products) AND (Increased speed and agility in the development cycle)
  2. (Increased participation in external open source projects) AND (More influence in open source communities)
  1. Number of people in the organization who make regular, repeat contributions to the same project: I like this addition. It can be in addition to the # of contributors KPI or replace it.
  2. The reputation of open source internally (internal awareness of open source and the OSPO work): In Q16, we already ask if "More awareness of open source use and commercial dependencies" is a benefit. I suggest we change this to "Internal awareness of the OSPO's work"
  3. Organization involvement in open source community: We are already asking about volume of upstream code contribution, # of code contributors and reach in open source communities How else should this be measured?
  4. How much friction is created for the organization's developers and workers (e.g., if a request needs to be approved internally to contribute, how long does it take?) We already have the following three answer choices that captures this type of information: "Developer velocity, efficiency, and/or productivity". "Frequency of dependency updates", "Mean time to detect vulnerabilities". If you want to add something more specific and/or adjust the answer choices, we can do that.
  5. Tracking the number of active contributors, the frequency of commits, the number of maintainers, and other project health metrics, including having users and contributors from many different organizations. We can change the "Number of contributors" to "Number of active contributors." I also support adding another project health metric, like, "Number of external or third-party contributors to projects maintained by the organization"
LawrenceHecht commented 1 year ago

Which of the following ways does your OSPO or similar open source initiative address open source security issues? (Select all that apply) [Only ask if the respondent has an OSPO] -- Make decisions on how the organization's workforce can identify security risks in open source projects -- Support the implementation of best practices (e.g., internal education, guidelines, policies) within specific open source projects -- Provide advise to the team, unit or department in charge of open source security issues -- We do not directly address open source security issues

Regarding the new section: Should this be worded differently so it can be asked of respondents that do not currently have an OSPO? I'm concerned that there is little distinction between OSPO Sustainability and sustainability of the projects the organizations is involved with. Also, the answers to previous questions (responsibilities, benefits, quantifying success) overlap with both the OSPO Sustainability and Open Source Sustainability questions. There is definitely a way to rework the questions to make them more valuable. Any thoughts?

anajsana commented 1 year ago

Thanks for reviewing @LawrenceHecht 🙏 answering bellow 👍

  • Q16: I don't understand the "more equitable" choices. How does access to DPGs help the organization itself? How do OSPOs make the organization's access to open source talent more equitable? I would argue that these are societal and/or external benefits.

Here is a few literature [1, 2] that highlights how social goals act as drivers and motivators for public administrators when establishing an OSPO in the public sector. The rationale (as far as I've seen in conversations and talks) is that open source could help achieve many of the goals proposed for interoperable cross-border and cross-sector public services, digital sovereignty, talent acquisition in the public sector, and more.

When analyzing the responses from Q.16 we can filter the responses by sector to identify if a large number of public sector entities with an OSPO have (or have not) actually benefited from any of these drivers. (maybe the word "area" is not the best one to use, but "drivers"? )

Re: DPG - It's important to note that OSPOs established in universities and NGOs are concerned not just with open source software, but also with open data, open AI models, open design, open standards, open content, etc., as a means of serving the public and addressing issues of digital equity and inclusion.

[1] OSPOs in government [2] University OSPO

Perhaps we add a question that is asked of everyone about what they think are the areas where OSPOs in general are having the greatest positive impact on society. Also, as a side note, these 2 sets of answer choices are highly correlated, so we might be able to cut them from the survey if we are looking for space:

1. (Faster time to market with new products) AND (Increased speed and agility in the development cycle)

2. (Increased participation in external open source projects) AND (More influence in open source communities)

LGTM! Let's merge those options 

* Q17. We are already asking about most of the topics that you suggested. We can replace existing choices, but that will prevent a time series comparison.  See below for comments on each of proposed additions:

1. **Number of people in the organization who make regular, repeat contributions to the same project**: I like this addition. It can be in addition to the # of contributors KPI or replace it.

replace it 👍

  1. The reputation of open source internally (internal awareness of open source and the OSPO work): In Q16, we already ask if "More awareness of open source use and commercial dependencies" is a benefit. I suggest we change this to "Internal awareness of the OSPO's work"

ok! but I'd include "open source" in the sentence: Internal awareness of open source and the OSPO's work

3. **Organization involvement in open source community**: We are already asking about volume of upstream code contribution, # of code contributors and reach in open source communities How else should this be measured?

4. **How much friction is created for the organization's developers and workers (e.g., if a request needs to be approved internally to contribute, how long does it take?)** We already have the following three answer choices that captures this type of information: "Developer velocity, efficiency, and/or productivity". "Frequency of dependency updates", "Mean time to detect vulnerabilities". If you want to add something more specific and/or adjust the answer choices, we can do that.

From my understanding, the term Developer Velocity typically refers to how quickly engineering teams can ship products within a given sprint period. However, this metric may not be applicable to other areas of an organization such as legal or marketing. For example, the time it takes for legal to review licenses or marketing to approve a logo may not be directly related to the speed at which engineering teams can develop and deploy code.

I think the alternative shared above clarifies the scenario by giving a clear example and using a plain and inclusive language that does not only involves developers work but eny team that is engaging in open source internal operations and external projects.

5. **Tracking the number of active contributors, the frequency of commits, the number of maintainers, and other project health metrics, including having users and contributors from many different organizations.** We can change the "Number of contributors" to "Number of active contributors." I also support adding another project health metric, like, "Number of external or third-party contributors to projects maintained by the organization"

It's important to keep in mind that the organization may not necessarily release open source projects, but may still contribute to existing ones as a means of supporting their sustainability.

IMO, the current option in question leaves room for interpretation, as it can be read as either referring to applying project health metrics to projects released by the organization as open source or contributing to existing open source projects. However, adding "Number of external or third-party contributors to projects maintained by the organization" could give readers the impression that it only applies to organizations that releases open source projects. I'd keep it as it is instead of adding more text to it 🙂

anajsana commented 1 year ago

3. "Department responsible for digital services", "IT" and "Computing Services"?

Different branding depending on the organization but similar responsibilities @LawrenceHecht

anajsana commented 1 year ago
  • Q19: Changing the wording of the question to "How critical is your open source program or initiative to the achieving organizational goals?"

LGTM

* Q20: Making the suggested change, BUT I'm not sure we should combine "Interoperability and technology transfer". Is there another way to say this?

Let's combine!

* OK about deleting Q32-Q42

* Regarding the suggested new question for the License compliance and security section, I would make it read as follows:

Which of the following ways does your OSPO or similar open source initiative address open source security issues? (Select all that apply) [Only ask if the respondent has an OSPO] -- Make decisions on how the organization's workforce can identify security risks in open source projects -- Support the implementation of best practices (e.g., internal education, guidelines, policies) within specific open source projects -- Provide advise to the team, unit or department in charge of open source security issues -- We do not directly address open source security issues

Regarding the new section: Should this be worded differently so it can be asked of respondents that do not currently have an OSPO? I'm concerned that there is little distinction between OSPO Sustainability and sustainability of the projects the organizations is involved with. Also, the answers to previous questions (responsibilities, benefits, quantifying success) overlap with both the OSPO Sustainability and Open Source Sustainability questions. There is definitely a way to rework the questions to make them more valuable. Any thoughts?

The focus of this report is to understand the status of OSPOs, so I will keep this section with the main focus on "how OSPOs are doing". However, if we want to analyze if organizations with an OSPO are more (or less) concerned and active on the sustainability issue, I'd include the following question in the set that will show only to those people that marked "don't have an OSPO or similar Open Source initiative"

  1. [NEW] Does your organization work on Open Source Sustainability at some level (giving back to the open source community and going beyond: helping open source projects become sustainable by identifying community and project health risks and providing funding, contributions from your organization, infrastructure, etc.)? (mark all that apply)
  2. [NEW] If answered not at all, what prevents your organization from working on this?