Confidential (private) issues on public repo #3217

Open dllud opened 6 years ago

Feature request: allow confidential issues on public repositories. As reference, check GitLab's confidential issues.

This is a most for organizations that handle coordinated disclosure and patching of security vulnerabilities on public open-source projects. It would allow discussions to go on between the early disclosure and the full disclosure dates. The issue could later be made public at the date of full disclosure.

This is the opposite of #639 but perhaps can be handled in the same work package as it will require similar constructs.

Should be fairly easy to integrate, we'd just need to add a private flag to the db and then ofc check for it in UI + API.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs during the next 2 weeks. Thank you for your contributions.

What? You should revise this bot. An issue should only be closed when it gets solved or it is no longer relevant. This issue, which is actually a feature request, reached none of those stages.

@dllud It had no activity since march last year, so ...

The bot also works as a reminder for everyone who is on the issue.

So, what? Could you please explain the rationale?

If you wish for the bot to work as a reminder then disable it from closing issues. As it is, it just forces issue reporters to make pointless comments. As you can see, the discussion we are having is completely irrelevant to the issue at hand.

Back on the topic of the issue, what would be a good bounty to get this one started?

We are willing to pay to get this feature added to Gitea, as all our company discussions happen on a public repository right now, but we would like to keep a few topics confidential — employee review, vulnerability disclosure and whatnot.

@jamesponddotco I don't think that it is planned that gitea manage to get paid for feature. But you can use bounty source like in : https://github.com/go-gitea/gitea/issues/2725 This will attract people, even daily gitea maintainer, to move the issue on top of their list.

@jamesponddotco If code is accessible by others it seems difficult to tighten security on code changes.

Anyway, I think the quickest way to solve this is to allow private forks from public forks (currently not allowed) and make your work on that fork. If you wish, I think you can create a fork and update the is_private column on the database for that repo (Gitea would force it to be public if the original is public as well).

@sapk Oh, I know, I am just not sure what sort of amount would be considered fair for something like this. $50? $100? More?

@guillep2k We are looking into keeping specific issues confidential, not code — all our code is open source. We discuss everything in public, kinda like what Gitlab does, but specific individual discussions need to be kept private for privacy or security reasons.

Right now we have a public repository called "company" for public discussions and a private repository called "company-private" for private discussions. We would prefer to have only the "company" repository, so we can maybe open some of these private issues to the public later on.

@jamesponddotco How do you imagine this feature at work? Will the issue be marked as "restricted" on creation? Will you have a specific team that has access to these restricted issues (this would only be possible if the project owner is an organization)? What about PRs? How do you plan to hide the work in progress (if you intend to)?

The thing is there's plenty of outlets in Gitea that can easily expose an issue to users (e.g. statistics, listings by label, author, milestone, etc., mail notifications, webhooks, API, migration, etc.). There's also the issue of how to deal with assignments, for example, or @mentions. Then you have the obvious "holes" in the issue numbering that point out to something fishy going on. It all depends on your level of paranoia, of course.

@guillep2k Honestly? No idea. Whatever take this would probably need to look at how Gitlab does it.

In the ideal world — at least to me —, there should be a checkbox when creating an issue that mark it as visible only to the creator of the issue and the members of the organisation.

This is how it looks in Gitlab:

Someone from outside would be able to open an issue to report a vulnerability, for example, and everyone from the organisation and the person who created the issue would participate in the discussion. Confidential issues should probably not be available over the API.

For the amount, I don't exactly know. I suggest you based yourself on current and past bounty. For the issue that I cited it was 200$. The main contributor post 100$ and added 95$ one month later and someone added 5$. If you look at current issue they should be around 100$ but they are stuck for a little time. If your need is urgent, I would say start at least at 100$. The issue is basic at first glance but will need refactor of other part of code like doing a private fork of a public repo (for PR/MR like gitlab). I will finish by saying thanks to propose a bounty this help contributor and even if it is tricky to put a number/price/time it is always really appreciated.

I'd say we could add private issues first, and then private PRs later on since that would require a lot more work - as @guillep2k already pointed out.

If someone would like to create a PR, he has to consider something as below.

[ ] An issue could be changed to private or to public.
[ ] Repository issues list and global issues list should ignore the private issues if they have no permissions.
[ ] APIs about issues list / single issue.
[ ] References and dependencies: When you reference an public issue on a private issue, it should not be displayed on public issue's comment if user has no permission.
[ ] Notifications: only send information to special users but not all repository watcher when comment on a private issue.

Just my 5 cents: we'd love a feature where issues in a special repository are being seen only by issue starter and maintainers. Scenario exactly as mentioned: vulnerabilities reporting. We love to get them reported but want to discuss them internally and only with the reporter before we patch the issue.

At the moment we achieved the same using templates-hack but it's a pain to maintain our changes when updating gitea.

If the above-mentioned will be done I am ready to add another $100 to the bounty.

Bounty posted: https://www.bountysource.com/issues/52815258-confidential-private-issues-on-public-repo

Just adding another use case that would be covered by this: a simple user support/help desk system. A service would have a public repo for end user support, and the user could decide if they want to have their request be public or confidential. And the support team can have access to the confidential issues.

I'll take it - If I have enouth time It perhaps get ready before feature freeze

Thanks for starting this. Any progress so far? It would be really useful to track security issues on public repositories, and to have helpdesk-like system, where not all issues could be public due to personal information disclosure and so on.

still planed, didnt had time to work on it at the moment.

I absolutely appreciate and support this idea. Should be fairly simple to implement and will have huge impact on open source projects that need to keep some critical tickets confidential (for security reasons, surprises, potential misuses, etc).

Should be fairly simple to implement

PRs are welcome in that case.

One thing I notice when talking about private issues is an additional security aspect: Such an issue will most likely get its ID from the public ID pool. Even if it will not get displayed, users will know that there is an issue that is not being displayed here because they will receive either a 404 - Not Found or a 401 - Forbidden, depending on the implementation. And since issues cannot be permanently deleted without access to the database, that is a pretty clear indicator. Are we fine with notifying users that such issues exist? In favor of this approach are the following two aspects:

there is most likely nothing to fear, if the user knows such an issue exists
Its way easier to implement this way

Against it speaks that if any part of the application is not robust enough to protect against accidental intrusion (see i.e. above, most likely there are even more - I can think spontaneously of linking to the issue in PRs, Milestones or Projects), then you have a huge security leak where hackers can see security issues before they are publicly known, and most likely before they are fixed. Even worse: How can you handle PRs referencing to fix this issue? Do you simply hide a link to a private issue from the user (i.e. in a fixes/closes #test), do you show it, or how do you handle that?

Depends on deleting issue or the index could be non-continuous.

Hi there, I was just searching for this feature and stumbled upon this thread. I did not read all of it but concerning the last comment by @delvh I think there is no harm in being open about private issues. In fact for user without access on the lowest level the issue may present itself as something like this:

ID: "visible"
Title: "Private issue"
Only Comment: "This issue is  for <minimum_access_level> only. Currently, you have no permission to its content. Please log in or request permission if you feel like you should be allowed to see this."
Permission: readonly

where may be different in case you can choose to make an issue private to Owner/Maintainer/Developer/Reporter... etc.

I dont understand what this comment is about, just an info: this is still on my todo list for features ... (1. rss, 2. this)

Just adding another use case that would be covered by this: a simple user support/help desk system. [...]

This is also the scenario I'm interested in. In this setting, it could also be cool to allow only one kind of issue on a repo (for privacy reasons, I may want to ensure that my users never write public issues). It could also be nice to be able to specify at the time of writing the issue who (among the developers) has the right to see this issue, for instance by specifying a list of team. This may be interesting on security issues (you may not want all the dev to learn about the issue).

well confidential issues should only be accessible if you are the A. creator; B. a Repo Admin; C. Assigned to this repo

well confidential issues should only be accessible if you are the A. creator; B. a Repo Admin; C. Assigned to this repo

D In future, creator or admin could invite others to view this issue.

I guess it makes sense for confidential issues. The application I have in mind is maybe a bit outside the scope of gitea, as I was also thinking to use gitea as a kind of project manager in the administration (issues with notifications and history, edition, wiki, being able to reference issues from other issues, assignee, kanboard, milestones, reactions, labels, time tracking... are many things that could certainly be beneficial there as well and could replace or complete emails). In this scenario, some issues should be shared only to parts of the administration : for instance, I guess not everybody in the administration should have access to discussions concerning the budget, and each team may have internal issues they don't want to share by default with others. Also, when external users want to fill tickets about some issues/questions, they often want the ticket to be private, and typically want to notify only a part of the team. That's why I think it would be really cool to provide a way to notify, restrict and invite different kinds of members/teams in the issue (the default configuration may be automatically configured by the issues templates).

we dont need an invite feature - just add the user as assignee and it's good to go ... if implemented that way

@tobiasBora what you try to do can already be done by teams and orgs ...

This would be really useful for reporting security issues, so a user can file an issue as usual but in a way where it's not necessarily paired with a maybe-not-so-responsible disclosure to everyone. What's the current stance on this? So far from above it reads like it won't be added, which would be a shame.

So far from above it reads like it won't be added, which would be a shame.

no there is just not enough time to implement atm and it's nothing one should contribute if he is not familiar with the codebase already

One thing I notice when talking about private issues is an additional security aspect: Such an issue will most likely get its ID from the public ID pool. Even if it will not get displayed, users will know that there is an issue that is not being displayed here because they will receive either a 404 - Not Found or a 401 - Forbidden, depending on the implementation. And since issues cannot be permanently deleted without access to the database, that is a pretty clear indicator. Are we fine with notifying users that such issues exist? In favor of this approach are the following two aspects:

there is most likely nothing to fear, if the user knows such an issue exists

Its way easier to implement this way

Against it speaks that if any part of the application is not robust enough to protect against accidental intrusion (see i.e. above, most likely there are even more - I can think spontaneously of linking to the issue in PRs, Milestones or Projects), then you have a huge security leak where hackers can see security issues before they are publicly known, and most likely before they are fixed. Even worse: How can you handle PRs referencing to fix this issue? Do you simply hide a link to a private issue from the user (i.e. in a fixes/closes #test), do you show it, or how do you handle that?

Since issue could be deleted today, the issue is gone. It maybe deleted.

Yep, at the time of writing I didn't think such a thing would ever exist. That's now of course completely outdated.

Since issue could be deleted today, the issue is gone. It maybe deleted.

Well… someone that really wants to attack softwares this way could create a bot that automatically logs all newly created issues, so that it can see if the issue was a really deleted or actually exists (I admit that such users would need to pool quite frequently the repository… but when it comes to security people can do crazy things sometimes). Or if an attacker knows that users typically prefer to close an issue than deleting it (maybe because the UI to delete an issue is hard to find), then they can safely assume that a missing issue is a security issue. It's maybe possible to randomly skip ID to mitigate this attack… but it's still not perfectly secure.

A solution that should be perfectly secure (and that avoids creating a new table) is to allow security issues to only be, say, multiple of 100 (i.e. ending with two zeros), and normal issues should not be multiples of 100. In practice you could create two counters, one to maintain the last ID of a normal issue, one to maintain the last ID of a security issue, skipping the forbidden numbers. As far as I see, this should not cause any backward compatibility issue, as skipping an issue is more or less like adding+directly removing an issue.

Warning: you still want to make sure to display the same error message when an issue does not exist and when you don't have the right to see an issue… or otherwise you could directly count the security issues of a repository (this is also true for any approach proposed so far).

I don't think just knowing a potential security issue exists should be considered a problem. Often, that is even officially announced in advance sometimes days before the patch drops so I don't think it's worth hiding with a lot of effort. (As long as the title and contents of the issue remain private.)

Maybe, I don't have strong opinion on that. If it significantly eases the development of the option not to consider that case, then it might be worth doing it without. But in any case it's something to keep in mind.

Whatever it's deleted or private, it will return 404 if they are visited by a person with no permission.

My use-case as a maintainer for a Private issue would be, to set an open issue to private to be able to ask the issue starter to add logfiles or config files that may contain GDPR-relevant information or credentials. Currently we switch to email for that situations, that usually is not comprehensible anymore for the other maintainers. Sometimes, issue openers aren't aware of such Information is contained in their logs or screenshots, so we would have a quick win to set an issue private instead of deleting or truncate such information.

Great to see that there's already a feature request for marking pull requests as confidential when fixing security vulnerabilities, and even better that the maintainers have expressed interest in implementing it.

I was recently reminded of the importance of such a feature after watching an informative video by Brodie Robertson, where he discusses the internal security vulnerability issues faced by the curl developers, as detailed in Daniel Stenberg's blog post on pre-notification dilemmas. It would be great to see this feature come to fruition, and I hope that it can be implemented soon to better support open source projects in managing security vulnerabilities.

go-gitea / gitea

Confidential (private) issues on public repo #3217