search

pycontribs / jira

Python Jira library. Development chat available on https://matrix.to/#/#pycontribs:matrix.org
https://jira.readthedocs.io
BSD 2-Clause "Simplified" License
1.95k stars 864 forks source link

💖 Python Jira needs CI maintenance help #896

Open ssbarnea opened 4 years ago

ssbarnea commented 4 years ago

That is a call for help on fixing JIRA CI project so we can test and merge change. I am personally against merging any change that does not pass CI but reality is that CI degrated considerably in the last years and cannot be fixed in one hour or two, probably not even an entire weekend.

If you care about its future, there are two ways you can pick

PS. Sadly but I am unable to review and PR that does not report as green.

truthbyron commented 4 years ago

Hi @ssbarnea, I believe this is failing because of Deprecation of basic authentication with passwords for Jira and Confluence APIs, given the Travis runs of my PR:

E           jira.exceptions.JIRAError: JiraError HTTP 403 url: https://pycontribs.atlassian.net/rest/auth/1/session
E               text: Basic auth with password is not allowed on this instance

I was able to create tie a test Jira account to the testing cloud instance, create an API token, and authenticate using a basic auth tuple of (email, token) and get a totally different set (and way more) errors, so I think that method works.

Is there any way we could get a dedicated token generated for the test and admin user and change to using email address instead of username for this block https://github.com/pycontribs/jira/blob/5c27f6ddafffc6110be1db4749fa67025852bcb6/tests/tests.py#L116-L134

I'm not sure how technically able to help I am, but I'm happy to research more/help if you have time to give any direction (otherwise I'll keep poking around as I can). Thanks for this project and for your time.

csullivannet commented 4 years ago

@ssbarnea I've run this as well, setting the environment variables for CI_JIRA_ADMIN, CI_JIRA_ADMIN_PASSWORD, CI_JIRA_USER, CI_JIRA_USER_PASSWORD to my own username (my e-mail) and my API token. I was able to successfully authenticate when running python -m tox locally.

Further, I disabled the check here to see whether I'm the admin: https://github.com/pycontribs/jira/blob/5c27f6ddafffc6110be1db4749fa67025852bcb6/tests/tests.py#L163-L165 This resulted in being able to make the API calls which returned 403. This shows that I was an authenticated but not authorized user. This gives me confidence that this will resolve this authentication issue.

Updating these two lines to use the token instead of password should fix the authentication issue, (although it may be possible the username must be replaced with the e-mail): https://github.com/pycontribs/jira/blob/5c27f6ddafffc6110be1db4749fa67025852bcb6/tests/tests.py#L124 https://github.com/pycontribs/jira/blob/5c27f6ddafffc6110be1db4749fa67025852bcb6/tests/tests.py#L134

Unfortunately neither I nor @truthbyron have access to the ci-admin user, so we cannot create this token. The token can be generated here when logged in as ci-admin: https://id.atlassian.com/manage/api-tokens

ssbarnea commented 4 years ago

@truthbyron @csullivannet I added both of you as side-admins on https://pycontribs.atlassian.net/ -- please use your powers with extra attention. My hope is that you will find time to fix the CI issues. Reply here if you need my help again on other aspects.

We are better off removing the hardcoded credentials from the code and replacing them with a fast fail when they are not defined. These variables are defined on Travis, so the values from code are likely not even used.

I also sent you invite to github team, which I suppose will allow you to configure travis credentials at https://travis-ci.com/pycontribs/jira/settings in case you need to update them.

Merging changes will still need review and I am more than happy to help with these once I see travis reporting green.

csullivannet commented 4 years ago

@ssbarnea Thanks, I've been able to update the build to use a new token I generated for the ci-admin/ci-user users. This has fixed authentication. It does appear that some tests may now be legitimately broken, but I'll need to investigate further to confirm. A preliminary glance shows almost all the errors are due to the missing name field, which is likely due to GDPR compliance updates server side.

truthbyron commented 4 years ago

Thank you both @ssbarnea and @csullivannet ! I hope to able to get back to this mid-next week and help fix any legitimate broken tests.

csullivannet commented 4 years ago

@ssbarnea @truthbyron I have the CI returned to green by https://github.com/pycontribs/jira/pull/909

It's likely there still remains code broken by Jira's GDPR compliance updates that's not covered by the tests.

truthbyron commented 4 years ago

Nice @csullivannet! I'll be able to do some review/testing tomorrow.

Addono commented 4 years ago

So over the last month or so sometimes a new solution has come up. I will just leave them here as food for thought for anyone who wants to pick it up.

Testing against Jira Cloud - old pipeline:

Testing against Jira Server:

ssbarnea commented 4 years ago

Both approaches are needed, but the 2nd one can be easier to implement by using a test container. We need to be sure that this container starts with ready to use jira instance without any modifications.

Once we have it, we can make use of https://github.com/tox-dev/tox-docker to allow it to start/stop it for testing.

Addono commented 4 years ago

That was also my gut feeling, however no Atlassian SDK containers existed last time I checked - which was ~2 months ago. So I put a Dockerfile together, which can start a Jira standalone instance. However, then you still need to click through the UI to enable Jira Software and create a test user. So, a Jira instance "without modifications" is not enough in order to be able to run the test suite. https://github.com/Addono/docker-atlassian-sdk/blob/master/Dockerfile

The Jira SDK seems quite stable - last release was mid 2019 - so I wouldn't expect it to require a lot of maintenance, hence maintaining it here would be an option - leaving it up to others requires trusting them.

Tox-docker looks cool, probably cleaner to use that than running the container detached as I was fiddling with 😅.

brainwane commented 4 years ago

Since you mentioned sponsorship: Just saw that this package is eligible for funding via Tidelift in case that is of interest to maintainers here.

valentijnscholten commented 3 years ago

I've been having good experiences with https://vcrpy.readthedocs.io/en/latest/ to write integration tests with a JIRA API, for example in https://github.com/DefectDojo/django-DefectDojo/blob/master/dojo/unittests/test_jira_import_and_pushing_api.py Might be less usefull if the goal of you tests is to keep up with changes on the JIRA side.

studioj commented 3 years ago

i would be glad to spend some time here, some one wants to get me up to speed?

i would also be ok to spend time to make this integration tests less needy => mocking out the responses then the only real thing to validate is the authentication mechanism and 1 or 2 very simple tests

@Addono maybe a stupid suggestion couldn't we do a docker commit after doing the setup manually? just getting started so probably shooting in the wild :-) I'll investigate if https://hub.docker.com/r/atlassian/jira-software could be used, maybe in combination with vcrpy that @valentijnscholten suggested

Addono commented 3 years ago

Hi @studioj, it has been a while since I was active here. So not entirely sure if I'm up to date with the latest changes.

In general, the best way to get up to speed is to read back a bit. I wrote plenty of long-form comments which describe the rather involved situation we're in. https://github.com/pycontribs/jira/pull/909 could also be of interest.

... to get the CI to pass again three issues needs to be resolved:

  1. Fixing the codebase and test-suite to be compatible with the latest version of Jira. As no fixes [were] merged, the drifting API specs cause the current test suite to fail.
  2. Provisioning of a Jira instance to test against. Previously, a public Jira Cloud instance was used. However, the credentials to this instance can only be used for branches inside the repository. As giving credentials to this Jira instance to CI runs against PRs from untrusted sources would risk having them exposed.
  3. Updating the CI to make use of the changes made to accommodate (1) and (2).

Dockerizing Jira is task (2), which I took a shot at here: https://github.com/pycontribs/jira/pull/942. So you can build on that.

I'll investigate if https://hub.docker.com/r/atlassian/jira-software could be used

If I remember correctly, the issue is to automatically provision a Jira instance with a test license. The default Docker images can provision a demo license, but that requires clicking through the UI and logging in. Instead I then opted for containerizing the atlassian-cli, see https://github.com/Addono/docker-jira-software-standalone.

couldn't we do a docker commit after doing the setup manually?

Reason being that the demo license is only valid for a short while, so now your CI depends on someone keeping releasing new versions of your image. It's probably not a dealbreaker, so if that's how you get it to work then by al means try, but not having to worry about that would be better 😄 .

studioj commented 3 years ago

thx for the answer. Would it be an idea the maintainers are open to, this vcr.py?

It doesnt seem like an easy thing to dockerize jira and have a full blown running instance to test against.

I also read Atlassian is not providing licenses for jira server and wants to end of life it by 2024 ... see their FAQ

about the forks and CI system, would it be maybe logical to have people merge their forked branch first to a branch of the repo. and then run a second PR to get it into master? it's this issue you're referencing, right?

maybe having a vcr.py test validating PR's from forks and a full blown test against a jira server if to master?

Addono commented 3 years ago

Would it be an idea the maintainers are open to, this vcr.py?

Just quickly glanced over it, I wouldn't now for sure. To me, it seems viable for writing integration tests. However, the old tests were end-to-end/system tests, so this would change the testing strategy.

Probably at this point anything which makes releasing new versions reliably is fine.

I also read Atlassian is not providing licenses for jira server and wants to end of life it by 2024 ...

Then one could argue that the project drops support for Jira server, however, the original CI ran against a cloud hosted instance, and then we ran into issues to protect the credentials.

about the forks and CI system, would it be maybe logical to have people merge their forked branch first to a branch of the repo. and then run a second PR to get it into master?

Yeah, I suggested something along those lines earlier.

studioj commented 3 years ago

i'll first try to document the steps needed to get the tests up and running locally :-) can't really find too much docu on that

Addono commented 3 years ago

AFAIK this is all there is. https://github.com/pycontribs/jira#setup

Be adviced, the tests are likely not to pass, since this tests against a Jira server instance rather than Jira Software Cloud. So at least you need to enable Jira Software.

studioj commented 3 years ago

ok cool, i got some tests up and running via tox (new to tox) (seems to be the jira cloud tests)

=> i cant find where the test requirements are stored :-S so dont know really where to add it. (maybe just contributing.rst?)

tests are failing now because of basic auth being deprecated, is that the correct state?

ok seems running the tests on a local instance can be done by setting env: CI_JIRA_URL=http://localhost:2990/jira CI_JIRA_ADMIN=admin CI_JIRA_ADMIN_PASSWORD=admin CI_JIRA_USER=jira_user CI_JIRA_USER_PASSWORD=jira

Addono commented 3 years ago

This is where the test dependencies are defined:

https://github.com/pycontribs/jira/blob/7fa3a454c08a14e2d7d7670fcfa87482e16936ba/setup.cfg#L76-L97

If you would install the package using pip install jira[test], then these deps should be installed (🤞 ).

studioj commented 3 years ago

trying to use your docker image to at least get the same results as the CI system ... but i'm not there yet :-)

the docker is now running on windows and i can login via the browser => but I can't run the tests yet

In my ubuntu vm its not working locally yet. 😒

Once I have that docker up and running and have the same state as the CI I'll open a PR with a shell script to help people join the effort.

studioj commented 3 years ago

Am I correct to assume the CI in fix_ci doesnt run on the jira cloud instance?

just commited this which could make getting started a little bit easier https://github.com/studioj/jira/commit/f760c30ad9b444e69febf3b8aa7390c46b8db61f

Addono commented 3 years ago

Am I correct to assume the CI in fix_ci doesnt run on the jira cloud instance?

Yes, that's correct.

ssbarnea commented 3 years ago

I invited @Addono @adehad and @studioj to the repository, giving them write access. (see invitation from github)

Please use them with care and wait for reviews on your own changes. First action you should take is to rebase existing open PRs and see if you can fix them, request them to be fixed or just close them if they do not make sense.

And yep, the invitation is still open. For release, ping me and I can probably reconfigure the publish pipelines.

adehad commented 3 years ago

Just echoing that we still can use all the help we can get !

We are definitely in better shape than we were, i.e. we have a passing test pipeline ( #1017 ). \ But there is still a lot of work on how we support and test against Jira Data Center / Server (self-hosted) and Jira Cloud (Atlassian hosted) moving forward, especially as their APIs diverge.

ssbarnea commented 3 years ago

If anyone is willing to show some love to jira library and help review/merge/abandon current PR and issues, ping zbr on #pycontribs on irc and so an sync and do a weekend clenaup.

adehad commented 3 years ago

@ssbarnea

can I ask if the credentials for the Cloud instance ci-user and ci-admin:

    def set_jira_cloud_details(self):
        self.CI_JIRA_URL = "https://pycontribs.atlassian.net"
        self.CI_JIRA_ADMIN = "ci-admin"
        self.CI_JIRA_ADMIN_PASSWORD = 
        self.CI_JIRA_USER = "ci-user"
        self.CI_JIRA_USER_PASSWORD =

exist as GitHub Actions Secrets?

I would like to propose they exist as:

CI_JIRA_CLOUD_URL
CI_JIRA_CLOUD_ADMIN
CI_JIRA_CLOUD_USER
CI_JIRA_CLOUD_ADMIN_TOKEN
CI_JIRA_CLOUD_USER_TOKEN

It looks like @csullivannet had setup these API tokens at some point on the old Travis system?

There also looks like there may be an OAUTH key, which perhaps could instead be put into a secret?

https://github.com/pycontribs/jira/blob/eb80088cd0da1d27043a2d457f2f045725ef97f0/tests/tests.py#L30-L40

But it might be an either/or situation.

I'm working on #1045 - which is hopefully setting some infrastructure for testing on both Cloud and Server instances.

ssbarnea commented 3 years ago

@adehad ping me again when you can on irc and I can give you extended access to both the repo and the test instance, so you perform maintenance work like updating the tokens, test accounts.

adehad commented 3 years ago

as an update I've got #1107 (and #1109) in a place where the testing on Jira Cloud works for me locally and I've set up the necessary variables in this repo so it should be able to run on GitHub Actions. I'm not sure I can get the new GA to run on that branch without it first existing on master, so I may try and get it working on a fork first. The current file I've setup attempts to wait for the Server CI to finish before running the Cloud CI as an added precaution. image

ssbarnea commented 3 years ago

I am afraid I do not have the time to look at what is happening with the code itself and I just want to handle its maintenance to those that have the time and drive needed.

In order to avoid accidents I will require not to merge changes without another core review and I do expect cores to do perform reviews of changes proposed by others. This can easily be achieved by adding them to CODEOWNERS file, so they would be automatically assigned to reviews.

Based on recent activity I am inclined to believe that @adehad and @studioj would first two to add, if they agree. Still having only two is bad as if one is in unavailable, the other one would not be able to merge his own changes to master. Anyone else interested? If anyone thinks it may not be a good idea to add someone, please reach me in private.

In general I am inclined to trust more github accounts that have real names and publicly identifiable information about their owners (like companies where they work, social links). I don't really want to endup in the news because jira library was used to inject some malware ;)

Zyzyx commented 2 years ago

Hi everyone, I can make time to help with this. I have a few features to work on ahead of me too.

adehad commented 9 months ago

For complete transparency I want to report on the sudden Jira Cloud failures a couple months ago.

It seems that some users had created 'Team' managed projects in our Jira instance. This resulted in the test framework (with some of its hard coded assumptions) failing. ie it was trying to use a permission scheme of another project. This resulted in the failures we saw.

What I have done is removed the permission for anyone creating these Team managed projects. Therefore only company managed projects can be made on that instance, which is done by our CI users.

To be honest I am not really sure why we have so many public users on the pycontribs jira. Currently there are around 300.

@ssbarnea do let me know if this affects anyone to your knowledge.

Setarland commented 3 months ago
Setarland commented 3 months ago

  • []()

.8yiklojghnv

dimitarOnGithub commented 3 months ago

Hi everyone,

I wanted to check in on how are things with this project and if there's anything, anywhere, where one can help out and pick up some bits maybe? I've been using this library for years and I love it, so would be more than happy to try and contribute in any meaningful way.

adehad commented 3 months ago

@dimitarOnGithub Speaking from my perspective I am not using this library in my current job so would be good to have maintainers that are using the package more regularly to ensure it can be kept up to date with changes from Atlassian as well as more thoughtful in any features etc.

Stealthii commented 3 months ago

One of the main issues right now with community contribution is that the Cloud CI credentials are still not set as Actions secrets since March, resulting in all cloud Actions jobs failing.

This will need an admin/maintainer to check:

I'd expect that CI_JIRA_CLOUD_* secrets are laid down under environment settings, but only project level admins will be able to see these. Bear in mind to check secrets and not environment variables, as it's the action workflow that's converting this secret to an env var for cloud runs. I've a suspicion this is not set properly.

dimitarOnGithub commented 3 months ago

@adehad thanks for the reply! I'll be happy to put some time aside and go through the reported issues and PRs, I had a brief look at these and I think there's a few that report/fix pretty much the same thing.

As for the builds situation, it looks like @Stealthii is on to something, but I'm not sure if it's something you'd have the permissions to check?

adehad commented 3 months ago

To my knowledge the issue regarding secrets is a GitHub level limitation, I'm happy to be proven wrong, but there is currently no way to my knowledge to share a 'secret' to contributors of a PR nor a way to manually approve individual PRs to get access to these secrets.

I'd be more than happy if you can review any PRs and flag them to myself or @studioj and I'm sure we can add you as a maintainer once we are aligned on the PR quality etc.