princenyeche / jiraone

A REST API Implementation to Jira Cloud APIs for creating reports and for performing other Jira queries.
https://jiraone.readthedocs.io
MIT License
25 stars 11 forks source link

JSON export #117

Closed enrique-sor56 closed 10 months ago

enrique-sor56 commented 1 year ago

thanks for the contributions

I been trying to export in JSON but it always get's stuck at the same point here is the script

from jiraone import LOGIN, issue_export import json

file = "C:\Users\ [path]\config.json" config = json.load(open(file)) LOGIN(**config)

jql = 'project in (project1, project2) and created >= "2023/10/23 00:00" AND created <= "2023/10/23 23:59" ORDER BY Updated DESC'

properties = ["users", "history"]

issue_export(jql=jql, extension="json", json_properties=properties)

it gets stuck here Converting Jira custom field names to Jira JSON compatible names. Verifying users and group membership

Is there way to bypass the group membership ? with CSV it downloads all the required info

Thanks in advance

princenyeche commented 1 year ago

Hey @enrique-sor56

I'm in the process of updating that JSON export feature in a new version v0.7.9, I should most probably get this finished next month before Christmas. I also had other feedback about it being stuck like you mentioned but essentially, what it's doing is the extraction of each group which takes so long (performance issue). In the upcoming update, this should allow a user to opt out of the group extraction or perform it as a background task (independently from the script - depending on which is performance reliable) and the cache should help maintain a faster extraction after the first run.

princenyeche commented 11 months ago

Hey @enrique-sor56

Thanks for waiting. Just uploaded the new version 0.7.9 which should improve drastically the previous issues with user and group membership

enrique-sor56 commented 10 months ago

Thank you @princenyeche

I update the the Jira One pkg but still seems to get stuck at the same part

Downloading issue export in JSON format. <Response [200]> OK ::downloading issues at page: 0 of 2 <Response [200]> OK ::downloading issues at page: 1 of 2 <Response [200]> OK ::downloading issues at page: 2 of 2 Processing. Current progress: 33% Processing. Current progress: 66% Processing. Current progress: 100% Converting Jira custom field names to Jira JSON compatible names. Verifying users and group membership

Is there another way to download the users and group membership?

princenyeche commented 10 months ago

Hey @enrique-sor56 please upgrade to v0.8.2 as the console printout seems to be an older version and not the latest.

If you're using pip try

python3 -m pip install --upgrade jiraone

To answer your question, no argument allows the input of users and groups from a file for JSON processing. I don't think that's the problem in your case as I believe you might still be on an older version (i.e. if you're using multiple versions of Python or probably on a virtualenv you might want to verify this). v0.7.9 fixed the being stuck problem and others but had a regression which the current version fixes, hence the need to upgrade. However, if you have a large user base then you can make use of these two arguments workers and flush for faster processing.

# previous statements
properties = ["users", "history"]
issue_export(jql=jql, extension="json", json_properties=properties, workers=50, flush=20)

Also, the way the extraction works now is less dependent on the script as the workers argument will help extract simultaneously the data while other operations are running and the flush argument allows a sort of cool-down timeframe prior to when the files are written to disk.

enrique-sor56 commented 10 months ago

thank you @princenyeche it's wroking th JSON file export