search

VUIIS / dax

Distributed Automation for XNAT
MIT License
25 stars 24 forks source link

Create XnatShareSession #424

Closed duettwe closed 11 months ago

duettwe commented 1 year ago

NOT DONE

Right now I'm just getting this to actually work with a test, then I'll go in and make it like the other tools.

duettwe commented 1 year ago

This almost works. It shares into the VUIIS_ABCD project, BUT it shares the ACCESSION NUMBER rather than the subject/experiment.

For instance, VUIISXNAT04_S17846 instead of 212533 for the subject and VUIISXNAT04_E232899 instead of 212533 for the experiment. This is from VUSTP to VUIIS_ABCD.

When I go into the subject, I also do not see any scans or experiments listed, but when I click delete, they are listed as something that can be deleted.

And yes, this is without sharing the experiment, just trying to share the subject.

baxpr commented 1 year ago

subj_str should be subj.label and similar for exp_str. Something along those lines

duettwe commented 1 year ago

So this works, but there's some issues:

  1. Only shares MR Session Experiments
  2. Requires csv in Xnatdownload format (download_report.csv)
  3. Doesn't accept subjects/sessions/experiments as input
  4. Doesn't pad when there are sessions like '001' (VUIIS_ABCD); this would end up being just '1'

I'd like to fix all of these, but this does work, it's just a pain.

I'm also going to change all the print statements to LOGGER so it looks better.

baxpr commented 1 year ago

1 is ok for now but eventually would also like to handle PET

2-3 goal would be to supply a source project, list of sessions, and destination project. First two could come from any csv if you read in with pandas

4 Sounds like just need to read in csv as string not numeric. Can force this with dtype=str in pandas.read_csv

duettwe commented 1 year ago

I think I have a way to fix all of these. The easiest way to fix 1-3 is that Xnatdownload is used. I can scrape experiment type (MR, CT, PET, HD) from there. Obviously can get project/sessions/subjects from that. Only would have to ask for destination.

This would require users to open the csv and remove what they didn't want though, or specify exactly what they want through Xnatdownload.

baxpr commented 1 year ago

Work from an Xnatreport instead of Xnatdownload

duettwe commented 1 year ago

Updated code to fix 1/4 and work better with 2/3. Now getting 502 errors though:

Traceback (most recent call last):
  File "./XnatShareSession_test", line 169, in share_subj
    subj.insert(**{
  File "/home/duettwe/.local/lib/python3.8/site-packages/pyxnat/core/resources.py", line 423, in create
    output = self._intf._exec(create_uri, 'PUT')
  File "/home/duettwe/.local/lib/python3.8/site-packages/pyxnat/core/interfaces.py", line 357, in _exec
    catch_error(response.content, '''pyxnat._exec failure:
  File "/home/duettwe/.local/lib/python3.8/site-packages/pyxnat/core/errors.py", line 84, in catch_error
    raise DatabaseError(full_response)
pyxnat.core.errors.DatabaseError: pyxnat._exec failure:
                    URI: https://xnat.vanderbilt.edu/xnat/data/projects/VUIIS_ABCD/subjects/001?xsiType=xnat:subjectData&xnat:subjectData/sharing/share/project=VUSTP&xnat:subjectData/sharing/share/label=001
                    status code: 502
                    headers: {'Date': 'Fri, 21 Apr 2023 15:19:50 GMT', 'Server': 'Apache/2.4.41 (Ubuntu)', 'Content-Length': '427', 'Keep-Alive': 'timeout=5, max=90', 'Connection': 'Keep-Alive', 'Content-Type': 'text/html; charset=iso-8859-1'}
                    content: b'<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">\n<html><head>\n<title>502 Proxy Error</title>\n</head><body>\n<h1>Proxy Error</h1>\n<p>The proxy server received an invalid\r\nresponse from an upstream server.<br />\r\nThe proxy server could not handle the request<p>Reason: <strong>Error reading from remote server</strong></p></p>\n<hr>\n<address>Apache/2.4.41 (Ubuntu) Server at xnat.vanderbilt.edu Port 443</address>\n</body></html>\n'

This did share the subject, but got none of the data over before crashing out.

duettwe commented 1 year ago

Do I need to do some sort of lockfile? Similar to what you did with the NIRS project sharing?

baxpr commented 1 year ago

No - lockfile was for some other reason iirc

duettwe commented 1 year ago

I also need to set it up where it grabs the sending project from the csv if it's not provided.

duettwe commented 1 year ago

If you guys want to give this a go. Let me know what needs to be changed and what not. It can use the output of Xnatreport csv as the --csv file. It's also REALLY slow, so if you have any ideas there too, that'd be great.

Also, I'm not sure how to make this work where it's just

XnatShareSession --to PID --csv XXX.csv

Right now I have to do the ./ in front.

baxpr commented 1 year ago

Suggest refering to "source" and "destination" project throughout, and name vars as src_whatever and dest_whatever - will help clarity a lot

baxpr commented 1 year ago

best practice - if possible put all xnat operations together in a

with pyxnat.Interface() as XNAT:

I think Interface method will refer to XNAT_HOST env var and .netrc when no options are given

... example: https://github.com/VUIIS/dax/blob/5045c8de2eeab6418c95e0a42ea2cf5c0d515708/bin/Xnat_tools/Xnatsetvar#L94

duettwe commented 1 year ago

Suggest refering to "source" and "destination" project throughout, and name vars as src_whatever and dest_whatever - will help clarity a lot

This is done. Let me know if you see any more.

best practice - if possible put all xnat operations together in a

with pyxnat.Interface() as XNAT:

I think Interface method will refer to XNAT_HOST env var and .netrc when no options are given

... example:

https://github.com/VUIIS/dax/blob/5045c8de2eeab6418c95e0a42ea2cf5c0d515708/bin/Xnat_tools/Xnatsetvar#L94

Working on this.

duettwe commented 1 year ago

best practice - if possible put all xnat operations together in a

with pyxnat.Interface() as XNAT:

I think Interface method will refer to XNAT_HOST env var and .netrc when no options are given

... example:

https://github.com/VUIIS/dax/blob/5045c8de2eeab6418c95e0a42ea2cf5c0d515708/bin/Xnat_tools/Xnatsetvar#L94

You no longer have to type your password.

This is giving me an 'read operation timed out' errors for every subject shared, but not the experiments. Not sure what's going on there yet. They do all end up getting shared though.

duettwe commented 1 year ago

Maybe that was just that particular project or something. This is working as expected on another test.