RAM usage rises dramatically without output

[Kiragroh]

Hi Dan,

I imported over 70 VMAT-Head&Neck cases and started a Query for parotis gland left. No problem but after that I tried to Query the PTV (or other bigger structures) and DVHA was loading and loading and loading. After 15min my 8GB-RAM-MacOS-machine was complaining (see attachement).

For this test I used the 0.8.1 source code without modifications. Do you know this behavior and have some tips?

[cutright]

I'm not able to reproduce this behavior, but I have a few thoughts:

• Can you let me know which versions of the following you're using?
  • macOS: Looks to me probably macOS 10.15?
  • python
  • pydicom
• Check out trace_memory_alloc_simple_stats in dvha.tools.utilities
• How big is your database (memory-wise)?
• Was the query in the same python instance as your import? Memory issues have historically been pydicom related, which is only used during import.

I invested a lot of time in how to handle memory allocation (see Issue #26 and pydicom user group), this is why the DICOM files get read and re-read in new threads so many times. What's interesting is that I experienced no memory issues on macOS 10.14 even when DVHA used to always store all pydicom objects in memory.

Can you take tracemalloc snapshots before and after exec_query in main.py? https://github.com/cutright/DVH-Analytics/blob/dbbfc30feb3e93af3e36060655eebddef4f28413/dvha/main.py#L753-L754

You'll need something like:

import tracemalloc
from dvha.tools.utilities import trace_memory_alloc_simple_stats
snapshot = tracemalloc.take_snapshot()
trace_memory_alloc_simple_stats(snapshot)

Docs for tracemalloc are here

[cutright]

I suppose I could upgrade my iMac to macOS 10.15 and try this out. I've been keeping my MBP at 10.14 so the executables will work for others who haven't upgraded, and I've also been procrastinating looking into the differences between bash and zshell.

[Kiragroh]

Hi Dan, I will try your suggestions on Sunday or Monday and keep you updated.

Site question: can your anonymizer work for many patients at once? Say: make 73 anonymized patients from 73 normal patients in one run. If so, I maybe could provide you with data.

[cutright]

Yes, DVHA DICOM Editor can edit many patients at once... just check the "Search Sub-Folders" option before browsing.

But admittedly I haven't put a lot a thought into all the tags that need editing to call it an "annonymizer". It could be that it's actually pretty easy... I remember coming across an article listing all of the tags that should be edited somewhere. Maybe I could provide a template covering all the necessary tags. Or maybe pydicom happens to have something built in? (I think not though)

And not to discourage you away from DVHA DICOM Editor... but if you happen to have MIM, their anonymizer is super easy and can also do many patients at once. I think the advantage my software has over MIM is that you can keep your cross-file UID links (unless MIM has fixed that by now).

[Kiragroh]

Thanks for the infos. I will try your "tag-editor" very soon. ;)

I found the reason for the problem. It is one specific patient but I dont know why.

Here is a link with dicom, roi-file and query-screenshot for testing. https://ukede-my.sharepoint.com/:f:/g/personal/m_grohmann_uke_de/ElnoRa3X7v1OhqWr1lxGab4Bmnbp6YgzpCH5R04ovPNDzA?e=7dtD8y

Maybe I have to export again but maybe it is a weird bug because I dont see a specific problem with the files.

[cutright]

Trying to import now... this plan definitely spikes up in RAM usage during certain DVH calculations, but then drops on the next DVH calculation. I haven't gone much past 6GB yet, drops to about 600MB between calculations.

I just installed Catalina on my iMac... will have to give that a shot to see if that OS isn't releasing the memory. Maybe each DVH calculation should get its own thread... in which case I bet this crashes on MSW.

I see this is Eclipse... do you have many "High Resolution" contours? I wonder if that's why the RAM usage is so high for certain DVH calculations?

[Kiragroh]

No High Resolution Structures in this case. In the past, I imported a data set with 50+ HighRes structure patients and had no problems. I do not see what is special about this case. It should be the same as all the other 70+ patients. Same tumour, same technique, nearly the same structure set like all the others. In dicompyler I can see the plan without problems. Something weird is happening but maybe it is really rare. Please dont focus to hard on this.

PS: TagEditor is working fine.

[cutright]

Fairly certain these memory spikes occur during the DVH calculation with dicompyler-core because you can see the percentage updating in the status window. Those percentages are received from the callback function in dicompylercore.dvhcalc.get_dvh.

@bastula pointed out to me there is a flag in get_dvh that might be helpful:

memmap_rtdose : bool, optional
        Use memory mapping to access the pixel array of the DICOM RT Dose.
        This reduces memory usage at the expense of increased calculation time.

But I also think there is a macOS Catalina problem. I'm concerned the garbage collector is behaving differently.

[bastula]

When I get a chance I'll try to profile it to see if I can resolve it inside @dicompyler-core or if it's a userland issue.

[cutright]

Catalina is definitely behaving differently. The quoted memory on the Activity Monitor barely gets above 1GB so far (half way through)... and it stays pretty flat the entire time. Mojave would spike up and down between different structures. But the "Real Memory" keeps climbing in Catalina. I'm over 12GB through the first 12 structures.

I'm thinking the answer is each DVH calc should get it's own thread? That might be quite a bit of work to implement... I wish I had pursued multi-threading before designing this code.

Update: It's taken about 40min just to get through 22 / 24 structures. My 70 patient H&N data set only takes about 25 minutes in total to import. So this is extremely slow.

Update 2: Took about 45min today with 12GB real memory used at the end. It only dropped to about 11GB after. Interestingly, the memory usage did not increase during the PTV Overlap and PTV distance calculations. But that queries ROI data points from the database rather than using pydicom objects.

[cutright]

This is turning out to be very interesting.

Took me a little while to get the tracemalloc working for this situation... but it seems like memory is not being released in between DVH calculations for both Mojave and Catalina. I think Mojave has issues too, but just does a much better job than Catalina.

Based on the tracemallac output below, each ROI resets the memory allocation. So I guess the OS is somehow still hanging onto all of it until the application closes?

It also looks like numpy histograms is using the most memory by far and it is the only one that seems to climb dramatically. I'm including so many iterations below so you can see that numpy histogram memory resets after each ROI. It also looks like Structure 1 never calculated a DVH, maybe it has no volume?

(venv) Steak-5k:DVH-Analytics Owls$ python dvha_app.py
WARNING: Skipping mlc_stat_data calculation because  'Dataset' object has no attribute 'BeamLimitingDevicePositionSequence'
WARNING: Skipping mlc_stat_data calculation because  'Dataset' object has no attribute 'BeamLimitingDevicePositionSequence'

**** Updating Structure (2 of 21): Larynx OAR [10%]
-----------------------------------------------------------------
[ Top 10 ]
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/numpy/lib/histograms.py:822: size=163 MiB, count=3, average=54.2 MiB
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/dicompylercore/dvhcalc.py:163: size=163 MiB, count=2, average=81.3 MiB
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/filereader.py:192: size=21.6 MiB, count=1366, average=16.2 KiB
<frozen importlib._bootstrap_external>:487: size=21.5 MiB, count=211032, average=107 B
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/pixel_data_handlers/numpy_handler.py:351: size=13.6 MiB, count=3, average=4643 KiB
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/multival.py:42: size=2454 KiB, count=44865, average=56 B
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/valuerep.py:528: size=2336 KiB, count=44548, average=54 B
<frozen importlib._bootstrap>:219: size=2316 KiB, count=21378, average=111 B
/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/abc.py:133: size=809 KiB, count=2168, average=382 B
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/bokeh/core/has_props.py:158: size=670 KiB, count=2953, average=232 B

**** Updating Structure (2 of 21): Larynx OAR [20%]
-----------------------------------------------------------------
[ Top 10 ]
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/numpy/lib/histograms.py:822: size=325 MiB, count=5, average=65.0 MiB
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/dicompylercore/dvhcalc.py:163: size=163 MiB, count=2, average=81.3 MiB
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/filereader.py:192: size=21.6 MiB, count=1366, average=16.2 KiB
<frozen importlib._bootstrap_external>:487: size=21.5 MiB, count=211032, average=107 B
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/pixel_data_handlers/numpy_handler.py:351: size=13.6 MiB, count=3, average=4643 KiB
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/multival.py:42: size=2454 KiB, count=44864, average=56 B
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/valuerep.py:528: size=2336 KiB, count=44548, average=54 B
<frozen importlib._bootstrap>:219: size=2316 KiB, count=21378, average=111 B
/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/abc.py:133: size=809 KiB, count=2168, average=382 B
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/bokeh/core/has_props.py:158: size=670 KiB, count=2953, average=232 B

**** Updating Structure (2 of 21): Larynx OAR [30%]
-----------------------------------------------------------------
[ Top 10 ]
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/numpy/lib/histograms.py:822: size=488 MiB, count=7, average=69.7 MiB
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/dicompylercore/dvhcalc.py:163: size=163 MiB, count=2, average=81.3 MiB
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/filereader.py:192: size=21.6 MiB, count=1366, average=16.2 KiB
<frozen importlib._bootstrap_external>:487: size=21.5 MiB, count=211032, average=107 B
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/pixel_data_handlers/numpy_handler.py:351: size=13.6 MiB, count=3, average=4643 KiB
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/multival.py:42: size=2454 KiB, count=44864, average=56 B
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/valuerep.py:528: size=2336 KiB, count=44548, average=54 B
<frozen importlib._bootstrap>:219: size=2316 KiB, count=21378, average=111 B
/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/abc.py:133: size=809 KiB, count=2168, average=382 B
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/bokeh/core/has_props.py:158: size=670 KiB, count=2953, average=232 B

**** Updating Structure (2 of 21): Larynx OAR [40%]
-----------------------------------------------------------------
[ Top 10 ]
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/numpy/lib/histograms.py:822: size=650 MiB, count=9, average=72.2 MiB
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/dicompylercore/dvhcalc.py:163: size=163 MiB, count=2, average=81.3 MiB
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/filereader.py:192: size=21.6 MiB, count=1366, average=16.2 KiB
<frozen importlib._bootstrap_external>:487: size=21.5 MiB, count=211032, average=107 B
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/pixel_data_handlers/numpy_handler.py:351: size=13.6 MiB, count=3, average=4643 KiB
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/multival.py:42: size=2454 KiB, count=44864, average=56 B
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/valuerep.py:528: size=2336 KiB, count=44548, average=54 B
<frozen importlib._bootstrap>:219: size=2316 KiB, count=21378, average=111 B
/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/abc.py:133: size=809 KiB, count=2168, average=382 B
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/bokeh/core/has_props.py:158: size=670 KiB, count=2953, average=232 B

**** Updating Structure (2 of 21): Larynx OAR [50%]
-----------------------------------------------------------------
[ Top 10 ]
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/numpy/lib/histograms.py:822: size=813 MiB, count=11, average=73.9 MiB
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/dicompylercore/dvhcalc.py:163: size=163 MiB, count=2, average=81.3 MiB
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/filereader.py:192: size=21.6 MiB, count=1366, average=16.2 KiB
<frozen importlib._bootstrap_external>:487: size=21.5 MiB, count=211032, average=107 B
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/pixel_data_handlers/numpy_handler.py:351: size=13.6 MiB, count=3, average=4643 KiB
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/multival.py:42: size=2454 KiB, count=44864, average=56 B
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/valuerep.py:528: size=2336 KiB, count=44548, average=54 B
<frozen importlib._bootstrap>:219: size=2316 KiB, count=21378, average=111 B
/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/abc.py:133: size=809 KiB, count=2168, average=382 B
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/bokeh/core/has_props.py:158: size=670 KiB, count=2953, average=232 B

**** Updating Structure (2 of 21): Larynx OAR [60%]
-----------------------------------------------------------------
[ Top 10 ]
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/numpy/lib/histograms.py:822: size=975 MiB, count=13, average=75.0 MiB
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/dicompylercore/dvhcalc.py:163: size=163 MiB, count=2, average=81.3 MiB
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/filereader.py:192: size=21.6 MiB, count=1366, average=16.2 KiB
<frozen importlib._bootstrap_external>:487: size=21.5 MiB, count=211032, average=107 B
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/pixel_data_handlers/numpy_handler.py:351: size=13.6 MiB, count=3, average=4643 KiB
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/multival.py:42: size=2454 KiB, count=44864, average=56 B
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/valuerep.py:528: size=2336 KiB, count=44548, average=54 B
<frozen importlib._bootstrap>:219: size=2316 KiB, count=21378, average=111 B
/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/abc.py:133: size=809 KiB, count=2168, average=382 B
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/bokeh/core/has_props.py:158: size=670 KiB, count=2953, average=232 B

**** Updating Structure (2 of 21): Larynx OAR [70%]
-----------------------------------------------------------------
[ Top 10 ]
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/numpy/lib/histograms.py:822: size=1138 MiB, count=15, average=75.8 MiB
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/dicompylercore/dvhcalc.py:163: size=163 MiB, count=2, average=81.3 MiB
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/filereader.py:192: size=21.6 MiB, count=1366, average=16.2 KiB
<frozen importlib._bootstrap_external>:487: size=21.5 MiB, count=211032, average=107 B
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/pixel_data_handlers/numpy_handler.py:351: size=13.6 MiB, count=3, average=4643 KiB
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/multival.py:42: size=2454 KiB, count=44864, average=56 B
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/valuerep.py:528: size=2336 KiB, count=44548, average=54 B
<frozen importlib._bootstrap>:219: size=2316 KiB, count=21378, average=111 B
/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/abc.py:133: size=809 KiB, count=2168, average=382 B
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/bokeh/core/has_props.py:158: size=670 KiB, count=2953, average=232 B

**** Updating Structure (2 of 21): Larynx OAR [80%]
-----------------------------------------------------------------
[ Top 10 ]
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/numpy/lib/histograms.py:822: size=1300 MiB, count=17, average=76.5 MiB
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/dicompylercore/dvhcalc.py:163: size=163 MiB, count=2, average=81.3 MiB
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/filereader.py:192: size=21.6 MiB, count=1366, average=16.2 KiB
<frozen importlib._bootstrap_external>:487: size=21.5 MiB, count=211032, average=107 B
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/pixel_data_handlers/numpy_handler.py:351: size=13.6 MiB, count=3, average=4643 KiB
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/multival.py:42: size=2454 KiB, count=44864, average=56 B
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/valuerep.py:528: size=2336 KiB, count=44548, average=54 B
<frozen importlib._bootstrap>:219: size=2316 KiB, count=21378, average=111 B
/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/abc.py:133: size=809 KiB, count=2168, average=382 B
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/bokeh/core/has_props.py:158: size=670 KiB, count=2953, average=232 B

**** Updating Structure (2 of 21): Larynx OAR [90%]
-----------------------------------------------------------------
[ Top 10 ]
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/numpy/lib/histograms.py:822: size=1463 MiB, count=19, average=77.0 MiB
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/dicompylercore/dvhcalc.py:163: size=163 MiB, count=2, average=81.3 MiB
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/filereader.py:192: size=21.6 MiB, count=1366, average=16.2 KiB
<frozen importlib._bootstrap_external>:487: size=21.5 MiB, count=211032, average=107 B
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/pixel_data_handlers/numpy_handler.py:351: size=13.6 MiB, count=3, average=4643 KiB
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/multival.py:42: size=2454 KiB, count=44864, average=56 B
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/valuerep.py:528: size=2336 KiB, count=44548, average=54 B
<frozen importlib._bootstrap>:219: size=2316 KiB, count=21378, average=111 B
/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/abc.py:133: size=809 KiB, count=2168, average=382 B
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/bokeh/core/has_props.py:158: size=670 KiB, count=2953, average=232 B

**** Updating Structure (2 of 21): Larynx OAR [100%]
-----------------------------------------------------------------
[ Top 10 ]
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/numpy/lib/histograms.py:822: size=1625 MiB, count=21, average=77.4 MiB
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/dicompylercore/dvhcalc.py:163: size=163 MiB, count=2, average=81.3 MiB
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/filereader.py:192: size=21.6 MiB, count=1366, average=16.2 KiB
<frozen importlib._bootstrap_external>:487: size=21.5 MiB, count=211032, average=107 B
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/pixel_data_handlers/numpy_handler.py:351: size=13.6 MiB, count=3, average=4643 KiB
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/multival.py:42: size=2454 KiB, count=44864, average=56 B
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/valuerep.py:528: size=2336 KiB, count=44548, average=54 B
<frozen importlib._bootstrap>:219: size=2316 KiB, count=21378, average=111 B
/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/abc.py:133: size=809 KiB, count=2168, average=382 B
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/bokeh/core/has_props.py:158: size=670 KiB, count=2953, average=232 B

**** Updating Structure (3 of 21): LinseL OAR [25%]
-----------------------------------------------------------------
[ Top 10 ]
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/numpy/lib/histograms.py:822: size=163 MiB, count=3, average=54.2 MiB
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/dicompylercore/dvhcalc.py:163: size=163 MiB, count=2, average=81.3 MiB
/Users/Owls/PycharmProjects/DVH-Analytics/dvha/db/dicom_parser.py:446: size=102 MiB, count=3, average=33.9 MiB
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/filereader.py:192: size=21.6 MiB, count=1369, average=16.2 KiB
<frozen importlib._bootstrap_external>:487: size=21.5 MiB, count=211032, average=107 B
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/pixel_data_handlers/numpy_handler.py:351: size=13.6 MiB, count=3, average=4643 KiB
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/multival.py:42: size=2460 KiB, count=44984, average=56 B
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/valuerep.py:528: size=2342 KiB, count=44660, average=54 B
<frozen importlib._bootstrap>:219: size=2316 KiB, count=21378, average=111 B
/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/abc.py:133: size=809 KiB, count=2168, average=382 B

**** Updating Structure (3 of 21): LinseL OAR [50%]
-----------------------------------------------------------------
[ Top 10 ]
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/numpy/lib/histograms.py:822: size=325 MiB, count=5, average=65.0 MiB
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/dicompylercore/dvhcalc.py:163: size=163 MiB, count=2, average=81.3 MiB
/Users/Owls/PycharmProjects/DVH-Analytics/dvha/db/dicom_parser.py:446: size=102 MiB, count=3, average=33.9 MiB
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/filereader.py:192: size=21.6 MiB, count=1369, average=16.2 KiB
<frozen importlib._bootstrap_external>:487: size=21.5 MiB, count=211032, average=107 B
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/pixel_data_handlers/numpy_handler.py:351: size=13.6 MiB, count=3, average=4643 KiB
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/multival.py:42: size=2460 KiB, count=44984, average=56 B
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/valuerep.py:528: size=2342 KiB, count=44660, average=54 B
<frozen importlib._bootstrap>:219: size=2316 KiB, count=21378, average=111 B
/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/abc.py:133: size=809 KiB, count=2168, average=382 B

**** Updating Structure (3 of 21): LinseL OAR [75%]
-----------------------------------------------------------------
[ Top 10 ]
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/numpy/lib/histograms.py:822: size=488 MiB, count=7, average=69.7 MiB
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/dicompylercore/dvhcalc.py:163: size=163 MiB, count=2, average=81.3 MiB
/Users/Owls/PycharmProjects/DVH-Analytics/dvha/db/dicom_parser.py:446: size=102 MiB, count=3, average=33.9 MiB
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/filereader.py:192: size=21.6 MiB, count=1369, average=16.2 KiB
<frozen importlib._bootstrap_external>:487: size=21.5 MiB, count=211032, average=107 B
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/pixel_data_handlers/numpy_handler.py:351: size=13.6 MiB, count=3, average=4643 KiB
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/multival.py:42: size=2460 KiB, count=44984, average=56 B
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/valuerep.py:528: size=2342 KiB, count=44660, average=54 B
<frozen importlib._bootstrap>:219: size=2316 KiB, count=21378, average=111 B
/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/abc.py:133: size=809 KiB, count=2168, average=382 B

**** Updating Structure (3 of 21): LinseL OAR [100%]
-----------------------------------------------------------------
[ Top 10 ]
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/numpy/lib/histograms.py:822: size=650 MiB, count=9, average=72.2 MiB
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/dicompylercore/dvhcalc.py:163: size=163 MiB, count=2, average=81.3 MiB
/Users/Owls/PycharmProjects/DVH-Analytics/dvha/db/dicom_parser.py:446: size=102 MiB, count=3, average=33.9 MiB
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/filereader.py:192: size=21.6 MiB, count=1369, average=16.2 KiB
<frozen importlib._bootstrap_external>:487: size=21.5 MiB, count=211032, average=107 B
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/pixel_data_handlers/numpy_handler.py:351: size=13.6 MiB, count=3, average=4643 KiB
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/multival.py:42: size=2460 KiB, count=44984, average=56 B
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/valuerep.py:528: size=2342 KiB, count=44660, average=54 B
<frozen importlib._bootstrap>:219: size=2316 KiB, count=21378, average=111 B
/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/abc.py:133: size=809 KiB, count=2168, average=382 B

**** Updating Structure (4 of 21): LinseR OAR [33%]
-----------------------------------------------------------------
[ Top 10 ]
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/numpy/lib/histograms.py:822: size=163 MiB, count=3, average=54.2 MiB
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/dicompylercore/dvhcalc.py:163: size=163 MiB, count=2, average=81.3 MiB
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/filereader.py:192: size=21.6 MiB, count=1371, average=16.1 KiB
<frozen importlib._bootstrap_external>:487: size=21.5 MiB, count=211032, average=107 B
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/pixel_data_handlers/numpy_handler.py:351: size=13.6 MiB, count=3, average=4643 KiB
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/multival.py:42: size=2464 KiB, count=45056, average=56 B
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/valuerep.py:528: size=2346 KiB, count=44732, average=54 B
<frozen importlib._bootstrap>:219: size=2316 KiB, count=21378, average=111 B
/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/abc.py:133: size=809 KiB, count=2168, average=382 B
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/bokeh/core/has_props.py:158: size=670 KiB, count=2953, average=232 B

**** Updating Structure (4 of 21): LinseR OAR [67%]
-----------------------------------------------------------------
[ Top 10 ]
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/numpy/lib/histograms.py:822: size=325 MiB, count=5, average=65.0 MiB
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/dicompylercore/dvhcalc.py:163: size=163 MiB, count=2, average=81.3 MiB
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/filereader.py:192: size=21.6 MiB, count=1371, average=16.1 KiB
<frozen importlib._bootstrap_external>:487: size=21.5 MiB, count=211032, average=107 B
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/pixel_data_handlers/numpy_handler.py:351: size=13.6 MiB, count=3, average=4643 KiB
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/multival.py:42: size=2464 KiB, count=45056, average=56 B
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/valuerep.py:528: size=2346 KiB, count=44732, average=54 B
<frozen importlib._bootstrap>:219: size=2316 KiB, count=21378, average=111 B
/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/abc.py:133: size=809 KiB, count=2168, average=382 B
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/bokeh/core/has_props.py:158: size=670 KiB, count=2953, average=232 B

**** Updating Structure (4 of 21): LinseR OAR [100%]
-----------------------------------------------------------------
[ Top 10 ]
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/numpy/lib/histograms.py:822: size=488 MiB, count=7, average=69.7 MiB
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/dicompylercore/dvhcalc.py:163: size=163 MiB, count=2, average=81.3 MiB
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/filereader.py:192: size=21.6 MiB, count=1371, average=16.1 KiB
<frozen importlib._bootstrap_external>:487: size=21.5 MiB, count=211032, average=107 B
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/pixel_data_handlers/numpy_handler.py:351: size=13.6 MiB, count=3, average=4643 KiB
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/multival.py:42: size=2464 KiB, count=45056, average=56 B
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/valuerep.py:528: size=2346 KiB, count=44732, average=54 B
<frozen importlib._bootstrap>:219: size=2316 KiB, count=21378, average=111 B
/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/abc.py:133: size=809 KiB, count=2168, average=382 B
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/bokeh/core/has_props.py:158: size=670 KiB, count=2953, average=232 B

**** Updating Structure (5 of 21): Lippen OAR [14%]
-----------------------------------------------------------------
[ Top 10 ]
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/numpy/lib/histograms.py:822: size=163 MiB, count=3, average=54.2 MiB
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/dicompylercore/dvhcalc.py:163: size=163 MiB, count=2, average=81.3 MiB
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/filereader.py:192: size=21.6 MiB, count=1377, average=16.1 KiB
<frozen importlib._bootstrap_external>:487: size=21.5 MiB, count=211032, average=107 B
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/pixel_data_handlers/numpy_handler.py:351: size=13.6 MiB, count=3, average=4643 KiB
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/multival.py:42: size=2535 KiB, count=46346, average=56 B
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/valuerep.py:528: size=2415 KiB, count=46027, average=54 B
<frozen importlib._bootstrap>:219: size=2316 KiB, count=21378, average=111 B
/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/abc.py:133: size=809 KiB, count=2168, average=382 B
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/bokeh/core/has_props.py:158: size=670 KiB, count=2953, average=232 B

**** Updating Structure (5 of 21): Lippen OAR [29%]
-----------------------------------------------------------------
[ Top 10 ]
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/numpy/lib/histograms.py:822: size=325 MiB, count=5, average=65.0 MiB
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/dicompylercore/dvhcalc.py:163: size=163 MiB, count=2, average=81.3 MiB
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/filereader.py:192: size=21.6 MiB, count=1377, average=16.1 KiB
<frozen importlib._bootstrap_external>:487: size=21.5 MiB, count=211032, average=107 B
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/pixel_data_handlers/numpy_handler.py:351: size=13.6 MiB, count=3, average=4643 KiB
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/multival.py:42: size=2535 KiB, count=46346, average=56 B
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/valuerep.py:528: size=2415 KiB, count=46027, average=54 B
<frozen importlib._bootstrap>:219: size=2316 KiB, count=21378, average=111 B
/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/abc.py:133: size=809 KiB, count=2168, average=382 B
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/bokeh/core/has_props.py:158: size=670 KiB, count=2953, average=232 B

**** Updating Structure (5 of 21): Lippen OAR [43%]
-----------------------------------------------------------------
[ Top 10 ]
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/numpy/lib/histograms.py:822: size=488 MiB, count=7, average=69.7 MiB
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/dicompylercore/dvhcalc.py:163: size=163 MiB, count=2, average=81.3 MiB
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/filereader.py:192: size=21.6 MiB, count=1377, average=16.1 KiB
<frozen importlib._bootstrap_external>:487: size=21.5 MiB, count=211032, average=107 B
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/pixel_data_handlers/numpy_handler.py:351: size=13.6 MiB, count=3, average=4643 KiB
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/multival.py:42: size=2535 KiB, count=46346, average=56 B
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/valuerep.py:528: size=2415 KiB, count=46027, average=54 B
<frozen importlib._bootstrap>:219: size=2316 KiB, count=21378, average=111 B
/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/abc.py:133: size=809 KiB, count=2168, average=382 B
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/bokeh/core/has_props.py:158: size=670 KiB, count=2953, average=232 B

**** Updating Structure (5 of 21): Lippen OAR [57%]
-----------------------------------------------------------------
[ Top 10 ]
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/numpy/lib/histograms.py:822: size=650 MiB, count=9, average=72.2 MiB
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/dicompylercore/dvhcalc.py:163: size=163 MiB, count=2, average=81.3 MiB
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/filereader.py:192: size=21.6 MiB, count=1377, average=16.1 KiB
<frozen importlib._bootstrap_external>:487: size=21.5 MiB, count=211032, average=107 B
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/pixel_data_handlers/numpy_handler.py:351: size=13.6 MiB, count=3, average=4643 KiB
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/multival.py:42: size=2535 KiB, count=46346, average=56 B
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/valuerep.py:528: size=2415 KiB, count=46027, average=54 B
<frozen importlib._bootstrap>:219: size=2316 KiB, count=21378, average=111 B
/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/abc.py:133: size=809 KiB, count=2168, average=382 B
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/bokeh/core/has_props.py:158: size=670 KiB, count=2953, average=232 B

**** Updating Structure (5 of 21): Lippen OAR [71%]
-----------------------------------------------------------------
[ Top 10 ]
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/numpy/lib/histograms.py:822: size=813 MiB, count=11, average=73.9 MiB
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/dicompylercore/dvhcalc.py:163: size=163 MiB, count=2, average=81.3 MiB
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/filereader.py:192: size=21.6 MiB, count=1377, average=16.1 KiB
<frozen importlib._bootstrap_external>:487: size=21.5 MiB, count=211032, average=107 B
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/pixel_data_handlers/numpy_handler.py:351: size=13.6 MiB, count=3, average=4643 KiB
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/multival.py:42: size=2535 KiB, count=46346, average=56 B
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/valuerep.py:528: size=2415 KiB, count=46027, average=54 B
<frozen importlib._bootstrap>:219: size=2316 KiB, count=21378, average=111 B
/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/abc.py:133: size=809 KiB, count=2168, average=382 B
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/bokeh/core/has_props.py:158: size=670 KiB, count=2953, average=232 B

**** Updating Structure (5 of 21): Lippen OAR [86%]
-----------------------------------------------------------------
[ Top 10 ]
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/numpy/lib/histograms.py:822: size=975 MiB, count=13, average=75.0 MiB
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/dicompylercore/dvhcalc.py:163: size=163 MiB, count=2, average=81.3 MiB
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/filereader.py:192: size=21.6 MiB, count=1377, average=16.1 KiB
<frozen importlib._bootstrap_external>:487: size=21.5 MiB, count=211032, average=107 B
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/pixel_data_handlers/numpy_handler.py:351: size=13.6 MiB, count=3, average=4643 KiB
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/multival.py:42: size=2535 KiB, count=46346, average=56 B
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/valuerep.py:528: size=2415 KiB, count=46027, average=54 B
<frozen importlib._bootstrap>:219: size=2316 KiB, count=21378, average=111 B
/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/abc.py:133: size=809 KiB, count=2168, average=382 B
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/bokeh/core/has_props.py:158: size=670 KiB, count=2953, average=232 B

**** Updating Structure (5 of 21): Lippen OAR [100%]
-----------------------------------------------------------------
[ Top 10 ]
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/numpy/lib/histograms.py:822: size=1138 MiB, count=15, average=75.8 MiB
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/dicompylercore/dvhcalc.py:163: size=163 MiB, count=2, average=81.3 MiB
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/filereader.py:192: size=21.6 MiB, count=1377, average=16.1 KiB
<frozen importlib._bootstrap_external>:487: size=21.5 MiB, count=211032, average=107 B
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/pixel_data_handlers/numpy_handler.py:351: size=13.6 MiB, count=3, average=4643 KiB
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/multival.py:42: size=2535 KiB, count=46346, average=56 B
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/valuerep.py:528: size=2415 KiB, count=46027, average=54 B
<frozen importlib._bootstrap>:219: size=2316 KiB, count=21378, average=111 B
/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/abc.py:133: size=809 KiB, count=2168, average=382 B
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/bokeh/core/has_props.py:158: size=670 KiB, count=2953, average=232 B

**** Updating Structure (6 of 21): LungeTeilL OAR [2%]
-----------------------------------------------------------------
[ Top 10 ]
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/numpy/lib/histograms.py:822: size=163 MiB, count=3, average=54.2 MiB
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/dicompylercore/dvhcalc.py:163: size=163 MiB, count=2, average=81.3 MiB
<frozen importlib._bootstrap_external>:487: size=21.5 MiB, count=211032, average=107 B
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/filereader.py:192: size=21.4 MiB, count=1432, average=15.3 KiB
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/pixel_data_handlers/numpy_handler.py:351: size=13.6 MiB, count=3, average=4643 KiB
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/multival.py:42: size=4180 KiB, count=76437, average=56 B
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/valuerep.py:528: size=4015 KiB, count=76166, average=54 B
<frozen importlib._bootstrap>:219: size=2316 KiB, count=21378, average=111 B
/Users/Owls/PycharmProjects/DVH-Analytics/venv/lib/python3.6/site-packages/pydicom/multival.py:78: size=860 KiB, count=20034, average=44 B
/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/abc.py:133: size=809 KiB, count=2168, average=382 B

[Kiragroh]

I am glad my found problem is a help to you and no random error. Calculation per thread sounds like a good way to start. Empty structure is unlikely. I can look at all structures in dicompylerGUI and I thought dvha is filtering for that?

[cutright]

It's been driving us nuts all night! I think @bastula tracked it down. Will follow-up soon.

RE: empty structure That was my fault I think. While testing I was killing processes mid calculation and it prevented partially imported data from being deleted.

[cutright]

The dose grid is reporting some VERY high doses, blowing up the np.histogram. If I add a dose limit=50000 kwarg when I call dicompylercore.dvhcalc.get_dvh the issue goes away.

Any reason why this dose grid is reporting doses >500Gy? I'll have to pull it up in a TPS and poke around. Coincidentally, I think this is why my brachy plans were taking so long to calculate.

[Kiragroh]

I found the problematic structures. Here a dvh without couch structure: Here a dvh with couch structure (CTV LAW_L_IV same issue):

This normally does not happen in Eclipse but I think I know why the high dose value is there. 1) Eclipse shows insanely high dose values in structures outside the dose matrix. Normally this only happens for external TPS. 2) Maybe, in this case, we have a mini hole in body structure and Eclipse has sometimes problems with that because dose is only calculated in body and support structures. But normally you get errors while optimization.

Still strange but maybe these infos are helping you.

[cutright]

Nice work. I think there may be a few more structures with the large histogram that maybe isn't showing up in the GUI? I think @bastula noticed the CTVs having large histograms. The dicompyler binary puts a limit of 500Gy. The dose grid you sent over shows a max dose of 2.1kGy!

>>> import pydicom
>>> import numpy as np
>>> ds = pydicom.read_file('Anon_2020_6_6_20_49_4_864/RTDOSE_EclipseDosen_0/RTDOSE_0.dcm')
>>> np.max(ds.pixel_array) * float(ds.DoseGridScaling) / 100
2130.316531624087

I'll put a default cap of 500Gy in the next DVHA version, but allow the user to change that if they want.

[Kiragroh]

That would be nice. Thanks.

One extra info: all structures I imported with the uploaded roi-file are looking fine in the dicompyler-gui but I understand that the dose matrix stay problematic, even if no affected structure will be imported.

Maybe you can make the body-max dose to the limit (if body is present and otherwise 500gy and editable)? In the uploaded case it would be enough and would make the import process even quicker because maxDose<60gy.

But maybe this not the safest solution?

[cutright]

I imagine you could apply a mask to the body then use np.max on that. I'll poke around to see how easy that is. Alternatively, I could let you define a max dose bin as percentage of Rx dose. Maybe set one for external beam and another for brachy. This problem seems rare enough to not warrant a complicated solution.

[cutright]

DVH v0.8.3 will let you cap DVH bin by absolute dose or % Rx. If no Rx is defined, the import will default to the absolute dose limit. This plan imported in less than 12sec. Way better than 45 minutes! 👍

Looking forward to timing this with my 70-patient H&N dataset.

Update:

• no difference in calc time with my 70-patient H&N dataset
• Default limits are 500Gy or 300% Rx