Scalene error: received signal SIGSEGV

[an99990]

Describe the bug I keep getting Scalene error: received signal SIGSEGV

To Reproduce Steps to reproduce the behavior: scalene test.py --reduced-profile

# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved
from pathlib import Path

from adet.config import get_cfg
from config import settings
from demo.predictor import VisualizationDemo
from detectron2.data.detection_utils import read_image
from detectron2.utils.logger import setup_logger

from scalene import scalene_profiler

def setup_cfg(args):
    # load config from file and command-line arguments
    cfg = get_cfg()
    cfg.merge_from_file(args[0])
    cfg.merge_from_list(args[-1])
    # Set score_threshold for builtin models
    cfg.MODEL.RETINANET.SCORE_THRESH_TEST = args[2]
    cfg.MODEL.ROI_HEADS.SCORE_THRESH_TEST = args[2]
    cfg.MODEL.FCOS.INFERENCE_TH_TEST = args[2]
    cfg.MODEL.MEInst.INFERENCE_TH_TEST = args[2]
    cfg.MODEL.PANOPTIC_FPN.COMBINE.INSTANCES_CONFIDENCE_THRESH = args[2]
    cfg.freeze()
    return cfg

def main():
    logger = setup_logger()
    root_dir = Path(__file__).parents[0]
    args = [
        f"{root_dir}/models/AdelaiDet/configs/BlendMask/R_101_dcni3_5x.yaml",
        f"{root_dir}/{settings.paths[0].image}",
        0.35,
        ["MODEL.WEIGHTS", f"{root_dir}/models_weights/R_101_dcni3_5x.pth"],
    ]
    cfg = setup_cfg(args)
    demo = VisualizationDemo(cfg)
    root_dir = Path(__file__).parents[1]
    path = root_dir / settings.paths[0].image
    logger.info(f"image: {path}")
    img = read_image(path, format="BGR")
    predictions, _ = demo.run_on_image(img)

if __name__ == "__main__":
    main()

Expected behavior Complete profile but it dies during the process

Screenshots Desktop (please complete the following information):

• OS: Windows + WSL
• Browser: chrome
• Version: scalene (2.0.0)

[emeryberger]

Thanks for the report. That's definitely not the right version of Scalene; how did you install it?

[emeryberger]

Also, can you try running it without importing Scalene, instead just invoking it from the command line?

[an99990]

you mean just calling scalene or just the test.py ?

[emeryberger]

python3 -m scalene ./test.py

[an99990]

i installed it with pip

[emeryberger]

or the invocation you use above with scalene, but either way, without the Scalene import.

[emeryberger]

You can check the version by doing this: python3 -m pip show scalene

[an99990]

ok so calling it with python3 -m scalene ./test.py doesnt give the error. However the report is giving me lines from outside the lines of my code. Is there a way to only report memory consumption form my code and not the internal code ?

[an99990]

actually i just did the command again and it gave me the error

Name: scalene
Version: 1.5.4
Summary: Scalene: A high-resolution, low-overhead CPU, GPU, and memory profiler for Python
Home-page: https://github.com/plasma-umass/scalene
Author: Emery Berger
Author-email: emery@cs.umass.edu
License: Apache License 2.0
Location: /opt/conda/lib/python3.8/site-packages
Requires: cloudpickle, numpy, nvidia-ml-py, rich

[an99990]

root@598aaf603db2:/opt/halodi/halodi-segmentation/halodi_segmentation# python3 -m scalene evaluate_blendmask_memory.py --reduced-profile
WARNING [02/15 15:04:44 d2.config.compat]: Config '/opt/halodi/halodi-segmentation/halodi_segmentation/models/AdelaiDet/configs/BlendMask/R_101_dcni3_5x.yaml' has no VERSION. Assuming it to be compatible with latest v2.
[02/15 15:04:47 detectron2]: image: /opt/halodi/halodi-segmentation/images/person_bike.jpg
/opt/conda/lib/python3.8/site-packages/torch/functional.py:568: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at  /opt/pytorch/pytorch/aten/src/ATen/native/TensorShape.cpp:2156.)
  return _VF.meshgrid(tensors, **kwargs)  # type: ignore[attr-defined]
/opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:3756: UserWarning: Default upsampling behavior when mode=bilinear is changed to align_corners=False since 0.4.0. Please specify align_corners=True if the old behavior is desired. See the documentation of nn.Upsample for details.
  warnings.warn(
                                       Memory usage: ▁▁▁▁▁▂▂▂▃▃▅▅▅▅▆▆▆▇▇▇▇██████ (max:   2.73GB, growth rate:  44%)
        /opt/halodi/halodi-segmentation/halodi_segmentation/models/detectron2/detectron2/layers/batch_norm.py: % of time =  33.68% out of   8.62s.
       ╷       ╷       ╷      ╷       ╷       ╷       ╷               ╷       ╷
       │Time   │–––––– │––––… │–––––– │Memory │–––––– │–––––––––––    │Copy   │
  Line │Python │native │syst… │GPU    │Python │peak   │timeline/%     │(MB/s) │/opt/halodi/halodi-segmentation/halodi_segmentation/models/detectron2/det…
╺━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸
   ... │       │       │      │       │       │       │               │       │
     4 │       │       │      │       │       │    1M │▁              │       │from fvcore.nn.distributed import differentiable_all_reduce
   ... │       │       │      │       │       │       │               │       │
    39 │    1% │   14% │      │       │       │    1M │▁              │       │        self.register_buffer("weight", torch.ones(num_features))
   ... │       │       │      │       │       │       │               │       │
    57 │       │   18% │      │       │       │       │               │       │            return F.batch_norm(
   ... │       │       │      │       │       │       │               │       │
       │       │       │      │       │       │       │               │       │
╶──────┼───────┼───────┼──────┼───────┼───────┼───────┼───────────────┼───────┼───────────────────────────────────────────────────────────────────────────╴
       │       │       │      │       │       │       │               │       │function summary for /opt/halodi/halodi-segmentation/halodi_segmentation/…
    35 │    1% │   14% │      │       │       │    1M │▁              │       │FrozenBatchNorm2d.__init__
    44 │       │   18% │      │       │       │       │               │       │FrozenBatchNorm2d.forward
       ╵       ╵       ╵      ╵       ╵       ╵       ╵               ╵       ╵
Top AVERAGE memory consumption, by line:
(1)     4:     1 MB
(2)    39:     1 MB
Top PEAK memory consumption, by line:
(1)     4:     1 MB
(2)    39:     1 MB
           /opt/halodi/halodi-segmentation/halodi_segmentation/models/AdelaiDet/adet/modeling/fcos/fcos.py: % of time =  18.91% out of   8.62s.
       ╷       ╷       ╷      ╷       ╷       ╷       ╷               ╷       ╷
       │Time   │–––––– │––––… │–––––– │Memory │–––––– │–––––––––––    │Copy   │
  Line │Python │native │syst… │GPU    │Python │peak   │timeline/%     │(MB/s) │/opt/halodi/halodi-segmentation/halodi_segmentation/models/AdelaiDet/adet…
╺━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸
   ... │       │       │      │       │       │       │               │       │
     3 │    6% │    9% │   1% │       │       │   83M │▁▁▁▁▁▁▁▁▁   1% │     8 │import torch
   ... │       │       │      │       │       │       │               │       │
   158 │       │    2% │      │       │       │    2M │▁▁▁▁▁▁▁▁       │       │                tower.append(conv_func(
   ... │       │       │      │       │       │       │               │       │
       │       │       │      │       │       │       │               │       │
╶──────┼───────┼───────┼──────┼───────┼───────┼───────┼───────────────┼───────┼───────────────────────────────────────────────────────────────────────────╴
       │       │       │      │       │       │       │               │       │function summary for /opt/halodi/halodi-segmentation/halodi_segmentation/…
   126 │       │    2% │      │       │       │    2M │▇▇▇▇▇▇▇▇       │       │FCOSHead.__init__
       ╵       ╵       ╵      ╵       ╵       ╵       ╵               ╵       ╵
Top AVERAGE memory consumption, by line:
(1)   158:     2 MB
(2)     3:     1 MB
Top PEAK memory consumption, by line:
(1)     3:    83 MB
(2)   158:     2 MB
     /opt/halodi/halodi-segmentation/halodi_segmentation/models/detectron2/detectron2/modeling/backbone/resnet.py: % of time =  15.04% out of   8.62s.
       ╷       ╷       ╷      ╷       ╷       ╷       ╷               ╷       ╷
       │Time   │–––––– │––––… │–––––– │Memory │–––––– │–––––––––––    │Copy   │
  Line │Python │native │syst… │GPU    │Python │peak   │timeline/%     │(MB/s) │/opt/halodi/halodi-segmentation/halodi_segmentation/models/detectron2/det…
╺━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸
   ... │       │       │      │       │       │       │               │       │
   180 │       │   10% │      │       │       │       │               │       │                weight_init.c2_msra_fill(layer)
   ... │       │       │      │       │       │       │               │       │
   298 │       │    3% │      │       │       │       │               │       │                weight_init.c2_msra_fill(layer)
   ... │       │       │      │       │       │       │               │       │
       │       │       │      │       │       │       │               │       │
╶──────┼───────┼───────┼──────┼───────┼───────┼───────┼───────────────┼───────┼───────────────────────────────────────────────────────────────────────────╴
       │       │       │      │       │       │       │               │       │function summary for /opt/halodi/halodi-segmentation/halodi_segmentation/…
   107 │       │   10% │      │       │       │       │               │       │BottleneckBlock.__init__
   219 │       │    4% │      │       │       │       │               │       │DeformBottleneckBlock.__init__
       ╵       ╵       ╵      ╵       ╵       ╵       ╵               ╵       ╵
         /opt/halodi/halodi-segmentation/halodi_segmentation/models/detectron2/detectron2/utils/visualizer.py: % of time =   9.18% out of   8.62s.
       ╷       ╷       ╷      ╷       ╷       ╷       ╷               ╷       ╷
       │Time   │–––––– │––––… │–––––– │Memory │–––––– │–––––––––––    │Copy   │
  Line │Python │native │syst… │GPU    │Python │peak   │timeline/%     │(MB/s) │/opt/halodi/halodi-segmentation/halodi_segmentation/models/detectron2/det…
╺━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸
   ... │       │       │      │       │       │       │               │       │
     8 │    1% │       │      │       │       │    9M │▁▁▁▁▁▁▁        │       │import matplotlib as mpl
   ... │       │       │      │       │       │       │               │       │
    10 │    5% │       │      │       │       │    7M │▁▁▁▁▁▁         │     9 │import matplotlib.figure as mplfigure
   ... │       │       │      │       │       │       │               │       │
    93 │       │       │      │       │       │   11M │▁▁▁▁           │       │            self._mask = m.astype("uint8")
   ... │       │       │      │       │       │       │               │       │
   125 │       │       │      │       │       │   24M │▁▁▁▁▁▁▁▁▁   2% │    10 │        res = cv2.findContours(mask.astype("uint8"), cv2.RETR_CCOMP, cv2.C
   ... │       │       │      │       │       │       │               │       │
   299 │       │       │      │       │       │   70M │▁▁▁   1%       │     4 │        img = img.astype("uint8")
   300 │       │       │      │       │       │   69M │▁▁▁▁   2%      │     4 │        self.ax.imshow(img, extent=(0, self.width, self.height, 0), interp
   ... │       │       │      │       │       │       │               │       │
   369 │       │       │      │       │       │   69M │▁▁▁▁▁   2%     │     7 │        self.img = np.asarray(img_rgb).clip(0, 255).astype(np.uint8)
   ... │       │       │      │       │       │       │               │       │
       │       │       │      │       │       │       │               │       │
╶──────┼───────┼───────┼──────┼───────┼───────┼───────┼───────────────┼───────┼───────────────────────────────────────────────────────────────────────────╴
       │       │       │      │       │       │       │               │       │function summary for /opt/halodi/halodi-segmentation/halodi_segmentation/…
    67 │       │       │      │       │       │   11M │████           │       │GenericMask.__init__
   119 │       │       │      │       │       │   24M │█████████   2% │    10 │GenericMask.mask_to_polygons
   294 │       │       │      │       │       │   70M │███████   3%   │     8 │VisImage.reset_image
   357 │       │       │      │       │       │   69M │█████   2%     │     7 │Visualizer.__init__
       ╵       ╵       ╵      ╵       ╵       ╵       ╵               ╵       ╵
Top AVERAGE memory consumption, by line:
(1)   299:    70 MB
(2)   369:    34 MB
(3)   125:    23 MB
(4)    93:    11 MB
(5)    10:     2 MB
Top PEAK memory consumption, by line:
(1)   299:    70 MB
(2)   369:    69 MB
(3)   300:    69 MB
(4)   125:    24 MB
(5)    93:    11 MB
      /opt/halodi/halodi-segmentation/halodi_segmentation/models/AdelaiDet/adet/modeling/blendmask/blendmask.py: % of time =   9.02% out of   8.62s.
       ╷       ╷       ╷      ╷       ╷       ╷       ╷               ╷       ╷
       │Time   │–––––– │––––… │–––––– │Memory │–––––– │–––––––––––    │Copy   │
  Line │Python │native │syst… │GPU    │Python │peak   │timeline/%     │(MB/s) │/opt/halodi/halodi-segmentation/halodi_segmentation/models/AdelaiDet/adet…
╺━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸
   ... │       │       │      │       │       │       │               │       │
    52 │       │       │      │       │       │    7M │▁              │       │        self.top_layer = nn.Conv2d(
   ... │       │       │      │       │       │       │               │       │
    58 │       │    4% │   4% │       │       │ 1.24G │▁▁▁▂▂▂▂▂▂  38% │   185 │        pixel_mean = torch.Tensor(cfg.MODEL.PIXEL_MEAN).to(self.device).vi
   ... │       │       │      │       │       │       │               │       │
       │       │       │      │       │       │       │               │       │
╶──────┼───────┼───────┼──────┼───────┼───────┼───────┼───────────────┼───────┼───────────────────────────────────────────────────────────────────────────╴
       │       │       │      │       │       │       │               │       │function summary for /opt/halodi/halodi-segmentation/halodi_segmentation/…
    27 │       │    5% │   3% │       │       │ 1.24G │█████████  38% │   185 │BlendMask.__init__
       ╵       ╵       ╵      ╵       ╵       ╵       ╵               ╵       ╵
Top AVERAGE memory consumption, by line:
(1)    58:  1269 MB
(2)    52:     7 MB
Top PEAK memory consumption, by line:
(1)    58:  1269 MB
(2)    52:     7 MB
                                             evaluate_blendmask_memory.py: % of time =   2.17% out of   8.62s.
       ╷       ╷       ╷      ╷       ╷       ╷       ╷               ╷       ╷
       │Time   │–––––– │––––… │–––––– │Memory │–––––– │–––––––––––    │Copy   │
  Line │Python │native │syst… │GPU    │Python │peak   │timeline/%     │(MB/s) │evaluate_blendmask_memory.py
╺━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸
   ... │       │       │      │       │       │       │               │       │
    31 │    1% │       │      │       │       │    2M │▁              │     2 │        f"{root_dir}/{settings.paths[0].image}",
   ... │       │       │      │       │       │       │               │       │
       │       │       │      │       │       │       │               │       │
╶──────┼───────┼───────┼──────┼───────┼───────┼───────┼───────────────┼───────┼───────────────────────────────────────────────────────────────────────────╴
       │       │       │      │       │       │       │               │       │function summary for evaluate_blendmask_memory.py
    26 │    1% │       │      │       │       │    2M │▁              │     2 │main
       ╵       ╵       ╵      ╵       ╵       ╵       ╵               ╵       ╵
Top AVERAGE memory consumption, by line:
(1)    31:     2 MB
Top PEAK memory consumption, by line:
(1)    31:     2 MB
    /opt/halodi/halodi-segmentation/halodi_segmentation/models/detectron2/detectron2/data/transforms/transform.py: % of time =   1.81% out of   8.62s.
       ╷       ╷       ╷      ╷       ╷       ╷       ╷               ╷       ╷
       │Time   │–––––– │––––… │–––––– │Memory │–––––– │–––––––––––    │Copy   │
  Line │Python │native │syst… │GPU    │Python │peak   │timeline/%     │(MB/s) │/opt/halodi/halodi-segmentation/halodi_segmentation/models/detectron2/det…
╺━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸
   ... │       │       │      │       │       │       │               │       │
   121 │       │    2% │      │       │       │   46M │▁▁▁▁   1%      │       │                pil_image = Image.fromarray(img)
   122 │       │       │      │       │       │   17M │▁▁▁            │       │            pil_image = pil_image.resize((self.new_w, self.new_h), interp_
   123 │       │       │      │       │       │    6M │▁▁▁            │       │            ret = np.asarray(pil_image)
   ... │       │       │      │       │       │       │               │       │
       │       │       │      │       │       │       │               │       │
╶──────┼───────┼───────┼──────┼───────┼───────┼───────┼───────────────┼───────┼───────────────────────────────────────────────────────────────────────────╴
       │       │       │      │       │       │       │               │       │function summary for /opt/halodi/halodi-segmentation/halodi_segmentation/…
   112 │       │    1% │      │       │       │   46M │███████   2%   │       │ResizeTransform.apply_image
       ╵       ╵       ╵      ╵       ╵       ╵       ╵               ╵       ╵
Top AVERAGE memory consumption, by line:
(1)   121:    35 MB
Top PEAK memory consumption, by line:
(1)   121:    46 MB
(2)   122:    17 MB
(3)   123:     6 MB
         /opt/halodi/halodi-segmentation/halodi_segmentation/models/detectron2/detectron2/layers/wrappers.py: % of time =   1.56% out of   8.62s.
       ╷       ╷       ╷      ╷       ╷       ╷       ╷               ╷       ╷
       │Time   │–––––– │––––… │–––––– │Memory │–––––– │–––––––––––    │Copy   │
  Line │Python │native │syst… │GPU    │Python │peak   │timeline/%     │(MB/s) │/opt/halodi/halodi-segmentation/halodi_segmentation/models/detectron2/det…
╺━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸
   ... │       │       │      │       │       │       │               │       │
    87 │       │    1% │      │       │   2%  │    9M │▁▁▁▁▁▁▁▁▁   2% │       │        super().__init__(*args, **kwargs)
   ... │       │       │      │       │       │       │               │       │
   106 │       │       │      │       │       │ 1.12G │▂▃▃▃▃▃▃▃▄  41% │   166 │        x = F.conv2d(
   ... │       │       │      │       │       │       │               │       │
       │       │       │      │       │       │       │               │       │
╶──────┼───────┼───────┼──────┼───────┼───────┼───────┼───────────────┼───────┼───────────────────────────────────────────────────────────────────────────╴
       │       │       │      │       │       │       │               │       │function summary for /opt/halodi/halodi-segmentation/halodi_segmentation/…
    75 │       │    1% │      │       │   2%  │    9M │█████████   2% │       │Conv2d.__init__
    92 │       │       │      │       │       │ 1.12G │█████████  41% │   166 │Conv2d.forward
       ╵       ╵       ╵      ╵       ╵       ╵       ╵               ╵       ╵
Top AVERAGE memory consumption, by line:
(1)   106:  1148 MB
(2)    87:     3 MB
Top PEAK memory consumption, by line:
(1)   106:  1148 MB
(2)    87:     9 MB
          /opt/halodi/halodi-segmentation/halodi_segmentation/models/AdelaiDet/adet/modeling/backbone/fpn.py: % of time =   1.22% out of   8.62s.
       ╷       ╷       ╷      ╷       ╷       ╷       ╷               ╷       ╷
       │Time   │–––––– │––––… │–––––– │Memory │–––––– │–––––––––––    │Copy   │
  Line │Python │native │syst… │GPU    │Python │peak   │timeline/%     │(MB/s) │/opt/halodi/halodi-segmentation/halodi_segmentation/models/AdelaiDet/adet…
╺━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸
   ... │       │       │      │       │       │       │               │       │
    24 │       │       │      │       │       │    2M │▁              │       │        self.p6 = nn.Conv2d(in_channels, out_channels, 3, 2, 1)
    25 │       │       │      │       │       │    2M │▁              │       │        self.p7 = nn.Conv2d(out_channels, out_channels, 3, 2, 1)
   ... │       │       │      │       │       │       │               │       │
       │       │       │      │       │       │       │               │       │
╶──────┼───────┼───────┼──────┼───────┼───────┼───────┼───────────────┼───────┼───────────────────────────────────────────────────────────────────────────╴
       │       │       │      │       │       │       │               │       │function summary for /opt/halodi/halodi-segmentation/halodi_segmentation/…
    20 │       │    1% │      │       │       │    2M │▁              │       │LastLevelP6P7.__init__
       ╵       ╵       ╵      ╵       ╵       ╵       ╵               ╵       ╵
Top AVERAGE memory consumption, by line:
(1)    24:     2 MB
Top PEAK memory consumption, by line:
(1)    24:     2 MB
(2)    25:     2 MB
                            /opt/halodi/halodi-segmentation/halodi_segmentation/config.py: % of time =   1.15% out of   8.62s.
       ╷       ╷       ╷      ╷       ╷       ╷       ╷               ╷       ╷
       │Time   │–––––– │––––… │–––––– │Memory │–––––– │–––––––––––    │Copy   │
  Line │Python │native │syst… │GPU    │Python │peak   │timeline/%     │(MB/s) │/opt/halodi/halodi-segmentation/halodi_segmentation/config.py
╺━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸
     1 │    1% │       │      │       │       │    7M │▁▁▁▁▁▁         │       │from dynaconf import Dynaconf
   ... │       │       │      │       │       │       │               │       │
       ╵       ╵       ╵      ╵       ╵       ╵       ╵               ╵       ╵
Top AVERAGE memory consumption, by line:
(1)     1:     1 MB
Top PEAK memory consumption, by line:
(1)     1:     7 MB
        /opt/halodi/halodi-segmentation/halodi_segmentation/models/detectron2/detectron2/layers/deform_conv.py: % of time =   1.02% out of   8.62s.
       ╷       ╷       ╷      ╷       ╷       ╷       ╷               ╷       ╷
       │Time   │–––––– │––––… │–––––– │Memory │–––––– │–––––––––––    │Copy   │
  Line │Python │native │syst… │GPU    │Python │peak   │timeline/%     │(MB/s) │/opt/halodi/halodi-segmentation/halodi_segmentation/models/detectron2/det…
╺━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸
   ... │       │       │      │       │       │       │               │       │
     9 │       │       │      │       │       │    4M │▁▁▁▁           │       │from torchvision.ops import deform_conv2d
   ... │       │       │      │       │       │       │               │       │
   452 │       │       │      │       │       │    9M │▁▁▁▁▁▁▁▁▁      │       │            torch.Tensor(out_channels, in_channels // groups, *self.kernel
   ... │       │       │      │       │       │       │               │       │
       │       │       │      │       │       │       │               │       │
╶──────┼───────┼───────┼──────┼───────┼───────┼───────┼───────────────┼───────┼───────────────────────────────────────────────────────────────────────────╴
       │       │       │      │       │       │       │               │       │function summary for /opt/halodi/halodi-segmentation/halodi_segmentation/…
   414 │       │       │      │       │       │    9M │▆▆▇▇█████      │       │ModulatedDeformConv.__init__
       ╵       ╵       ╵      ╵       ╵       ╵       ╵               ╵       ╵
Top AVERAGE memory consumption, by line:
(1)   452:     3 MB
(2)     9:     2 MB
Top PEAK memory consumption, by line:
(1)   452:     9 MB
(2)     9:     4 MB
 /opt/halodi/halodi-segmentation/halodi_segmentation/models/detectron2/detectron2/checkpoint/detection_checkpoint.py: % of time =   0.99% out of   8.62s.
       ╷       ╷       ╷      ╷       ╷       ╷       ╷               ╷       ╷
       │Time   │–––––– │––––… │–––––– │Memory │–––––– │–––––––––––    │Copy   │
  Line │Python │native │syst… │GPU    │Python │peak   │timeline/%     │(MB/s) │/opt/halodi/halodi-segmentation/halodi_segmentation/models/detectron2/det…
╺━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸
   ... │       │       │      │       │       │       │               │       │
    88 │       │       │      │       │   3%  │  211M │▁▁▁▁▁▁▁▁▁   3% │       │        loaded = super()._load_file(filename)  # load native pth checkpoin
   ... │       │       │      │       │       │       │               │       │
       │       │       │      │       │       │       │               │       │
╶──────┼───────┼───────┼──────┼───────┼───────┼───────┼───────────────┼───────┼───────────────────────────────────────────────────────────────────────────╴
       │       │       │      │       │       │       │               │       │function summary for /opt/halodi/halodi-segmentation/halodi_segmentation/…
    59 │       │       │      │       │   3%  │  211M │█████████   3% │       │DetectionCheckpointer._load_file
       ╵       ╵       ╵      ╵       ╵       ╵       ╵               ╵       ╵
Top AVERAGE memory consumption, by line:
(1)    88:     3 MB
Top PEAK memory consumption, by line:
(1)    88:   211 MB
       /opt/halodi/halodi-segmentation/halodi_segmentation/models/detectron2/detectron2/data/detection_utils.py: % of time =   0.70% out of   8.62s.
       ╷       ╷       ╷      ╷       ╷       ╷       ╷               ╷       ╷
       │Time   │–––––– │––––… │–––––– │Memory │–––––– │–––––––––––    │Copy   │
  Line │Python │native │syst… │GPU    │Python │peak   │timeline/%     │(MB/s) │/opt/halodi/halodi-segmentation/halodi_segmentation/models/detectron2/det…
╺━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸
   ... │       │       │      │       │       │       │               │       │
    76 │       │       │      │       │       │   47M │▁▁▁▁▁▁   1%    │    11 │        image = image.convert(conversion_format)
    77 │       │       │      │       │       │   69M │▁▁▁▁▁▁▁▁▁   1% │     9 │    image = np.asarray(image)
   ... │       │       │      │       │       │       │               │       │
       │       │       │      │       │       │       │               │       │
╶──────┼───────┼───────┼──────┼───────┼───────┼───────┼───────────────┼───────┼───────────────────────────────────────────────────────────────────────────╴
       │       │       │      │       │       │       │               │       │function summary for /opt/halodi/halodi-segmentation/halodi_segmentation/…
    60 │       │       │      │       │       │   69M │█████████   3% │    20 │convert_PIL_to_numpy
       ╵       ╵       ╵      ╵       ╵       ╵       ╵               ╵       ╵
Top AVERAGE memory consumption, by line:
(1)    76:    47 MB
Top PEAK memory consumption, by line:
(1)    77:    69 MB
(2)    76:    47 MB
           /opt/halodi/halodi-segmentation/halodi_segmentation/models/detectron2/detectron2/config/lazy.py: % of time =   0.36% out of   8.62s.
       ╷       ╷       ╷      ╷       ╷       ╷       ╷               ╷       ╷
       │Time   │–––––– │––––… │–––––– │Memory │–––––– │–––––––––––    │Copy   │
  Line │Python │native │syst… │GPU    │Python │peak   │timeline/%     │(MB/s) │/opt/halodi/halodi-segmentation/halodi_segmentation/models/detectron2/det…
╺━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸
   ... │       │       │      │       │       │       │               │       │
    16 │       │       │      │       │       │    1M │▁▁             │       │from omegaconf import DictConfig, ListConfig, OmegaConf
   ... │       │       │      │       │       │       │               │       │
       ╵       ╵       ╵      ╵       ╵       ╵       ╵               ╵       ╵
Top AVERAGE memory consumption, by line:
(1)    16:     1 MB
Top PEAK memory consumption, by line:
(1)    16:     1 MB
         /opt/halodi/halodi-segmentation/halodi_segmentation/models/detectron2/detectron2/engine/defaults.py: % of time =   0.31% out of   8.62s.
       ╷       ╷       ╷      ╷       ╷       ╷       ╷               ╷       ╷
       │Time   │–––––– │––––… │–––––– │Memory │–––––– │–––––––––––    │Copy   │
  Line │Python │native │syst… │GPU    │Python │peak   │timeline/%     │(MB/s) │/opt/halodi/halodi-segmentation/halodi_segmentation/models/detectron2/det…
╺━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸
   ... │       │       │      │       │       │       │               │       │
   314 │       │       │      │       │       │   10M │▁              │       │            image = torch.as_tensor(image.astype("float32").transpose(2, 0
   ... │       │       │      │       │       │       │               │       │
       │       │       │      │       │       │       │               │       │
╶──────┼───────┼───────┼──────┼───────┼───────┼───────┼───────────────┼───────┼───────────────────────────────────────────────────────────────────────────╴
       │       │       │      │       │       │       │               │       │function summary for /opt/halodi/halodi-segmentation/halodi_segmentation/…
   297 │       │       │      │       │       │   10M │█              │       │DefaultPredictor.__call__
       ╵       ╵       ╵      ╵       ╵       ╵       ╵               ╵       ╵
Top AVERAGE memory consumption, by line:
(1)   314:    10 MB
Top PEAK memory consumption, by line:
(1)   314:    10 MB
       /opt/halodi/halodi-segmentation/halodi_segmentation/models/AdelaiDet/adet/modeling/condinst/condinst.py: % of time =   0.31% out of   8.62s.
       ╷       ╷       ╷      ╷       ╷       ╷       ╷               ╷       ╷
       │Time   │–––––– │––––… │–––––– │Memory │–––––– │–––––––––––    │Copy   │
  Line │Python │native │syst… │GPU    │Python │peak   │timeline/%     │(MB/s) │/opt/halodi/halodi-segmentation/halodi_segmentation/models/AdelaiDet/adet…
╺━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸
   ... │       │       │      │       │       │       │               │       │
     3 │       │       │      │       │       │    1M │▁▁             │     1 │from skimage import color
   ... │       │       │      │       │       │       │               │       │
       ╵       ╵       ╵      ╵       ╵       ╵       ╵               ╵       ╵
Top AVERAGE memory consumption, by line:
(1)     3:     1 MB
Top PEAK memory consumption, by line:
(1)     3:     1 MB
     /opt/halodi/halodi-segmentation/halodi_segmentation/models/AdelaiDet/adet/modeling/blendmask/basis_module.py: % of time =   0.06% out of   8.62s.
       ╷       ╷       ╷      ╷       ╷       ╷       ╷               ╷       ╷
       │Time   │–––––– │––––… │–––––– │Memory │–––––– │–––––––––––    │Copy   │
  Line │Python │native │syst… │GPU    │Python │peak   │timeline/%     │(MB/s) │/opt/halodi/halodi-segmentation/halodi_segmentation/models/AdelaiDet/adet…
╺━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸
   ... │       │       │      │       │       │       │               │       │
    70 │       │       │      │       │       │    2M │▁              │       │            self.seg_head = nn.Sequential(nn.Conv2d(inplanes, planes, kern
   ... │       │       │      │       │       │       │               │       │
       │       │       │      │       │       │       │               │       │
╶──────┼───────┼───────┼──────┼───────┼───────┼───────┼───────────────┼───────┼───────────────────────────────────────────────────────────────────────────╴
       │       │       │      │       │       │       │               │       │function summary for /opt/halodi/halodi-segmentation/halodi_segmentation/…
       ╵       ╵       ╵      ╵       ╵       ╵       ╵               ╵       ╵
Top AVERAGE memory consumption, by line:
(1)    70:     2 MB
Top PEAK memory consumption, by line:
(1)    70:     2 MB
            /opt/halodi/halodi-segmentation/halodi_segmentation/models/AdelaiDet/adet/modeling/__init__.py: % of time =   0.00% out of   8.62s.
       ╷       ╷       ╷      ╷       ╷       ╷       ╷               ╷       ╷
       │Time   │–––––– │––––… │–––––– │Memory │–––––– │–––––––––––    │Copy   │
  Line │Python │native │syst… │GPU    │Python │peak   │timeline/%     │(MB/s) │/opt/halodi/halodi-segmentation/halodi_segmentation/models/AdelaiDet/adet…
╺━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸
   ... │       │       │      │       │       │       │               │       │
     5 │       │       │      │       │       │    1M │▁              │       │from .one_stage_detector import OneStageDetector, OneStageRCNN
   ... │       │       │      │       │       │       │               │       │
       ╵       ╵       ╵      ╵       ╵       ╵       ╵               ╵       ╵
Top AVERAGE memory consumption, by line:
(1)     5:     1 MB
Top PEAK memory consumption, by line:
(1)     5:     1 MB
        /opt/halodi/halodi-segmentation/halodi_segmentation/models/detectron2/detectron2/data/datasets/lvis.py: % of time =   0.00% out of   8.62s.
       ╷       ╷       ╷      ╷       ╷       ╷       ╷               ╷       ╷
       │Time   │–––––– │––––… │–––––– │Memory │–––––– │–––––––––––    │Copy   │
  Line │Python │native │syst… │GPU    │Python │peak   │timeline/%     │(MB/s) │/opt/halodi/halodi-segmentation/halodi_segmentation/models/detectron2/det…
╺━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸
   ... │       │       │      │       │       │       │               │       │
    11 │       │       │      │       │       │    1M │▁              │       │from .lvis_v0_5_categories import LVIS_CATEGORIES as LVIS_V0_5_CATEGORIES
   ... │       │       │      │       │       │       │               │       │
       ╵       ╵       ╵      ╵       ╵       ╵       ╵               ╵       ╵
Top AVERAGE memory consumption, by line:
(1)    11:     1 MB
Top PEAK memory consumption, by line:
(1)    11:     1 MB
        /opt/halodi/halodi-segmentation/halodi_segmentation/models/detectron2/detectron2/layers/shape_spec.py: % of time =   0.00% out of   8.62s.
       ╷       ╷       ╷      ╷       ╷       ╷       ╷               ╷       ╷
       │Time   │–––––– │––––… │–––––– │Memory │–––––– │–––––––––––    │Copy   │
  Line │Python │native │syst… │GPU    │Python │peak   │timeline/%     │(MB/s) │/opt/halodi/halodi-segmentation/halodi_segmentation/models/detectron2/det…
╺━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸
   ... │       │       │      │       │       │       │               │       │
     6 │       │       │      │       │       │    2M │▁              │       │class ShapeSpec(namedtuple("_ShapeSpec", ["channels", "height", "width", "
   ... │       │       │      │       │       │       │               │       │
       ╵       ╵       ╵      ╵       ╵       ╵       ╵               ╵       ╵
Top AVERAGE memory consumption, by line:
(1)     6:     2 MB
Top PEAK memory consumption, by line:
(1)     6:     2 MB
/opt/halodi/halodi-segmentation/halodi_segmentation/models/detectron2/detectron2/modeling/proposal_generator/build.py: % of time =   0.00% out of   8.62s.
       ╷       ╷       ╷      ╷       ╷       ╷       ╷               ╷       ╷
       │Time   │–––––– │––––… │–––––– │Memory │–––––– │–––––––––––    │Copy   │
  Line │Python │native │syst… │GPU    │Python │peak   │timeline/%     │(MB/s) │/opt/halodi/halodi-segmentation/halodi_segmentation/models/detectron2/det…
╺━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸
   ... │       │       │      │       │       │       │               │       │
    12 │       │       │      │       │       │    1M │▁              │       │from . import rpn, rrpn  # noqa F401 isort:skip
   ... │       │       │      │       │       │       │               │       │
       ╵       ╵       ╵      ╵       ╵       ╵       ╵               ╵       ╵
Top AVERAGE memory consumption, by line:
(1)    12:     1 MB
Top PEAK memory consumption, by line:
(1)    12:     1 MB
       /opt/halodi/halodi-segmentation/halodi_segmentation/models/detectron2/detectron2/structures/instances.py: % of time =   0.00% out of   8.62s.
       ╷       ╷       ╷      ╷       ╷       ╷       ╷               ╷       ╷
       │Time   │–––––– │––––… │–––––– │Memory │–––––– │–––––––––––    │Copy   │
  Line │Python │native │syst… │GPU    │Python │peak   │timeline/%     │(MB/s) │/opt/halodi/halodi-segmentation/halodi_segmentation/models/detectron2/det…
╺━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸
   ... │       │       │      │       │       │       │               │       │
   118 │       │       │      │       │       │   46M │▁              │       │                v = v.to(*args, **kwargs)
   ... │       │       │      │       │       │       │               │       │
       │       │       │      │       │       │       │               │       │
╶──────┼───────┼───────┼──────┼───────┼───────┼───────┼───────────────┼───────┼───────────────────────────────────────────────────────────────────────────╴
       │       │       │      │       │       │       │               │       │function summary for /opt/halodi/halodi-segmentation/halodi_segmentation/…
   110 │       │       │      │       │       │   46M │█              │       │Instances.to
       ╵       ╵       ╵      ╵       ╵       ╵       ╵               ╵       ╵
Top AVERAGE memory consumption, by line:
(1)   118:    46 MB
Top PEAK memory consumption, by line:
(1)   118:    46 MB
Scalene error: received signal SIGSEGV
root@598aaf603db2:/opt/halodi/halodi-segmentation/halodi_segmentation# python3 -m pip show scalene
Name: scalene
Version: 1.5.4
Summary: Scalene: A high-resolution, low-overhead CPU, GPU, and memory profiler for Python
Home-page: https://github.com/plasma-umass/scalene
Author: Emery Berger
Author-email: emery@cs.umass.edu
License: Apache License 2.0
Location: /opt/conda/lib/python3.8/site-packages
Requires: cloudpickle, numpy, nvidia-ml-py, rich
Required-by:
root@598aaf603db2:/opt/halodi/halodi-segmentation/halodi_segmentation# python3 -m scalene evaluate_blendmask_memory.py --reduced-profile
WARNING [02/15 15:06:45 d2.config.compat]: Config '/opt/halodi/halodi-segmentation/halodi_segmentation/models/AdelaiDet/configs/BlendMask/R_101_dcni3_5x.yaml' has no VERSION. Assuming it to be compatible with latest v2.
[02/15 15:06:48 detectron2]: image: /opt/halodi/halodi-segmentation/images/person_bike.jpg
/opt/conda/lib/python3.8/site-packages/torch/functional.py:568: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at  /opt/pytorch/pytorch/aten/src/ATen/native/TensorShape.cpp:2156.)
  return _VF.meshgrid(tensors, **kwargs)  # type: ignore[attr-defined]
/opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:3756: UserWarning: Default upsampling behavior when mode=bilinear is changed to align_corners=False since 0.4.0. Please specify align_corners=True if the old behavior is desired. See the documentation of nn.Upsample for details.
  warnings.warn(
                                       Memory usage: ▁▁▁▁▃▃▃▃▃▃▃▅▅▅▅▅▅▅▆▆▆▆▆▇███ (max:   2.73GB, growth rate:  43%)
        /opt/halodi/halodi-segmentation/halodi_segmentation/models/detectron2/detectron2/layers/batch_norm.py: % of time =  32.87% out of   8.54s.
       ╷       ╷       ╷      ╷       ╷       ╷       ╷               ╷       ╷
       │Time   │–––––– │––––… │–––––– │Memory │–––––– │–––––––––––    │Copy   │
  Line │Python │native │syst… │GPU    │Python │peak   │timeline/%     │(MB/s) │/opt/halodi/halodi-segmentation/halodi_segmentation/models/detectron2/det…
╺━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸
   ... │       │       │      │       │       │       │               │       │
     4 │       │       │      │       │       │    1M │▁              │       │from fvcore.nn.distributed import differentiable_all_reduce
   ... │       │       │      │       │       │       │               │       │
    39 │    1% │   13% │      │       │       │       │               │       │        self.register_buffer("weight", torch.ones(num_features))
   ... │       │       │      │       │       │       │               │       │
    57 │       │   15% │   2% │       │       │       │               │       │            return F.batch_norm(
   ... │       │       │      │       │       │       │               │       │
       │       │       │      │       │       │       │               │       │
╶──────┼───────┼───────┼──────┼───────┼───────┼───────┼───────────────┼───────┼───────────────────────────────────────────────────────────────────────────╴
       │       │       │      │       │       │       │               │       │function summary for /opt/halodi/halodi-segmentation/halodi_segmentation/…
    35 │    1% │   13% │      │       │       │       │               │       │FrozenBatchNorm2d.__init__
    44 │       │   15% │   2% │       │       │       │               │       │FrozenBatchNorm2d.forward
       ╵       ╵       ╵      ╵       ╵       ╵       ╵               ╵       ╵
Top AVERAGE memory consumption, by line:
(1)     4:     1 MB
Top PEAK memory consumption, by line:
(1)     4:     1 MB
           /opt/halodi/halodi-segmentation/halodi_segmentation/models/AdelaiDet/adet/modeling/fcos/fcos.py: % of time =  19.56% out of   8.54s.
       ╷       ╷       ╷      ╷       ╷       ╷       ╷               ╷       ╷
       │Time   │–––––– │––––… │–––––– │Memory │–––––– │–––––––––––    │Copy   │
  Line │Python │native │syst… │GPU    │Python │peak   │timeline/%     │(MB/s) │/opt/halodi/halodi-segmentation/halodi_segmentation/models/AdelaiDet/adet…
╺━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸
   ... │       │       │      │       │       │       │               │       │
     3 │    6% │    9% │   2% │       │       │   82M │▁▁▁▁▁▁▁▁▁   1% │     7 │import torch
   ... │       │       │      │       │       │       │               │       │
   158 │       │    2% │      │       │       │    2M │▁▁▁▁▁▁▁▁       │       │                tower.append(conv_func(
   ... │       │       │      │       │       │       │               │       │
       │       │       │      │       │       │       │               │       │
╶──────┼───────┼───────┼──────┼───────┼───────┼───────┼───────────────┼───────┼───────────────────────────────────────────────────────────────────────────╴
       │       │       │      │       │       │       │               │       │function summary for /opt/halodi/halodi-segmentation/halodi_segmentation/…
   126 │       │    3% │      │       │       │    2M │▇▇▇▇▇▇▇▇       │       │FCOSHead.__init__
       ╵       ╵       ╵      ╵       ╵       ╵       ╵               ╵       ╵
Top AVERAGE memory consumption, by line:
(1)   158:     2 MB
(2)     3:     1 MB
Top PEAK memory consumption, by line:
(1)     3:    82 MB
(2)   158:     2 MB
     /opt/halodi/halodi-segmentation/halodi_segmentation/models/detectron2/detectron2/modeling/backbone/resnet.py: % of time =  15.99% out of   8.54s.
       ╷       ╷       ╷      ╷       ╷       ╷       ╷               ╷       ╷
       │Time   │–––––– │––––… │–––––– │Memory │–––––– │–––––––––––    │Copy   │
  Line │Python │native │syst… │GPU    │Python │peak   │timeline/%     │(MB/s) │/opt/halodi/halodi-segmentation/halodi_segmentation/models/detectron2/det…
╺━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸
   ... │       │       │      │       │       │       │               │       │
   180 │       │    7% │      │       │       │       │               │       │                weight_init.c2_msra_fill(layer)
   ... │       │       │      │       │       │       │               │       │
   298 │       │    5% │      │       │       │       │               │       │                weight_init.c2_msra_fill(layer)
   ... │       │       │      │       │       │       │               │       │
   300 │       │    1% │      │       │       │       │               │       │        nn.init.constant_(self.conv2_offset.weight, 0)
   ... │       │       │      │       │       │       │               │       │
       │       │       │      │       │       │       │               │       │
╶──────┼───────┼───────┼──────┼───────┼───────┼───────┼───────────────┼───────┼───────────────────────────────────────────────────────────────────────────╴
       │       │       │      │       │       │       │               │       │function summary for /opt/halodi/halodi-segmentation/halodi_segmentation/…
   107 │       │    7% │      │       │       │       │               │       │BottleneckBlock.__init__
   219 │       │    7% │      │       │       │       │               │       │DeformBottleneckBlock.__init__
       ╵       ╵       ╵      ╵       ╵       ╵       ╵               ╵       ╵
         /opt/halodi/halodi-segmentation/halodi_segmentation/models/detectron2/detectron2/utils/visualizer.py: % of time =   9.27% out of   8.54s.
       ╷       ╷       ╷      ╷       ╷       ╷       ╷               ╷       ╷
       │Time   │–––––– │––––… │–––––– │Memory │–––––– │–––––––––––    │Copy   │
  Line │Python │native │syst… │GPU    │Python │peak   │timeline/%     │(MB/s) │/opt/halodi/halodi-segmentation/halodi_segmentation/models/detectron2/det…
╺━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸
   ... │       │       │      │       │       │       │               │       │
     8 │    1% │       │      │       │       │    9M │▁▁▁▁▁▁▁        │       │import matplotlib as mpl
   ... │       │       │      │       │       │       │               │       │
    10 │    5% │       │      │       │       │    7M │▁▁▁▁▁▁         │     8 │import matplotlib.figure as mplfigure
   ... │       │       │      │       │       │       │               │       │
    93 │       │       │      │       │       │   11M │▁▁▁▁           │       │            self._mask = m.astype("uint8")
   ... │       │       │      │       │       │       │               │       │
   125 │       │       │      │       │       │   23M │▁▁▁▁▁▁▁▁▁   2% │    11 │        res = cv2.findContours(mask.astype("uint8"), cv2.RETR_CCOMP, cv2.C
   ... │       │       │      │       │       │       │               │       │
   299 │       │       │      │       │       │   70M │▁▁▁   1%       │     9 │        img = img.astype("uint8")
   300 │       │       │      │       │       │   69M │▁▁▁▁   2%      │     4 │        self.ax.imshow(img, extent=(0, self.width, self.height, 0), interp
   ... │       │       │      │       │       │       │               │       │
   369 │       │       │      │       │       │   69M │▁▁▁▁▁   2%     │       │        self.img = np.asarray(img_rgb).clip(0, 255).astype(np.uint8)
   ... │       │       │      │       │       │       │               │       │
       │       │       │      │       │       │       │               │       │
╶──────┼───────┼───────┼──────┼───────┼───────┼───────┼───────────────┼───────┼───────────────────────────────────────────────────────────────────────────╴
       │       │       │      │       │       │       │               │       │function summary for /opt/halodi/halodi-segmentation/halodi_segmentation/…
    67 │       │       │      │       │       │   11M │████           │       │GenericMask.__init__
   119 │       │       │      │       │       │   23M │█████████   2% │    11 │GenericMask.mask_to_polygons
   294 │       │       │      │       │       │   70M │███████   3%   │    13 │VisImage.reset_image
   357 │       │       │      │       │       │   69M │█████   2%     │       │Visualizer.__init__
       ╵       ╵       ╵      ╵       ╵       ╵       ╵               ╵       ╵
Top AVERAGE memory consumption, by line:
(1)   299:    70 MB
(2)   369:    35 MB
(3)   125:    23 MB
(4)    93:    11 MB
(5)    10:     2 MB
Top PEAK memory consumption, by line:
(1)   299:    70 MB
(2)   369:    69 MB
(3)   300:    69 MB
(4)   125:    23 MB
(5)    93:    11 MB
      /opt/halodi/halodi-segmentation/halodi_segmentation/models/AdelaiDet/adet/modeling/blendmask/blendmask.py: % of time =   8.95% out of   8.54s.
       ╷       ╷       ╷      ╷       ╷       ╷       ╷               ╷       ╷
       │Time   │–––––– │––––… │–––––– │Memory │–––––– │–––––––––––    │Copy   │
  Line │Python │native │syst… │GPU    │Python │peak   │timeline/%     │(MB/s) │/opt/halodi/halodi-segmentation/halodi_segmentation/models/AdelaiDet/adet…
╺━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸
   ... │       │       │      │       │       │       │               │       │
    52 │       │       │      │       │       │    7M │▁              │       │        self.top_layer = nn.Conv2d(
   ... │       │       │      │       │       │       │               │       │
    58 │       │    4% │   4% │       │       │ 1.24G │▁▁▁▁▁▂▂▂▂  38% │   190 │        pixel_mean = torch.Tensor(cfg.MODEL.PIXEL_MEAN).to(self.device).vi
   ... │       │       │      │       │       │       │               │       │
       │       │       │      │       │       │       │               │       │
╶──────┼───────┼───────┼──────┼───────┼───────┼───────┼───────────────┼───────┼───────────────────────────────────────────────────────────────────────────╴
       │       │       │      │       │       │       │               │       │function summary for /opt/halodi/halodi-segmentation/halodi_segmentation/…
    27 │       │    6% │   2% │       │       │ 1.24G │█████████  38% │   190 │BlendMask.__init__
       ╵       ╵       ╵      ╵       ╵       ╵       ╵               ╵       ╵
Top AVERAGE memory consumption, by line:
(1)    58:  1274 MB
(2)    52:     7 MB
Top PEAK memory consumption, by line:
(1)    58:  1274 MB
(2)    52:     7 MB
                                             evaluate_blendmask_memory.py: % of time =   1.89% out of   8.54s.
       ╷       ╷       ╷      ╷       ╷       ╷       ╷               ╷       ╷
       │Time   │–––––– │––––… │–––––– │Memory │–––––– │–––––––––––    │Copy   │
  Line │Python │native │syst… │GPU    │Python │peak   │timeline/%     │(MB/s) │evaluate_blendmask_memory.py
╺━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸
   ... │       │       │      │       │       │       │               │       │
    31 │    1% │       │      │       │       │    1M │▁              │       │        f"{root_dir}/{settings.paths[0].image}",
   ... │       │       │      │       │       │       │               │       │
       │       │       │      │       │       │       │               │       │
╶──────┼───────┼───────┼──────┼───────┼───────┼───────┼───────────────┼───────┼───────────────────────────────────────────────────────────────────────────╴
       │       │       │      │       │       │       │               │       │function summary for evaluate_blendmask_memory.py
    26 │    1% │       │      │       │       │    1M │▁              │       │main
       ╵       ╵       ╵      ╵       ╵       ╵       ╵               ╵       ╵
Top AVERAGE memory consumption, by line:
(1)    31:     1 MB
Top PEAK memory consumption, by line:
(1)    31:     1 MB
    /opt/halodi/halodi-segmentation/halodi_segmentation/models/detectron2/detectron2/data/transforms/transform.py: % of time =   1.80% out of   8.54s.
       ╷       ╷       ╷      ╷       ╷       ╷       ╷               ╷       ╷
       │Time   │–––––– │––––… │–––––– │Memory │–––––– │–––––––––––    │Copy   │
  Line │Python │native │syst… │GPU    │Python │peak   │timeline/%     │(MB/s) │/opt/halodi/halodi-segmentation/halodi_segmentation/models/detectron2/det…
╺━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸
   ... │       │       │      │       │       │       │               │       │
   121 │       │    1% │      │       │       │   46M │▁▁▁▁   1%      │     2 │                pil_image = Image.fromarray(img)
   122 │       │       │      │       │       │   17M │▁▁▁            │       │            pil_image = pil_image.resize((self.new_w, self.new_h), interp_
   123 │       │       │      │       │       │    6M │▁▁▁            │       │            ret = np.asarray(pil_image)
   ... │       │       │      │       │       │       │               │       │
       │       │       │      │       │       │       │               │       │
╶──────┼───────┼───────┼──────┼───────┼───────┼───────┼───────────────┼───────┼───────────────────────────────────────────────────────────────────────────╴
       │       │       │      │       │       │       │               │       │function summary for /opt/halodi/halodi-segmentation/halodi_segmentation/…
   112 │       │    1% │      │       │       │   46M │████   1%      │     2 │ResizeTransform.apply_image
       ╵       ╵       ╵      ╵       ╵       ╵       ╵               ╵       ╵
Top AVERAGE memory consumption, by line:
(1)   121:    35 MB
Top PEAK memory consumption, by line:
(1)   121:    46 MB
(2)   122:    17 MB
(3)   123:     6 MB
         /opt/halodi/halodi-segmentation/halodi_segmentation/models/detectron2/detectron2/layers/wrappers.py: % of time =   1.46% out of   8.54s.
       ╷       ╷       ╷      ╷       ╷       ╷       ╷               ╷       ╷
       │Time   │–––––– │––––… │–––––– │Memory │–––––– │–––––––––––    │Copy   │
  Line │Python │native │syst… │GPU    │Python │peak   │timeline/%     │(MB/s) │/opt/halodi/halodi-segmentation/halodi_segmentation/models/detectron2/det…
╺━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸
   ... │       │       │      │       │       │       │               │       │
    87 │       │    1% │      │       │   2%  │    9M │▁▁▁▁▁▁▁▁▁   2% │       │        super().__init__(*args, **kwargs)
   ... │       │       │      │       │       │       │               │       │
   106 │       │       │      │       │       │ 1.12G │▃▃▃▃▃▃▃▃▃  41% │   169 │        x = F.conv2d(
   ... │       │       │      │       │       │       │               │       │
       │       │       │      │       │       │       │               │       │
╶──────┼───────┼───────┼──────┼───────┼───────┼───────┼───────────────┼───────┼───────────────────────────────────────────────────────────────────────────╴
       │       │       │      │       │       │       │               │       │function summary for /opt/halodi/halodi-segmentation/halodi_segmentation/…
    75 │       │    1% │      │       │   2%  │    9M │█████████   2% │       │Conv2d.__init__
    92 │       │       │      │       │       │ 1.12G │█████████  41% │   169 │Conv2d.forward
       ╵       ╵       ╵      ╵       ╵       ╵       ╵               ╵       ╵
Top AVERAGE memory consumption, by line:
(1)   106:  1149 MB
(2)    87:     3 MB
Top PEAK memory consumption, by line:
(1)   106:  1149 MB
(2)    87:     9 MB
          /opt/halodi/halodi-segmentation/halodi_segmentation/models/AdelaiDet/adet/modeling/backbone/fpn.py: % of time =   1.24% out of   8.54s.
       ╷       ╷       ╷      ╷       ╷       ╷       ╷               ╷       ╷
       │Time   │–––––– │––––… │–––––– │Memory │–––––– │–––––––––––    │Copy   │
  Line │Python │native │syst… │GPU    │Python │peak   │timeline/%     │(MB/s) │/opt/halodi/halodi-segmentation/halodi_segmentation/models/AdelaiDet/adet…
╺━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸
   ... │       │       │      │       │       │       │               │       │
    24 │       │       │      │       │       │    2M │▁              │       │        self.p6 = nn.Conv2d(in_channels, out_channels, 3, 2, 1)
    25 │       │       │      │       │       │    2M │▁              │       │        self.p7 = nn.Conv2d(out_channels, out_channels, 3, 2, 1)
   ... │       │       │      │       │       │       │               │       │
       │       │       │      │       │       │       │               │       │
╶──────┼───────┼───────┼──────┼───────┼───────┼───────┼───────────────┼───────┼───────────────────────────────────────────────────────────────────────────╴
       │       │       │      │       │       │       │               │       │function summary for /opt/halodi/halodi-segmentation/halodi_segmentation/…
    20 │       │    1% │      │       │       │    2M │▁              │       │LastLevelP6P7.__init__
       ╵       ╵       ╵      ╵       ╵       ╵       ╵               ╵       ╵
Top AVERAGE memory consumption, by line:
(1)    24:     2 MB
Top PEAK memory consumption, by line:
(1)    24:     2 MB
(2)    25:     2 MB
                            /opt/halodi/halodi-segmentation/halodi_segmentation/config.py: % of time =   1.21% out of   8.54s.
       ╷       ╷       ╷      ╷       ╷       ╷       ╷               ╷       ╷
       │Time   │–––––– │––––… │–––––– │Memory │–––––– │–––––––––––    │Copy   │
  Line │Python │native │syst… │GPU    │Python │peak   │timeline/%     │(MB/s) │/opt/halodi/halodi-segmentation/halodi_segmentation/config.py
╺━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸
     1 │    1% │       │      │       │       │    7M │▁▁▁▁▁▁         │     1 │from dynaconf import Dynaconf
   ... │       │       │      │       │       │       │               │       │
       ╵       ╵       ╵      ╵       ╵       ╵       ╵               ╵       ╵
Top AVERAGE memory consumption, by line:
(1)     1:     1 MB
Top PEAK memory consumption, by line:
(1)     1:     7 MB
 /opt/halodi/halodi-segmentation/halodi_segmentation/models/detectron2/detectron2/checkpoint/detection_checkpoint.py: % of time =   1.13% out of   8.54s.
       ╷       ╷       ╷      ╷       ╷       ╷       ╷               ╷       ╷
       │Time   │–––––– │––––… │–––––– │Memory │–––––– │–––––––––––    │Copy   │
  Line │Python │native │syst… │GPU    │Python │peak   │timeline/%     │(MB/s) │/opt/halodi/halodi-segmentation/halodi_segmentation/models/detectron2/det…
╺━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸
   ... │       │       │      │       │       │       │               │       │
    88 │       │       │      │       │   3%  │  211M │▁▁▁▁▁▁▁▁▁   3% │       │        loaded = super()._load_file(filename)  # load native pth checkpoin
   ... │       │       │      │       │       │       │               │       │
       │       │       │      │       │       │       │               │       │
╶──────┼───────┼───────┼──────┼───────┼───────┼───────┼───────────────┼───────┼───────────────────────────────────────────────────────────────────────────╴
       │       │       │      │       │       │       │               │       │function summary for /opt/halodi/halodi-segmentation/halodi_segmentation/…
    59 │       │       │      │       │   3%  │  211M │█████████   3% │       │DetectionCheckpointer._load_file
       ╵       ╵       ╵      ╵       ╵       ╵       ╵               ╵       ╵
Top AVERAGE memory consumption, by line:
(1)    88:     3 MB
Top PEAK memory consumption, by line:
(1)    88:   211 MB
        /opt/halodi/halodi-segmentation/halodi_segmentation/models/detectron2/detectron2/layers/deform_conv.py: % of time =   0.99% out of   8.54s.
       ╷       ╷       ╷      ╷       ╷       ╷       ╷               ╷       ╷
       │Time   │–––––– │––––… │–––––– │Memory │–––––– │–––––––––––    │Copy   │
  Line │Python │native │syst… │GPU    │Python │peak   │timeline/%     │(MB/s) │/opt/halodi/halodi-segmentation/halodi_segmentation/models/detectron2/det…
╺━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸
   ... │       │       │      │       │       │       │               │       │
     9 │       │       │      │       │       │    4M │▁▁▁▁           │       │from torchvision.ops import deform_conv2d
   ... │       │       │      │       │       │       │               │       │
   452 │       │       │      │       │       │    9M │▁▁▁▁▁▁▁▁▁      │       │            torch.Tensor(out_channels, in_channels // groups, *self.kernel
   ... │       │       │      │       │       │       │               │       │
       │       │       │      │       │       │       │               │       │
╶──────┼───────┼───────┼──────┼───────┼───────┼───────┼───────────────┼───────┼───────────────────────────────────────────────────────────────────────────╴
       │       │       │      │       │       │       │               │       │function summary for /opt/halodi/halodi-segmentation/halodi_segmentation/…
   414 │       │       │      │       │       │    9M │▆▆▇▇▇████      │       │ModulatedDeformConv.__init__
       ╵       ╵       ╵      ╵       ╵       ╵       ╵               ╵       ╵
Top AVERAGE memory consumption, by line:
(1)   452:     3 MB
(2)     9:     2 MB
Top PEAK memory consumption, by line:
(1)   452:     9 MB
(2)     9:     4 MB
       /opt/halodi/halodi-segmentation/halodi_segmentation/models/detectron2/detectron2/data/detection_utils.py: % of time =   0.69% out of   8.54s.
       ╷       ╷       ╷      ╷       ╷       ╷       ╷               ╷       ╷
       │Time   │–––––– │––––… │–––––– │Memory │–––––– │–––––––––––    │Copy   │
  Line │Python │native │syst… │GPU    │Python │peak   │timeline/%     │(MB/s) │/opt/halodi/halodi-segmentation/halodi_segmentation/models/detectron2/det…
╺━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸
   ... │       │       │      │       │       │       │               │       │
    76 │       │       │      │       │       │   47M │▁▁▁▁▁▁   1%    │    11 │        image = image.convert(conversion_format)
    77 │       │       │      │       │       │   69M │▁▁▁▁▁▁▁▁▁   1% │     8 │    image = np.asarray(image)
   ... │       │       │      │       │       │       │               │       │
       │       │       │      │       │       │       │               │       │
╶──────┼───────┼───────┼──────┼───────┼───────┼───────┼───────────────┼───────┼───────────────────────────────────────────────────────────────────────────╴
       │       │       │      │       │       │       │               │       │function summary for /opt/halodi/halodi-segmentation/halodi_segmentation/…
    60 │       │       │      │       │       │   69M │█████████   3% │    19 │convert_PIL_to_numpy
       ╵       ╵       ╵      ╵       ╵       ╵       ╵               ╵       ╵
Top AVERAGE memory consumption, by line:
(1)    76:    47 MB
Top PEAK memory consumption, by line:
(1)    77:    69 MB
(2)    76:    47 MB
           /opt/halodi/halodi-segmentation/halodi_segmentation/models/detectron2/detectron2/config/lazy.py: % of time =   0.37% out of   8.54s.
       ╷       ╷       ╷      ╷       ╷       ╷       ╷               ╷       ╷
       │Time   │–––––– │––––… │–––––– │Memory │–––––– │–––––––––––    │Copy   │
  Line │Python │native │syst… │GPU    │Python │peak   │timeline/%     │(MB/s) │/opt/halodi/halodi-segmentation/halodi_segmentation/models/detectron2/det…
╺━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸
   ... │       │       │      │       │       │       │               │       │
    16 │       │       │      │       │       │    1M │▁▁             │     2 │from omegaconf import DictConfig, ListConfig, OmegaConf
   ... │       │       │      │       │       │       │               │       │
       ╵       ╵       ╵      ╵       ╵       ╵       ╵               ╵       ╵
Top AVERAGE memory consumption, by line:
(1)    16:     1 MB
Top PEAK memory consumption, by line:
(1)    16:     1 MB
         /opt/halodi/halodi-segmentation/halodi_segmentation/models/detectron2/detectron2/engine/defaults.py: % of time =   0.35% out of   8.54s.
       ╷       ╷       ╷      ╷       ╷       ╷       ╷               ╷       ╷
       │Time   │–––––– │––––… │–––––– │Memory │–––––– │–––––––––––    │Copy   │
  Line │Python │native │syst… │GPU    │Python │peak   │timeline/%     │(MB/s) │/opt/halodi/halodi-segmentation/halodi_segmentation/models/detectron2/det…
╺━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸
   ... │       │       │      │       │       │       │               │       │
   314 │       │       │      │       │       │   10M │▁              │       │            image = torch.as_tensor(image.astype("float32").transpose(2, 0
   ... │       │       │      │       │       │       │               │       │
       │       │       │      │       │       │       │               │       │
╶──────┼───────┼───────┼──────┼───────┼───────┼───────┼───────────────┼───────┼───────────────────────────────────────────────────────────────────────────╴
       │       │       │      │       │       │       │               │       │function summary for /opt/halodi/halodi-segmentation/halodi_segmentation/…
   297 │       │       │      │       │       │   10M │█              │       │DefaultPredictor.__call__
       ╵       ╵       ╵      ╵       ╵       ╵       ╵               ╵       ╵
Top AVERAGE memory consumption, by line:
(1)   314:    10 MB
Top PEAK memory consumption, by line:
(1)   314:    10 MB
       /opt/halodi/halodi-segmentation/halodi_segmentation/models/AdelaiDet/adet/modeling/condinst/condinst.py: % of time =   0.30% out of   8.54s.
       ╷       ╷       ╷      ╷       ╷       ╷       ╷               ╷       ╷
       │Time   │–––––– │––––… │–––––– │Memory │–––––– │–––––––––––    │Copy   │
  Line │Python │native │syst… │GPU    │Python │peak   │timeline/%     │(MB/s) │/opt/halodi/halodi-segmentation/halodi_segmentation/models/AdelaiDet/adet…
╺━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸
   ... │       │       │      │       │       │       │               │       │
     3 │       │       │      │       │       │    1M │▁▁             │     1 │from skimage import color
   ... │       │       │      │       │       │       │               │       │
       ╵       ╵       ╵      ╵       ╵       ╵       ╵               ╵       ╵
Top AVERAGE memory consumption, by line:
(1)     3:     1 MB
Top PEAK memory consumption, by line:
(1)     3:     1 MB
            /opt/halodi/halodi-segmentation/halodi_segmentation/models/detectron2/detectron2/utils/env.py: % of time =   0.20% out of   8.54s.
       ╷       ╷       ╷      ╷       ╷       ╷       ╷               ╷       ╷
       │Time   │–––––– │––––… │–––––– │Memory │–––––– │–––––––––––    │Copy   │
  Line │Python │native │syst… │GPU    │Python │peak   │timeline/%     │(MB/s) │/opt/halodi/halodi-segmentation/halodi_segmentation/models/detectron2/det…
╺━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸
   ... │       │       │      │       │       │       │               │       │
    72 │       │       │      │       │       │    1M │▁              │       │            import cv2
   ... │       │       │      │       │       │       │               │       │
       │       │       │      │       │       │       │               │       │
╶──────┼───────┼───────┼──────┼───────┼───────┼───────┼───────────────┼───────┼───────────────────────────────────────────────────────────────────────────╴
       │       │       │      │       │       │       │               │       │function summary for /opt/halodi/halodi-segmentation/halodi_segmentation/…
    58 │       │       │      │       │       │    1M │▁              │       │SourceFileLoader._configure_libraries
       ╵       ╵       ╵      ╵       ╵       ╵       ╵               ╵       ╵
Top AVERAGE memory consumption, by line:
(1)    72:     1 MB
Top PEAK memory consumption, by line:
(1)    72:     1 MB
     /opt/halodi/halodi-segmentation/halodi_segmentation/models/detectron2/detectron2/modeling/box_regression.py: % of time =   0.12% out of   8.54s.
       ╷       ╷       ╷      ╷       ╷       ╷       ╷               ╷       ╷
       │Time   │–––––– │––––… │–––––– │Memory │–––––– │–––––––––––    │Copy   │
  Line │Python │native │syst… │GPU    │Python │peak   │timeline/%     │(MB/s) │/opt/halodi/halodi-segmentation/halodi_segmentation/models/detectron2/det…
╺━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸
   ... │       │       │      │       │       │       │               │       │
   120 │       │       │      │       │       │    1M │▁              │       │class Box2BoxTransformRotated(object):
   ... │       │       │      │       │       │       │               │       │
       ╵       ╵       ╵      ╵       ╵       ╵       ╵               ╵       ╵
Top AVERAGE memory consumption, by line:
(1)   120:     1 MB
Top PEAK memory consumption, by line:
(1)   120:     1 MB
        /opt/halodi/halodi-segmentation/halodi_segmentation/models/detectron2/detectron2/data/datasets/lvis.py: % of time =   0.06% out of   8.54s.
       ╷       ╷       ╷      ╷       ╷       ╷       ╷               ╷       ╷
       │Time   │–––––– │––––… │–––––– │Memory │–––––– │–––––––––––    │Copy   │
  Line │Python │native │syst… │GPU    │Python │peak   │timeline/%     │(MB/s) │/opt/halodi/halodi-segmentation/halodi_segmentation/models/detectron2/det…
╺━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸
   ... │       │       │      │       │       │       │               │       │
    12 │       │       │      │       │       │    1M │▁              │       │from .lvis_v1_categories import LVIS_CATEGORIES as LVIS_V1_CATEGORIES
   ... │       │       │      │       │       │       │               │       │
       ╵       ╵       ╵      ╵       ╵       ╵       ╵               ╵       ╵
Top AVERAGE memory consumption, by line:
(1)    12:     1 MB
Top PEAK memory consumption, by line:
(1)    12:     1 MB
     /opt/halodi/halodi-segmentation/halodi_segmentation/models/AdelaiDet/adet/modeling/blendmask/basis_module.py: % of time =   0.00% out of   8.54s.
       ╷       ╷       ╷      ╷       ╷       ╷       ╷               ╷       ╷
       │Time   │–––––– │––––… │–––––– │Memory │–––––– │–––––––––––    │Copy   │
  Line │Python │native │syst… │GPU    │Python │peak   │timeline/%     │(MB/s) │/opt/halodi/halodi-segmentation/halodi_segmentation/models/AdelaiDet/adet…
╺━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸
   ... │       │       │      │       │       │       │               │       │
    70 │       │       │      │       │       │    2M │▁              │       │            self.seg_head = nn.Sequential(nn.Conv2d(inplanes, planes, kern
   ... │       │       │      │       │       │       │               │       │
       ╵       ╵       ╵      ╵       ╵       ╵       ╵               ╵       ╵
Top AVERAGE memory consumption, by line:
(1)    70:     2 MB
Top PEAK memory consumption, by line:
(1)    70:     2 MB
     /opt/halodi/halodi-segmentation/halodi_segmentation/models/detectron2/detectron2/data/transforms/__init__.py: % of time =   0.00% out of   8.54s.
       ╷       ╷       ╷      ╷       ╷       ╷       ╷               ╷       ╷
       │Time   │–––––– │––––… │–––––– │Memory │–––––– │–––––––––––    │Copy   │
  Line │Python │native │syst… │GPU    │Python │peak   │timeline/%     │(MB/s) │/opt/halodi/halodi-segmentation/halodi_segmentation/models/detectron2/det…
╺━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸
   ... │       │       │      │       │       │       │               │       │
     2 │       │       │      │       │       │    1M │▁              │       │from fvcore.transforms.transform import Transform, TransformList  # order
   ... │       │       │      │       │       │       │               │       │
       ╵       ╵       ╵      ╵       ╵       ╵       ╵               ╵       ╵
Top AVERAGE memory consumption, by line:
(1)     2:     1 MB
Top PEAK memory consumption, by line:
(1)     2:     1 MB
           /opt/halodi/halodi-segmentation/halodi_segmentation/models/detectron2/detectron2/engine/hooks.py: % of time =   0.00% out of   8.54s.
       ╷       ╷       ╷      ╷       ╷       ╷       ╷               ╷       ╷
       │Time   │–––––– │––––… │–––––– │Memory │–––––– │–––––––––––    │Copy   │
  Line │Python │native │syst… │GPU    │Python │peak   │timeline/%     │(MB/s) │/opt/halodi/halodi-segmentation/halodi_segmentation/models/detectron2/det…
╺━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸
   ... │       │       │      │       │       │       │               │       │
   156 │       │       │      │       │       │    1M │▁              │       │class PeriodicWriter(HookBase):
   ... │       │       │      │       │       │       │               │       │
       ╵       ╵       ╵      ╵       ╵       ╵       ╵               ╵       ╵
Top AVERAGE memory consumption, by line:
(1)   156:     1 MB
Top PEAK memory consumption, by line:
(1)   156:     1 MB
       /opt/halodi/halodi-segmentation/halodi_segmentation/models/detectron2/detectron2/structures/instances.py: % of time =   0.00% out of   8.54s.
       ╷       ╷       ╷      ╷       ╷       ╷       ╷               ╷       ╷
       │Time   │–––––– │––––… │–––––– │Memory │–––––– │–––––––––––    │Copy   │
  Line │Python │native │syst… │GPU    │Python │peak   │timeline/%     │(MB/s) │/opt/halodi/halodi-segmentation/halodi_segmentation/models/detectron2/det…
╺━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━┿━━━━━━━┿━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸
   ... │       │       │      │       │       │       │               │       │
   118 │       │       │      │       │       │   46M │▁              │       │                v = v.to(*args, **kwargs)
   ... │       │       │      │       │       │       │               │       │
       │       │       │      │       │       │       │               │       │
╶──────┼───────┼───────┼──────┼───────┼───────┼───────┼───────────────┼───────┼───────────────────────────────────────────────────────────────────────────╴
       │       │       │      │       │       │       │               │       │function summary for /opt/halodi/halodi-segmentation/halodi_segmentation/…
   110 │       │       │      │       │       │   46M │█              │       │Instances.to
       ╵       ╵       ╵      ╵       ╵       ╵       ╵               ╵       ╵
Top AVERAGE memory consumption, by line:
(1)   118:    46 MB
Top PEAK memory consumption, by line:
(1)   118:    46 MB
Scalene error: received signal SIGSEGV

[emeryberger]

I'm not sure what's causing the SIGSEGV but we will track it down. Meanwhile, you can invoke scalene --reduced-profile for the CLI invocation. I am surprised that it's not bringing up the new web GUI - are you on a headless machine?

[an99990]

No actually when i run the command on the vscode terminal it prompts an localhost website that crashed. but when i call it though windows terminal i get it printed out the console. and i have used reduced-profile but it gives me the same issue : /. thank you for your support

[emeryberger]

FWIW, you might try these options:

  --profile-only PROFILE_ONLY
                        profile only code in filenames that contain the given strings, separated by commas (default: no restrictions)
  --profile-exclude PROFILE_EXCLUDE
                        do not profile code in filenames that contain the given strings, separated by commas (default: no restrictions)

[an99990]

thank you , i have tried these option, but it seems that the program still goes inside the modules imports. Is there a way to only get profile from file and not the imports ?

thank you

[merijn]

I am seeing the exact same termination with SIGSEGV behaviour with Scalene 1.5.4 and Python 3.8.12

[sternj]

What is the config package here?

[emeryberger]

@an99990 can you share the exact command lines you have used where you get all of these libraries included (and your current working directory)? Scalene looks inside the current working directory and includes anything rooted from there; only --profile-all should include code outside that directory.