search

kordk / torch-ecpg

(GPU accelerated) eCpG mapper
BSD 3-Clause "New" or "Revised" License
2 stars 0 forks source link

Region filtration for regression_full.py #22

Closed liamgd closed 1 year ago

liamgd commented 1 year ago

The regression full algorithm should have region mapping capabilities as outlined in #7.

Optional annotation files M_annot and G_annot will supplement the M and G dataframes and determine the distance between loci. They will be required for cis, distal, and trans analyses.

All: regress every methylation and gene expression pair. Cis: regress every methylation and gene expression pair within a default window on the same chromosome. Distal: regress every methylation and gene expression pair outside of a default window on the same chromosome. Trans: regress every methylation and gene expression pair on different chromosomes.

liamgd commented 1 year ago

Various implementations of region mapping should be tested for performance.

Creation of region filtration boolean matrix

  1. Full matrix during constant generation
  2. Matrix generated for the next chunk every chunk
  3. List generated every loop over methylation row

Filtering tensor using boolean matrix

  1. Filter M_np in the beginning and XtXi_Xt and X each iteration
  2. Filter Y, XtXi_Xt, and X each iteration
  3. Filter P after each iteration
  4. Filter the output dataframe after each iteration
liamgd commented 1 year ago

Testing region filtration matrix creation with kernprof

Test of speed of creation of region filtration matrix with 300 samples, 50,000 methylation loci, and 10,000 gene expression loci.

Line #      Hits         Time  Per Hit   % Time  Line Contents
...

    77                                               # Chromosomal filtration
    78         1     153939.6 153939.6      1.4      M_chrom_t[:, None] == G_chrom_t
    79         1     136644.3 136644.3      1.3      G_chrom_t[:, None] == M_chrom_t
    80                                           
    81                                               # Position filtration
    82         1    2145274.1 2145274.1     19.8      torch.isclose(M_pos_t[:, None], G_pos_t, rtol=0, atol=window)
    83         1    2169992.9 2169992.9     20.0      torch.isclose(G_pos_t[:, None], M_pos_t, rtol=0, atol=window)
    84         1     720406.6 720406.6      6.6      (M_pos_t[:, None] < G_pos_t + window).logical_and(M_pos_t[:, None] > G_pos_t - window)
    85         1     671274.2 671274.2      6.2      (G_pos_t[:, None] < M_pos_t + window).logical_and(G_pos_t[:, None] > M_pos_t - window)
    86         1    1111070.2 1111070.2     10.3      (M_pos_t[:, None] - G_pos_t).abs() < window
    87         1    1208527.4 1208527.4     11.2      (G_pos_t[:, None] - M_pos_t).abs() < window
    88         1    1144775.9 1144775.9     10.6      ((t := (M_pos_t[:, None] - G_pos_t)) < window).logical_and(t > -window)
    89         1    1358843.5 1358843.5     12.5      ((t := (G_pos_t[:, None] - M_pos_t)) < window).logical_and(t > -window)
Setup code ``` 60 1 0.2 0.2 0.0 M_annot = ( 61 1 1005.2 1005.2 0.0 M_annot.drop(columns=['chromEnd', 'score', 'strand']) 62 1 2547.3 2547.3 0.0 .reindex(M.index) 63 1 224.8 224.8 0.0 .replace({'X': -1, 'Y': -2}) 64 ) 65 1 0.2 0.2 0.0 G_annot = ( 66 1 657.5 657.5 0.0 G_annot.drop(columns=['chromEnd', 'score', 'strand']) 67 1 2669.6 2669.6 0.0 .reindex(G.index) 68 1 190.3 190.3 0.0 .replace({'X': -1, 'Y': -2}) 69 ) 70 1 21.7 21.7 0.0 M_chrom, M_pos = M_annot.to_numpy().T 71 1 13.9 13.9 0.0 G_chrom, G_pos = G_annot.to_numpy().T 72 1 761.6 761.6 0.0 M_chrom_t = torch.tensor(M_chrom, device=device, dtype=int) 73 1 49.5 49.5 0.0 M_pos_t = torch.tensor(M_pos, device=device, dtype=int) 74 1 29.9 29.9 0.0 G_chrom_t = torch.tensor(G_chrom, device=device, dtype=int) 75 1 32.5 32.5 0.0 G_pos_t = torch.tensor(G_pos, device=device, dtype=int) ```

From this, we gather that the fastest chromosomal filtration is:

G_chrom_t[:, None] == M_chrom_t

And the fastest chromStart position filtration within a window is:

(G_pos_t[:, None] < M_pos_t + window).logical_and(G_pos_t[:, None] > M_pos_t - window)
liamgd commented 1 year ago

Tests of speed of combining these three boolean matrices with with 300 samples, 50,000 methylation loci, and 10,000 gene expression loci.

    81         1     812456.0 812456.0     30.3      (G_chrom_t[:, None] == M_chrom_t).logical_and(G_pos_t[:, None] < M_pos_t + window).logical_and(G_pos_t[:, None] > M_pos_t - window)
    82         1     895275.5 895275.5     33.4      (G_chrom_t[:, None] == M_chrom_t) + (G_pos_t[:, None] < M_pos_t + window) + (G_pos_t[:, None] > M_pos_t - window)
    83         1     957611.4 957611.4     35.8      torch.stack((G_chrom_t[:, None] == M_chrom_t, G_pos_t[:, None] < M_pos_t + window, G_pos_t[:, None] > M_pos_t - window)).sum(dim=0, dtype=bool)

A series of logical_ands is the fastest:

(G_chrom_t[:, None] == M_chrom_t).logical_and(G_pos_t[:, None] < M_pos_t + window).logical_and(G_pos_t[:, None] > M_pos_t - window)

(This example snippet is for a cis analysis, but is very similar for distal. Trans analyses would include only the first expression before the first logical_and)

liamgd commented 1 year ago

Current region filtration:

if p_thresh is not None:
                if region == 'all':
                    indices = P[:, 1] <= p_thresh
                elif region == 'cis':
                    indices = (
                        (P[:, 1] <= p_thresh)
                        .logical_and(M_chrom_t[index - 1, None] == G_chrom_t)
                        .logical_and(
                            M_pos_t[index - 1, None] < G_pos_t + window
                        )
                        .logical_and(
                            M_pos_t[index - 1, None] > G_pos_t - window
                        )
                    )
                elif region == 'distal':
                    indices = (
                        (P[:, 1] <= p_thresh)
                        .logical_and(M_chrom_t[index - 1, None] == G_chrom_t)
                        .logical_and(
                            (
                                M_pos_t[index - 1, None] < G_pos_t - window
                            ).logical_or(
                                M_pos_t[index - 1, None] > G_pos_t + window
                            )
                        )
                    )
                elif region == 'trans':
                    indices = (P[:, 1] <= p_thresh).logical_and(
                        M_chrom_t[index - 1, None] != G_chrom_t
                    )
            else:
                if region == 'cis':
                    indices = (
                        (M_chrom_t[index - 1, None] == G_chrom_t)
                        .logical_and(
                            M_pos_t[index - 1, None] < G_pos_t + window
                        )
                        .logical_and(
                            M_pos_t[index - 1, None] > G_pos_t - window
                        )
                    )
                elif region == 'distal':
                    indices = (
                        M_chrom_t[index - 1, None] == G_chrom_t
                    ).logical_and(
                        (
                            M_pos_t[index - 1, None] < G_pos_t - window
                        ).logical_or(
                            M_pos_t[index - 1, None] > G_pos_t + window
                        )
                    )
                elif region == 'trans':
                    indices = M_chrom_t[index - 1, None] != G_chrom_t
            if indices_list is not None:
                indices_list.append(indices)
                P = P[indices]
                output_sizes.append(len(P))
            results.append(P)

It is important to note that while this may seem overly repetitive and verbose, this is the fastest way I found region filtration could be implemented. This implementation prevents unnecessary variable assignment which seems to slow the algorithm significantly.

For example, the code would be shorter if filtration tensors were generated for p-value filtration, chromosome filtration, and location window filtration. Then, element-wise boolean algebra dependent on the region and p-value threshold would determine the indices. Unfortunately, this seems to be much slower than simply enumerating every combination of region and p-value.

liamgd commented 1 year ago
Tests with 100 samples, 100 methylation loci, and 100 gene expression loci:
tecpg run mlr-full ```python [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.0056 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.0091 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 0.0098 seconds [INFOTIMER] Finished reading 3 dataframes in 0.0245 seconds. [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 96 degrees of freedom [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0003 seconds [INFOTIMER] Converted G to tensor in 0.0002 seconds [INFOTIMER] Created ones in 0.0001 seconds [INFOTIMER] Created X in 0.0001 seconds [INFOTIMER] Transposed X in 0.0 seconds [INFOTIMER] Calculated XtXi in 0.5592 seconds [INFOTIMER] Calculated XtXi_diag in 0.0001 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0001 seconds [INFOTIMER] Calculated X constants in 0.5601 seconds [INFO] CUDA device memory: 0.324096 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOTIMER] Looped over methylation loci in 0.0885 seconds [INFOTIMER] Calculated regression_full in 0.6487 seconds [INFO] Generating dataframe from results... [INFOTIMER] Finished creating indices in 0.0002 seconds [INFOTIMER] Finished creating preliminary dataframe in 0.0028 seconds [INFOTIMER] Created output dataframe in 0.003 total seconds [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Saving 1 dataframes... [INFOTIMER] Saving 1/1: out.csv [INFOTIMER] Saved 1/1 in 0.0324 seconds [INFOTIMER] Finished saving 1 dataframes in 0.0325 seconds. ```
tecpg run mlr-full --cis ```python [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.0017 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.0048 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 0.0045 seconds [INFOTIMER] Finished reading 3 dataframes in 0.0112 seconds. [INFO] No region window provided. Resorting to default. [INFO] Using default window for cis of 1000000 bases [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 96 degrees of freedom [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0016 seconds [INFOTIMER] Converted G to tensor in 0.0002 seconds [INFOTIMER] Created ones in 0.0001 seconds [INFOTIMER] Created X in 0.0001 seconds [INFOTIMER] Transposed X in 0.0 seconds [INFOTIMER] Calculated XtXi in 0.6096 seconds [INFOTIMER] Calculated XtXi_diag in 0.0001 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0001 seconds [INFOTIMER] Calculated X constants in 0.6119 seconds [INFO] CUDA device memory: 0.326144 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOTIMER] Looped over methylation loci in 0.129 seconds [INFOTIMER] Calculated regression_full in 0.741 seconds [INFO] Generating dataframe from results... [INFOTIMER] Finished creating indices in 0.0003 seconds [INFOTIMER] Finished creating preliminary dataframe in 0.0013 seconds [INFOTIMER] Created output dataframe in 0.0017 total seconds [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Saving 1 dataframes... [INFOTIMER] Saving 1/1: out.csv [INFOTIMER] Saved 1/1 in 0.0016 seconds [INFOTIMER] Finished saving 1 dataframes in 0.0016 seconds. ```
tecpg run mlr-full --distal ```python [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.0017 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.005 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 0.0042 seconds [INFOTIMER] Finished reading 3 dataframes in 0.011 seconds. [INFO] No region window provided. Resorting to default. [INFO] Using default window for distal of 50000 bases [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 96 degrees of freedom [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0015 seconds [INFOTIMER] Converted G to tensor in 0.0001 seconds [INFOTIMER] Created ones in 0.0001 seconds [INFOTIMER] Created X in 0.0001 seconds [INFOTIMER] Transposed X in 0.0 seconds [INFOTIMER] Calculated XtXi in 0.5569 seconds [INFOTIMER] Calculated XtXi_diag in 0.0001 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0001 seconds [INFOTIMER] Calculated X constants in 0.5589 seconds [INFO] CUDA device memory: 0.326144 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOTIMER] Looped over methylation loci in 0.1325 seconds [INFOTIMER] Calculated regression_full in 0.6915 seconds [INFO] Generating dataframe from results... [INFOTIMER] Finished creating indices in 0.0003 seconds [INFOTIMER] Finished creating preliminary dataframe in 0.0017 seconds [INFOTIMER] Created output dataframe in 0.002 total seconds [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Saving 1 dataframes... [INFOTIMER] Saving 1/1: out.csv [INFOTIMER] Saved 1/1 in 0.0031 seconds [INFOTIMER] Finished saving 1 dataframes in 0.0031 seconds. ```
tecpg run mlr-full --trans ```python [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.0018 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.0052 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 0.0047 seconds [INFOTIMER] Finished reading 3 dataframes in 0.0117 seconds. [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 96 degrees of freedom [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0016 seconds [INFOTIMER] Converted G to tensor in 0.0002 seconds [INFOTIMER] Created ones in 0.0001 seconds [INFOTIMER] Created X in 0.0001 seconds [INFOTIMER] Transposed X in 0.0 seconds [INFOTIMER] Calculated XtXi in 0.5704 seconds [INFOTIMER] Calculated XtXi_diag in 0.0001 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0001 seconds [INFOTIMER] Calculated X constants in 0.5726 seconds [INFO] CUDA device memory: 0.326144 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOTIMER] Looped over methylation loci in 0.1103 seconds [INFOTIMER] Calculated regression_full in 0.683 seconds [INFO] Generating dataframe from results... [INFOTIMER] Finished creating indices in 0.0004 seconds [INFOTIMER] Finished creating preliminary dataframe in 0.0028 seconds [INFOTIMER] Created output dataframe in 0.0032 total seconds [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Saving 1 dataframes... [INFOTIMER] Saving 1/1: out.csv [INFOTIMER] Saved 1/1 in 0.0309 seconds [INFOTIMER] Finished saving 1 dataframes in 0.031 seconds. ```
tecpg run mlr-full -p 0.05 ```python [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.0017 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.0048 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 0.0045 seconds [INFOTIMER] Finished reading 3 dataframes in 0.0111 seconds. [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 96 degrees of freedom [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0003 seconds [INFOTIMER] Converted G to tensor in 0.0002 seconds [INFOTIMER] Created ones in 0.0001 seconds [INFOTIMER] Created X in 0.0001 seconds [INFOTIMER] Transposed X in 0.0 seconds [INFOTIMER] Calculated XtXi in 0.5931 seconds [INFOTIMER] Calculated XtXi_diag in 0.0001 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0001 seconds [INFOTIMER] Calculated X constants in 0.594 seconds [INFO] CUDA device memory: 0.324096 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOTIMER] Looped over methylation loci in 0.1117 seconds [INFOTIMER] Calculated regression_full in 0.7057 seconds [INFO] Generating dataframe from results... [INFOTIMER] Finished creating indices in 0.0003 seconds [INFOTIMER] Finished creating preliminary dataframe in 0.0016 seconds [INFOTIMER] Created output dataframe in 0.002 total seconds [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Saving 1 dataframes... [INFOTIMER] Saving 1/1: out.csv [INFOTIMER] Saved 1/1 in 0.003 seconds [INFOTIMER] Finished saving 1 dataframes in 0.003 seconds. ```
tecpg run mlr-full -p 0.05 --cis ```python [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.0018 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.0052 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 0.0047 seconds [INFOTIMER] Finished reading 3 dataframes in 0.0118 seconds. [INFO] No region window provided. Resorting to default. [INFO] Using default window for cis of 1000000 bases [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 96 degrees of freedom [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0015 seconds [INFOTIMER] Converted G to tensor in 0.0002 seconds [INFOTIMER] Created ones in 0.0001 seconds [INFOTIMER] Created X in 0.0001 seconds [INFOTIMER] Transposed X in 0.0 seconds [INFOTIMER] Calculated XtXi in 0.5787 seconds [INFOTIMER] Calculated XtXi_diag in 0.0001 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0001 seconds [INFOTIMER] Calculated X constants in 0.5808 seconds [INFO] CUDA device memory: 0.326144 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOTIMER] Looped over methylation loci in 0.1368 seconds [INFOTIMER] Calculated regression_full in 0.7177 seconds [INFO] Generating dataframe from results... [INFOTIMER] Finished creating indices in 0.0003 seconds [INFOTIMER] Finished creating preliminary dataframe in 0.0012 seconds [INFOTIMER] Created output dataframe in 0.0015 total seconds [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Saving 1 dataframes... [INFOTIMER] Saving 1/1: out.csv [INFOTIMER] Saved 1/1 in 0.0012 seconds [INFOTIMER] Finished saving 1 dataframes in 0.0013 seconds. ```
tecpg run mlr-full -p 0.05 --distal ```python [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.0019 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.005 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 0.0043 seconds [INFOTIMER] Finished reading 3 dataframes in 0.0112 seconds. [INFO] No region window provided. Resorting to default. [INFO] Using default window for distal of 50000 bases [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 96 degrees of freedom [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0015 seconds [INFOTIMER] Converted G to tensor in 0.0001 seconds [INFOTIMER] Created ones in 0.0001 seconds [INFOTIMER] Created X in 0.0001 seconds [INFOTIMER] Transposed X in 0.0 seconds [INFOTIMER] Calculated XtXi in 0.5698 seconds [INFOTIMER] Calculated XtXi_diag in 0.0001 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0001 seconds [INFOTIMER] Calculated X constants in 0.5718 seconds [INFO] CUDA device memory: 0.326144 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOTIMER] Looped over methylation loci in 0.1387 seconds [INFOTIMER] Calculated regression_full in 0.7106 seconds [INFO] Generating dataframe from results... [INFOTIMER] Finished creating indices in 0.0003 seconds [INFOTIMER] Finished creating preliminary dataframe in 0.0013 seconds [INFOTIMER] Created output dataframe in 0.0017 total seconds [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Saving 1 dataframes... [INFOTIMER] Saving 1/1: out.csv [INFOTIMER] Saved 1/1 in 0.0017 seconds [INFOTIMER] Finished saving 1 dataframes in 0.0017 seconds. ```
tecpg run mlr-full -p 0.05 --trans ```python [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.0018 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.0048 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 0.0047 seconds [INFOTIMER] Finished reading 3 dataframes in 0.0113 seconds. [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 96 degrees of freedom [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0016 seconds [INFOTIMER] Converted G to tensor in 0.0002 seconds [INFOTIMER] Created ones in 0.0001 seconds [INFOTIMER] Created X in 0.0001 seconds [INFOTIMER] Transposed X in 0.0 seconds [INFOTIMER] Calculated XtXi in 0.6028 seconds [INFOTIMER] Calculated XtXi_diag in 0.0001 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0001 seconds [INFOTIMER] Calculated X constants in 0.6051 seconds [INFO] CUDA device memory: 0.326144 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOTIMER] Looped over methylation loci in 0.1162 seconds [INFOTIMER] Calculated regression_full in 0.7213 seconds [INFO] Generating dataframe from results... [INFOTIMER] Finished creating indices in 0.0003 seconds [INFOTIMER] Finished creating preliminary dataframe in 0.0015 seconds [INFOTIMER] Created output dataframe in 0.0019 total seconds [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Saving 1 dataframes... [INFOTIMER] Saving 1/1: out.csv [INFOTIMER] Saved 1/1 in 0.0029 seconds [INFOTIMER] Finished saving 1 dataframes in 0.003 seconds. ```
tecpg run mlr-full -l 2000 ```python [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.0019 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.0048 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 0.0042 seconds [INFOTIMER] Finished reading 3 dataframes in 0.011 seconds. [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 96 degrees of freedom [INFO] Initializing output directory [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0003 seconds [INFOTIMER] Converted G to tensor in 0.0001 seconds [INFOTIMER] Created ones in 0.0001 seconds [INFOTIMER] Created X in 0.0001 seconds [INFOTIMER] Transposed X in 0.0 seconds [INFOTIMER] Calculated XtXi in 0.5585 seconds [INFOTIMER] Calculated XtXi_diag in 0.0001 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0001 seconds [INFOTIMER] Calculated X constants in 0.5594 seconds [INFO] CUDA device memory: 0.324096 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOCOUNT] Saving part 1/1: [INFOTIMER] Completed chunk 1/100 in 0.0989 seconds. Average chunk time: 0.0989 seconds [INFOTIMER] Looped over methylation loci in 0.0998 seconds [INFOTIMER] Calculated regression_full in 0.6592 seconds [INFOTIMER] Waiting for chunks to save... [INFOCOUNT] Done saving part 1 [INFOTIMER] Finished waiting for chunks to save in 1.0995 seconds ```
tecpg run mlr-full -l 2000 --cis ```python [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.0019 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.0053 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 0.0049 seconds [INFOTIMER] Finished reading 3 dataframes in 0.0122 seconds. [INFO] No region window provided. Resorting to default. [INFO] Using default window for cis of 1000000 bases [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 96 degrees of freedom [INFO] Initializing output directory [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0015 seconds [INFOTIMER] Converted G to tensor in 0.0001 seconds [INFOTIMER] Created ones in 0.0001 seconds [INFOTIMER] Created X in 0.0001 seconds [INFOTIMER] Transposed X in 0.0 seconds [INFOTIMER] Calculated XtXi in 0.568 seconds [INFOTIMER] Calculated XtXi_diag in 0.0001 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0001 seconds [INFOTIMER] Calculated X constants in 0.57 seconds [INFO] CUDA device memory: 0.326144 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOCOUNT] Saving part 1/1: [INFOTIMER] Completed chunk 1/100 in 0.1342 seconds. Average chunk time: 0.1342 seconds [INFOTIMER] Looped over methylation loci in 0.135 seconds [INFOTIMER] Calculated regression_full in 0.705 seconds [INFOTIMER] Waiting for chunks to save... [INFOCOUNT] Done saving part 1 [INFOTIMER] Finished waiting for chunks to save in 1.0618 seconds ```
tecpg run mlr-full -l 2000 --distal ```python [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.0017 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.0052 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 0.0043 seconds [INFOTIMER] Finished reading 3 dataframes in 0.0114 seconds. [INFO] No region window provided. Resorting to default. [INFO] Using default window for distal of 50000 bases [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 96 degrees of freedom [INFO] Initializing output directory [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0015 seconds [INFOTIMER] Converted G to tensor in 0.0002 seconds [INFOTIMER] Created ones in 0.0001 seconds [INFOTIMER] Created X in 0.0001 seconds [INFOTIMER] Transposed X in 0.0 seconds [INFOTIMER] Calculated XtXi in 0.5936 seconds [INFOTIMER] Calculated XtXi_diag in 0.0002 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0001 seconds [INFOTIMER] Calculated X constants in 0.5957 seconds [INFO] CUDA device memory: 0.326144 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOCOUNT] Saving part 1/1: [INFOTIMER] Completed chunk 1/100 in 0.1604 seconds. Average chunk time: 0.1604 seconds [INFOTIMER] Looped over methylation loci in 0.1614 seconds [INFOTIMER] Calculated regression_full in 0.7571 seconds [INFOTIMER] Waiting for chunks to save... [INFOCOUNT] Done saving part 1 [INFOTIMER] Finished waiting for chunks to save in 1.0338 seconds ```
tecpg run mlr-full -l 2000 --trans ```python [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.0023 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.0049 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 0.0046 seconds [INFOTIMER] Finished reading 3 dataframes in 0.0119 seconds. [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 96 degrees of freedom [INFO] Initializing output directory [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0015 seconds [INFOTIMER] Converted G to tensor in 0.0002 seconds [INFOTIMER] Created ones in 0.0001 seconds [INFOTIMER] Created X in 0.0001 seconds [INFOTIMER] Transposed X in 0.0 seconds [INFOTIMER] Calculated XtXi in 0.5624 seconds [INFOTIMER] Calculated XtXi_diag in 0.0001 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0001 seconds [INFOTIMER] Calculated X constants in 0.5644 seconds [INFO] CUDA device memory: 0.326144 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOCOUNT] Saving part 1/1: [INFOTIMER] Completed chunk 1/100 in 0.1197 seconds. Average chunk time: 0.1197 seconds [INFOTIMER] Looped over methylation loci in 0.1208 seconds [INFOTIMER] Calculated regression_full in 0.6853 seconds [INFOTIMER] Waiting for chunks to save... [INFOCOUNT] Done saving part 1 [INFOTIMER] Finished waiting for chunks to save in 1.1015 seconds ```
tecpg run mlr-full -l 2000 -p 0.05 ```python [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.0017 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.0051 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 0.0044 seconds [INFOTIMER] Finished reading 3 dataframes in 0.0114 seconds. [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 96 degrees of freedom [INFO] Initializing output directory [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0003 seconds [INFOTIMER] Converted G to tensor in 0.0002 seconds [INFOTIMER] Created ones in 0.0001 seconds [INFOTIMER] Created X in 0.0001 seconds [INFOTIMER] Transposed X in 0.0 seconds [INFOTIMER] Calculated XtXi in 0.5587 seconds [INFOTIMER] Calculated XtXi_diag in 0.0001 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0001 seconds [INFOTIMER] Calculated X constants in 0.5597 seconds [INFO] CUDA device memory: 0.324096 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOCOUNT] Saving part 1/1: [INFOTIMER] Completed chunk 1/100 in 0.1244 seconds. Average chunk time: 0.1244 seconds [INFOTIMER] Looped over methylation loci in 0.1251 seconds [INFOTIMER] Calculated regression_full in 0.6848 seconds [INFOTIMER] Waiting for chunks to save... [INFOCOUNT] Done saving part 1 [INFOTIMER] Finished waiting for chunks to save in 1.0988 seconds ```
tecpg run mlr-full -l 2000 -p 0.05 --cis ```python [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.0018 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.0047 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 0.0042 seconds [INFOTIMER] Finished reading 3 dataframes in 0.0107 seconds. [INFO] No region window provided. Resorting to default. [INFO] Using default window for cis of 1000000 bases [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 96 degrees of freedom [INFO] Initializing output directory [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0015 seconds [INFOTIMER] Converted G to tensor in 0.0001 seconds [INFOTIMER] Created ones in 0.0001 seconds [INFOTIMER] Created X in 0.0001 seconds [INFOTIMER] Transposed X in 0.0 seconds [INFOTIMER] Calculated XtXi in 0.5619 seconds [INFOTIMER] Calculated XtXi_diag in 0.0001 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0001 seconds [INFOTIMER] Calculated X constants in 0.5639 seconds [INFO] CUDA device memory: 0.326144 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOCOUNT] Saving part 1/1: [INFOTIMER] Completed chunk 1/100 in 0.1499 seconds. Average chunk time: 0.1499 seconds [INFOTIMER] Looped over methylation loci in 0.1502 seconds [INFOTIMER] Calculated regression_full in 0.7141 seconds [INFOTIMER] Waiting for chunks to save... [INFOCOUNT] Done saving part 1 [INFOTIMER] Finished waiting for chunks to save in 1.0279 seconds ```
tecpg run mlr-full -l 2000 -p 0.05 --distal ```python [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.0018 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.0048 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 0.0042 seconds [INFOTIMER] Finished reading 3 dataframes in 0.0108 seconds. [INFO] No region window provided. Resorting to default. [INFO] Using default window for distal of 50000 bases [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 96 degrees of freedom [INFO] Initializing output directory [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0015 seconds [INFOTIMER] Converted G to tensor in 0.0002 seconds [INFOTIMER] Created ones in 0.0001 seconds [INFOTIMER] Created X in 0.0001 seconds [INFOTIMER] Transposed X in 0.0 seconds [INFOTIMER] Calculated XtXi in 0.5545 seconds [INFOTIMER] Calculated XtXi_diag in 0.0001 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0001 seconds [INFOTIMER] Calculated X constants in 0.5566 seconds [INFO] CUDA device memory: 0.326144 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOCOUNT] Saving part 1/1: [INFOTIMER] Completed chunk 1/100 in 0.1461 seconds. Average chunk time: 0.1461 seconds [INFOTIMER] Looped over methylation loci in 0.1469 seconds [INFOTIMER] Calculated regression_full in 0.7035 seconds [INFOTIMER] Waiting for chunks to save... [INFOCOUNT] Done saving part 1 [INFOTIMER] Finished waiting for chunks to save in 1.0387 seconds ```
tecpg run mlr-full -l 2000 -p 0.05 --trans ```python [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.0018 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.0053 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 0.0047 seconds [INFOTIMER] Finished reading 3 dataframes in 0.0119 seconds. [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 96 degrees of freedom [INFO] Initializing output directory [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0015 seconds [INFOTIMER] Converted G to tensor in 0.0001 seconds [INFOTIMER] Created ones in 0.0001 seconds [INFOTIMER] Created X in 0.0001 seconds [INFOTIMER] Transposed X in 0.0 seconds [INFOTIMER] Calculated XtXi in 0.56 seconds [INFOTIMER] Calculated XtXi_diag in 0.0001 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0001 seconds [INFOTIMER] Calculated X constants in 0.562 seconds [INFO] CUDA device memory: 0.326144 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOCOUNT] Saving part 1/1: [INFOTIMER] Completed chunk 1/100 in 0.1265 seconds. Average chunk time: 0.1265 seconds [INFOTIMER] Looped over methylation loci in 0.1273 seconds [INFOTIMER] Calculated regression_full in 0.6894 seconds [INFOTIMER] Waiting for chunks to save... [INFOCOUNT] Done saving part 1 [INFOTIMER] Finished waiting for chunks to save in 1.0443 seconds ```
Tests with 300 samples, 1000 methylation loci, and 1000 gene expression loci:
tecpg run mlr-full ```python [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.0059 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.055 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 0.0528 seconds [INFOTIMER] Finished reading 3 dataframes in 0.1138 seconds. [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 296 degrees of freedom [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0005 seconds [INFOTIMER] Converted G to tensor in 0.0022 seconds [INFOTIMER] Created ones in 0.0001 seconds [INFOTIMER] Created X in 0.0001 seconds [INFOTIMER] Transposed X in 0.0 seconds [INFOTIMER] Calculated XtXi in 0.6634 seconds [INFOTIMER] Calculated XtXi_diag in 0.0001 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0001 seconds [INFOTIMER] Calculated X constants in 0.6666 seconds [INFO] CUDA device memory: 9.618432 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOTIMER] Looped over methylation loci in 0.4429 seconds [INFOTIMER] Calculated regression_full in 1.1094 seconds [INFO] Generating dataframe from results... [INFOTIMER] Finished creating indices in 0.0059 seconds [INFOTIMER] Finished creating preliminary dataframe in 0.0731 seconds [INFOTIMER] Created output dataframe in 0.0791 total seconds [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Saving 1 dataframes... [INFOTIMER] Saving 1/1: out.csv [INFOTIMER] Saved 1/1 in 3.3045 seconds [INFOTIMER] Finished saving 1 dataframes in 3.3046 seconds. ```
tecpg run mlr-full --cis ```python [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.002 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.052 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 0.0497 seconds [INFOTIMER] Finished reading 3 dataframes in 0.1038 seconds. [INFO] No region window provided. Resorting to default. [INFO] Using default window for cis of 1000000 bases [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 296 degrees of freedom [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.002 seconds [INFOTIMER] Converted G to tensor in 0.002 seconds [INFOTIMER] Created ones in 0.0001 seconds [INFOTIMER] Created X in 0.0001 seconds [INFOTIMER] Transposed X in 0.0 seconds [INFOTIMER] Calculated XtXi in 0.6498 seconds [INFOTIMER] Calculated XtXi_diag in 0.0001 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0001 seconds [INFOTIMER] Calculated X constants in 0.6544 seconds [INFO] CUDA device memory: 9.634816 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOTIMER] Looped over methylation loci in 0.6735 seconds [INFOTIMER] Calculated regression_full in 1.3279 seconds [INFO] Generating dataframe from results... [INFOTIMER] Finished creating indices in 0.0047 seconds [INFOTIMER] Finished creating preliminary dataframe in 0.0016 seconds [INFOTIMER] Created output dataframe in 0.0063 total seconds [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Saving 1 dataframes... [INFOTIMER] Saving 1/1: out.csv [INFOTIMER] Saved 1/1 in 0.0041 seconds [INFOTIMER] Finished saving 1 dataframes in 0.0042 seconds. ```
tecpg run mlr-full --distal ```python [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.0019 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.0513 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 0.0504 seconds [INFOTIMER] Finished reading 3 dataframes in 0.1036 seconds. [INFO] No region window provided. Resorting to default. [INFO] Using default window for distal of 50000 bases [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 296 degrees of freedom [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0019 seconds [INFOTIMER] Converted G to tensor in 0.0017 seconds [INFOTIMER] Created ones in 0.0001 seconds [INFOTIMER] Created X in 0.0001 seconds [INFOTIMER] Transposed X in 0.0 seconds [INFOTIMER] Calculated XtXi in 0.5943 seconds [INFOTIMER] Calculated XtXi_diag in 0.0001 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0001 seconds [INFOTIMER] Calculated X constants in 0.5984 seconds [INFO] CUDA device memory: 9.634816 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOTIMER] Looped over methylation loci in 0.6362 seconds [INFOTIMER] Calculated regression_full in 1.2347 seconds [INFO] Generating dataframe from results... [INFOTIMER] Finished creating indices in 0.0053 seconds [INFOTIMER] Finished creating preliminary dataframe in 0.0053 seconds [INFOTIMER] Created output dataframe in 0.0106 total seconds [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Saving 1 dataframes... [INFOTIMER] Saving 1/1: out.csv [INFOTIMER] Saved 1/1 in 0.1357 seconds [INFOTIMER] Finished saving 1 dataframes in 0.1358 seconds. ```
tecpg run mlr-full --trans ```python [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.0018 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.05 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 0.0484 seconds [INFOTIMER] Finished reading 3 dataframes in 0.1004 seconds. [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 296 degrees of freedom [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0018 seconds [INFOTIMER] Converted G to tensor in 0.0017 seconds [INFOTIMER] Created ones in 0.0001 seconds [INFOTIMER] Created X in 0.0001 seconds [INFOTIMER] Transposed X in 0.0001 seconds [INFOTIMER] Calculated XtXi in 0.6409 seconds [INFOTIMER] Calculated XtXi_diag in 0.0001 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0001 seconds [INFOTIMER] Calculated X constants in 0.6449 seconds [INFO] CUDA device memory: 9.634816 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOTIMER] Looped over methylation loci in 0.5267 seconds [INFOTIMER] Calculated regression_full in 1.1716 seconds [INFO] Generating dataframe from results... [INFOTIMER] Finished creating indices in 0.0111 seconds [INFOTIMER] Finished creating preliminary dataframe in 0.0726 seconds [INFOTIMER] Created output dataframe in 0.0837 total seconds [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Saving 1 dataframes... [INFOTIMER] Saving 1/1: out.csv [INFOTIMER] Saved 1/1 in 3.0954 seconds [INFOTIMER] Finished saving 1 dataframes in 3.0954 seconds. ```
tecpg run mlr-full -p 0.05 ```python [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.0019 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.0769 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 0.0734 seconds [INFOTIMER] Finished reading 3 dataframes in 0.1522 seconds. [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 296 degrees of freedom [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0007 seconds [INFOTIMER] Converted G to tensor in 0.0016 seconds [INFOTIMER] Created ones in 0.0001 seconds [INFOTIMER] Created X in 0.0001 seconds [INFOTIMER] Transposed X in 0.0 seconds [INFOTIMER] Calculated XtXi in 0.6535 seconds [INFOTIMER] Calculated XtXi_diag in 0.0001 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0001 seconds [INFOTIMER] Calculated X constants in 0.6562 seconds [INFO] CUDA device memory: 9.618432 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOTIMER] Looped over methylation loci in 0.5736 seconds [INFOTIMER] Calculated regression_full in 1.2299 seconds [INFO] Generating dataframe from results... [INFOTIMER] Finished creating indices in 0.0058 seconds [INFOTIMER] Finished creating preliminary dataframe in 0.0054 seconds [INFOTIMER] Created output dataframe in 0.0112 total seconds [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Saving 1 dataframes... [INFOTIMER] Saving 1/1: out.csv [INFOTIMER] Saved 1/1 in 0.145 seconds [INFOTIMER] Finished saving 1 dataframes in 0.145 seconds. ```
tecpg run mlr-full -p 0.05 --cis ```python [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.002 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.0517 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 0.0544 seconds [INFOTIMER] Finished reading 3 dataframes in 0.1081 seconds. [INFO] No region window provided. Resorting to default. [INFO] Using default window for cis of 1000000 bases [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 296 degrees of freedom [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0017 seconds [INFOTIMER] Converted G to tensor in 0.002 seconds [INFOTIMER] Created ones in 0.0001 seconds [INFOTIMER] Created X in 0.0001 seconds [INFOTIMER] Transposed X in 0.0001 seconds [INFOTIMER] Calculated XtXi in 0.6116 seconds [INFOTIMER] Calculated XtXi_diag in 0.0001 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0001 seconds [INFOTIMER] Calculated X constants in 0.6158 seconds [INFO] CUDA device memory: 9.634816 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOTIMER] Looped over methylation loci in 0.6727 seconds [INFOTIMER] Calculated regression_full in 1.2886 seconds [INFO] Generating dataframe from results... [INFOTIMER] Finished creating indices in 0.0045 seconds [INFOTIMER] Finished creating preliminary dataframe in 0.0013 seconds [INFOTIMER] Created output dataframe in 0.0058 total seconds [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Saving 1 dataframes... [INFOTIMER] Saving 1/1: out.csv [INFOTIMER] Saved 1/1 in 0.0018 seconds [INFOTIMER] Finished saving 1 dataframes in 0.0018 seconds. ```
tecpg run mlr-full -p 0.05 --distal ```python [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.0018 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.0494 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 0.0493 seconds [INFOTIMER] Finished reading 3 dataframes in 0.1006 seconds. [INFO] No region window provided. Resorting to default. [INFO] Using default window for distal of 50000 bases [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 296 degrees of freedom [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0018 seconds [INFOTIMER] Converted G to tensor in 0.0017 seconds [INFOTIMER] Created ones in 0.0001 seconds [INFOTIMER] Created X in 0.0001 seconds [INFOTIMER] Transposed X in 0.0 seconds [INFOTIMER] Calculated XtXi in 0.5734 seconds [INFOTIMER] Calculated XtXi_diag in 0.0001 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0001 seconds [INFOTIMER] Calculated X constants in 0.5773 seconds [INFO] CUDA device memory: 9.634816 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOTIMER] Looped over methylation loci in 0.6855 seconds [INFOTIMER] Calculated regression_full in 1.2628 seconds [INFO] Generating dataframe from results... [INFOTIMER] Finished creating indices in 0.0045 seconds [INFOTIMER] Finished creating preliminary dataframe in 0.0019 seconds [INFOTIMER] Created output dataframe in 0.0063 total seconds [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Saving 1 dataframes... [INFOTIMER] Saving 1/1: out.csv [INFOTIMER] Saved 1/1 in 0.0079 seconds [INFOTIMER] Finished saving 1 dataframes in 0.008 seconds. ```
tecpg run mlr-full -p 0.05 --trans ```python [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.002 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.0517 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 0.0489 seconds [INFOTIMER] Finished reading 3 dataframes in 0.1027 seconds. [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 296 degrees of freedom [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0018 seconds [INFOTIMER] Converted G to tensor in 0.0018 seconds [INFOTIMER] Created ones in 0.0001 seconds [INFOTIMER] Created X in 0.0001 seconds [INFOTIMER] Transposed X in 0.0 seconds [INFOTIMER] Calculated XtXi in 0.6111 seconds [INFOTIMER] Calculated XtXi_diag in 0.0001 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0001 seconds [INFOTIMER] Calculated X constants in 0.6151 seconds [INFO] CUDA device memory: 9.634816 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOTIMER] Looped over methylation loci in 0.5693 seconds [INFOTIMER] Calculated regression_full in 1.1845 seconds [INFO] Generating dataframe from results... [INFOTIMER] Finished creating indices in 0.0054 seconds [INFOTIMER] Finished creating preliminary dataframe in 0.0053 seconds [INFOTIMER] Created output dataframe in 0.0107 total seconds [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Saving 1 dataframes... [INFOTIMER] Saving 1/1: out.csv [INFOTIMER] Saved 1/1 in 0.1317 seconds [INFOTIMER] Finished saving 1 dataframes in 0.1317 seconds. ```
tecpg run mlr-full -l 2000 ```python [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.002 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.0565 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 0.0509 seconds [INFOTIMER] Finished reading 3 dataframes in 0.1094 seconds. [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 296 degrees of freedom [INFO] Initializing output directory [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0005 seconds [INFOTIMER] Converted G to tensor in 0.0017 seconds [INFOTIMER] Created ones in 0.0001 seconds [INFOTIMER] Created X in 0.0001 seconds [INFOTIMER] Transposed X in 0.0 seconds [INFOTIMER] Calculated XtXi in 0.6057 seconds [INFOTIMER] Calculated XtXi_diag in 0.0001 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0001 seconds [INFOTIMER] Calculated X constants in 0.6083 seconds [INFO] CUDA device memory: 9.618432 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOCOUNT] Saving part 1/1: [INFOTIMER] Completed chunk 1/1000 in 0.5241 seconds. Average chunk time: 0.5241 seconds [INFOTIMER] Looped over methylation loci in 0.5249 seconds [INFOTIMER] Calculated regression_full in 1.1332 seconds [INFOTIMER] Waiting for chunks to save... [INFOCOUNT] Done saving part 1 [INFOTIMER] Finished waiting for chunks to save in 4.2539 seconds ```
tecpg run mlr-full -l 2000 --cis ```python [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.0019 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.0539 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 0.0554 seconds [INFOTIMER] Finished reading 3 dataframes in 0.1113 seconds. [INFO] No region window provided. Resorting to default. [INFO] Using default window for cis of 1000000 bases [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 296 degrees of freedom [INFO] Initializing output directory [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0018 seconds [INFOTIMER] Converted G to tensor in 0.002 seconds [INFOTIMER] Created ones in 0.0001 seconds [INFOTIMER] Created X in 0.0001 seconds [INFOTIMER] Transposed X in 0.0 seconds [INFOTIMER] Calculated XtXi in 0.6347 seconds [INFOTIMER] Calculated XtXi_diag in 0.0001 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0001 seconds [INFOTIMER] Calculated X constants in 0.6391 seconds [INFO] CUDA device memory: 9.634816 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOCOUNT] Saving part 1/1: [INFOTIMER] Completed chunk 1/1000 in 0.668 seconds. Average chunk time: 0.668 seconds [INFOTIMER] Looped over methylation loci in 0.6691 seconds [INFOTIMER] Calculated regression_full in 1.3082 seconds [INFOTIMER] Waiting for chunks to save... [INFOCOUNT] Done saving part 1 [INFOTIMER] Finished waiting for chunks to save in 0.9911 seconds ```
tecpg run mlr-full -l 2000 --distal ```python [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.0021 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.0506 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 0.0516 seconds [INFOTIMER] Finished reading 3 dataframes in 0.1044 seconds. [INFO] No region window provided. Resorting to default. [INFO] Using default window for distal of 50000 bases [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 296 degrees of freedom [INFO] Initializing output directory [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0018 seconds [INFOTIMER] Converted G to tensor in 0.0017 seconds [INFOTIMER] Created ones in 0.0001 seconds [INFOTIMER] Created X in 0.0001 seconds [INFOTIMER] Transposed X in 0.0 seconds [INFOTIMER] Calculated XtXi in 0.6245 seconds [INFOTIMER] Calculated XtXi_diag in 0.0001 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0001 seconds [INFOTIMER] Calculated X constants in 0.6283 seconds [INFO] CUDA device memory: 9.634816 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOCOUNT] Saving part 1/1: [INFOTIMER] Completed chunk 1/1000 in 0.6587 seconds. Average chunk time: 0.6587 seconds [INFOTIMER] Looped over methylation loci in 0.6602 seconds [INFOTIMER] Calculated regression_full in 1.2886 seconds [INFOTIMER] Waiting for chunks to save... [INFOCOUNT] Done saving part 1 [INFOTIMER] Finished waiting for chunks to save in 1.1118 seconds ```
tecpg run mlr-full -l 2000 --trans ```python [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.0019 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.054 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 0.0512 seconds [INFOTIMER] Finished reading 3 dataframes in 0.1073 seconds. [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 296 degrees of freedom [INFO] Initializing output directory [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0018 seconds [INFOTIMER] Converted G to tensor in 0.0015 seconds [INFOTIMER] Created ones in 0.0001 seconds [INFOTIMER] Created X in 0.0001 seconds [INFOTIMER] Transposed X in 0.0 seconds [INFOTIMER] Calculated XtXi in 0.5979 seconds [INFOTIMER] Calculated XtXi_diag in 0.0001 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0001 seconds [INFOTIMER] Calculated X constants in 0.6017 seconds [INFO] CUDA device memory: 9.634816 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOCOUNT] Saving part 1/1: [INFOTIMER] Completed chunk 1/1000 in 0.6314 seconds. Average chunk time: 0.6314 seconds [INFOTIMER] Looped over methylation loci in 0.6468 seconds [INFOTIMER] Calculated regression_full in 1.2486 seconds [INFOTIMER] Waiting for chunks to save... [INFOCOUNT] Done saving part 1 [INFOTIMER] Finished waiting for chunks to save in 4.0747 seconds ```
tecpg run mlr-full -l 2000 -p 0.05 ```python [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.0019 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.051 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 0.055 seconds [INFOTIMER] Finished reading 3 dataframes in 0.1081 seconds. [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 296 degrees of freedom [INFO] Initializing output directory [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0005 seconds [INFOTIMER] Converted G to tensor in 0.0019 seconds [INFOTIMER] Created ones in 0.0001 seconds [INFOTIMER] Created X in 0.0001 seconds [INFOTIMER] Transposed X in 0.0 seconds [INFOTIMER] Calculated XtXi in 0.6448 seconds [INFOTIMER] Calculated XtXi_diag in 0.0001 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0001 seconds [INFOTIMER] Calculated X constants in 0.6478 seconds [INFO] CUDA device memory: 9.618432 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOCOUNT] Saving part 1/1: [INFOTIMER] Completed chunk 1/1000 in 0.5562 seconds. Average chunk time: 0.5562 seconds [INFOTIMER] Looped over methylation loci in 0.5586 seconds [INFOTIMER] Calculated regression_full in 1.2064 seconds [INFOTIMER] Waiting for chunks to save... [INFOCOUNT] Done saving part 1 [INFOTIMER] Finished waiting for chunks to save in 1.2202 seconds ```
tecpg run mlr-full -l 2000 -p 0.05 --cis ```python [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.0022 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.0501 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 0.0504 seconds [INFOTIMER] Finished reading 3 dataframes in 0.1028 seconds. [INFO] No region window provided. Resorting to default. [INFO] Using default window for cis of 1000000 bases [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 296 degrees of freedom [INFO] Initializing output directory [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0018 seconds [INFOTIMER] Converted G to tensor in 0.0021 seconds [INFOTIMER] Created ones in 0.0001 seconds [INFOTIMER] Created X in 0.0001 seconds [INFOTIMER] Transposed X in 0.0 seconds [INFOTIMER] Calculated XtXi in 0.6158 seconds [INFOTIMER] Calculated XtXi_diag in 0.0001 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0001 seconds [INFOTIMER] Calculated X constants in 0.6202 seconds [INFO] CUDA device memory: 9.634816 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOCOUNT] Saving part 1/1: [INFOTIMER] Completed chunk 1/1000 in 0.7247 seconds. Average chunk time: 0.7247 seconds [INFOTIMER] Looped over methylation loci in 0.7257 seconds [INFOTIMER] Calculated regression_full in 1.3459 seconds [INFOTIMER] Waiting for chunks to save... [INFOCOUNT] Done saving part 1 [INFOTIMER] Finished waiting for chunks to save in 0.9792 seconds ```
tecpg run mlr-full -l 2000 -p 0.05 --distal ```python [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.0021 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.0493 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 0.0494 seconds [INFOTIMER] Finished reading 3 dataframes in 0.101 seconds. [INFO] No region window provided. Resorting to default. [INFO] Using default window for distal of 50000 bases [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 296 degrees of freedom [INFO] Initializing output directory [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0018 seconds [INFOTIMER] Converted G to tensor in 0.0017 seconds [INFOTIMER] Created ones in 0.0001 seconds [INFOTIMER] Created X in 0.0001 seconds [INFOTIMER] Transposed X in 0.0 seconds [INFOTIMER] Calculated XtXi in 0.6146 seconds [INFOTIMER] Calculated XtXi_diag in 0.0001 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0001 seconds [INFOTIMER] Calculated X constants in 0.6185 seconds [INFO] CUDA device memory: 9.634816 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOCOUNT] Saving part 1/1: [INFOTIMER] Completed chunk 1/1000 in 0.6874 seconds. Average chunk time: 0.6874 seconds [INFOTIMER] Looped over methylation loci in 0.6887 seconds [INFOTIMER] Calculated regression_full in 1.3073 seconds [INFOTIMER] Waiting for chunks to save... [INFOCOUNT] Done saving part 1 [INFOTIMER] Finished waiting for chunks to save in 0.9777 seconds ```
tecpg run mlr-full -l 2000 -p 0.05 --trans ```python [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.0019 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.0518 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 0.0503 seconds [INFOTIMER] Finished reading 3 dataframes in 0.1041 seconds. [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 296 degrees of freedom [INFO] Initializing output directory [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0018 seconds [INFOTIMER] Converted G to tensor in 0.0016 seconds [INFOTIMER] Created ones in 0.0001 seconds [INFOTIMER] Created X in 0.0001 seconds [INFOTIMER] Transposed X in 0.0 seconds [INFOTIMER] Calculated XtXi in 0.6005 seconds [INFOTIMER] Calculated XtXi_diag in 0.0001 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0001 seconds [INFOTIMER] Calculated X constants in 0.6043 seconds [INFO] CUDA device memory: 9.634816 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOCOUNT] Saving part 1/1: [INFOTIMER] Completed chunk 1/1000 in 0.5785 seconds. Average chunk time: 0.5785 seconds [INFOTIMER] Looped over methylation loci in 0.5799 seconds [INFOTIMER] Calculated regression_full in 1.1842 seconds [INFOTIMER] Waiting for chunks to save... [INFOCOUNT] Done saving part 1 [INFOTIMER] Finished waiting for chunks to save in 1.1335 seconds ```
Tests with 300 samples, 50000 methylation loci, and 10000 gene expression loci:
tecpg run mlr-full ```python [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.0059 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.4542 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 2.0871 seconds [INFOTIMER] Finished reading 3 dataframes in 2.5473 seconds. [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 296 degrees of freedom [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0005 seconds [INFOTIMER] Converted G to tensor in 0.0108 seconds [INFOTIMER] Created ones in 0.0001 seconds [INFOTIMER] Created X in 0.0003 seconds [INFOTIMER] Transposed X in 0.0 seconds [INFOTIMER] Calculated XtXi in 0.6773 seconds [INFOTIMER] Calculated XtXi_diag in 0.0001 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0002 seconds [INFOTIMER] Calculated X constants in 0.6894 seconds [INFO] CUDA device memory: 96.630784 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... Traceback (most recent call last): File "\Python310\Scripts\tecpg-script.py", line 33, in sys.exit(load_entry_point('tecpg', 'console_scripts', 'tecpg')()) File "\Python310\Scripts\tecpg-script.py", line 25, in importlib_load_entry_point return next(matches).load() File "\Python310\lib\importlib\metadata\__init__.py", line 162, in load module = import_module(match.group('module')) File "\Python310\lib\importlib\__init__.py", line 126, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "", line 1050, in _gcd_import File "", line 1027, in _find_and_load File "", line 1006, in _find_and_load_unlocked File "", line 688, in _load_unlocked File "", line 883, in exec_module File "", line 241, in _call_with_frames_removed File "\tecpg\__main__.py", line 9, in main() File "\tecpg\__main__.py", line 6, in main start() File "\tecpg\cli.py", line 536, in start cli(obj={}) File "\Python310\lib\site-packages\click\core.py", line 1128, in __call__ return self.main(*args, **kwargs) File "\Python310\lib\site-packages\click\core.py", line 1053, in main rv = self.invoke(ctx) File "\Python310\lib\site-packages\click\core.py", line 1659, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "\Python310\lib\site-packages\click\core.py", line 1659, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "\Python310\lib\site-packages\click\core.py", line 1395, in invoke return ctx.invoke(self.callback, **ctx.params) File "\Python310\lib\site-packages\click\core.py", line 754, in invoke return __callback(*args, **kwargs) File "\Python310\lib\site-packages\click\decorators.py", line 26, in new_func return f(get_current_context(), *args, **kwargs) File "\tecpg\cli.py", line 273, in mlr_full output = regression_full(*args, **logger) File "\tecpg\regression_full.py", line 156, in regression_full E = (Y.unsqueeze(1) - X.bmm(B.unsqueeze(2))).squeeze(2) RuntimeError: CUDA out of memory. Tried to allocate 12.00 MiB (GPU 0; 8.00 GiB total capacity; 7.24 GiB already allocated; 0 bytes free; 7.29 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF ```
tecpg run mlr-full --cis ```python [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.0019 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.4006 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 2.0415 seconds [INFOTIMER] Finished reading 3 dataframes in 2.4441 seconds. [INFO] No region window provided. Resorting to default. [INFO] Using default window for cis of 1000000 bases [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 296 degrees of freedom [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0062 seconds [INFOTIMER] Converted G to tensor in 0.0088 seconds [INFOTIMER] Created ones in 0.0001 seconds [INFOTIMER] Created X in 0.0003 seconds [INFOTIMER] Transposed X in 0.0 seconds [INFOTIMER] Calculated XtXi in 0.59 seconds [INFOTIMER] Calculated XtXi_diag in 0.0001 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0002 seconds [INFOTIMER] Calculated X constants in 0.6058 seconds [INFO] CUDA device memory: 97.112064 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOTIMER] Looped over methylation loci in 92.587 seconds [INFOTIMER] Calculated regression_full in 93.1929 seconds [INFO] Generating dataframe from results... [INFOTIMER] Finished creating indices in 2.5222 seconds [INFOTIMER] Finished creating preliminary dataframe in 0.1001 seconds [INFOTIMER] Created output dataframe in 2.6223 total seconds [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Saving 1 dataframes... [INFOTIMER] Saving 1/1: out.csv [INFOTIMER] Saved 1/1 in 0.9577 seconds [INFOTIMER] Finished saving 1 dataframes in 0.9577 seconds. ```
tecpg run mlr-full --distal ```python [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.0019 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.4186 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 2.0964 seconds [INFOTIMER] Finished reading 3 dataframes in 2.517 seconds. [INFO] No region window provided. Resorting to default. [INFO] Using default window for distal of 50000 bases [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 296 degrees of freedom [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0066 seconds [INFOTIMER] Converted G to tensor in 0.0125 seconds [INFOTIMER] Created ones in 0.0002 seconds [INFOTIMER] Created X in 0.0004 seconds [INFOTIMER] Transposed X in 0.0001 seconds [INFOTIMER] Calculated XtXi in 0.6163 seconds [INFOTIMER] Calculated XtXi_diag in 0.0001 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0002 seconds [INFOTIMER] Calculated X constants in 0.6364 seconds [INFO] CUDA device memory: 97.112064 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOTIMER] Looped over methylation loci in 93.6999 seconds [INFOTIMER] Calculated regression_full in 94.3363 seconds [INFO] Generating dataframe from results... [INFOTIMER] Finished creating indices in 3.2998 seconds [INFOTIMER] Finished creating preliminary dataframe in 1.9467 seconds [INFOTIMER] Created output dataframe in 5.2466 total seconds [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Saving 1 dataframes... [INFOTIMER] Saving 1/1: out.csv [INFOTIMER] Saved 1/1 in 75.742 seconds [INFOTIMER] Finished saving 1 dataframes in 75.7421 seconds. ```
tecpg run mlr-full --trans ```python [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.0019 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.4297 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 2.193 seconds [INFOTIMER] Finished reading 3 dataframes in 2.6247 seconds. [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 296 degrees of freedom [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0067 seconds [INFOTIMER] Converted G to tensor in 0.0129 seconds [INFOTIMER] Created ones in 0.0002 seconds [INFOTIMER] Created X in 0.0004 seconds [INFOTIMER] Transposed X in 0.0001 seconds [INFOTIMER] Calculated XtXi in 0.6625 seconds [INFOTIMER] Calculated XtXi_diag in 0.0001 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0002 seconds [INFOTIMER] Calculated X constants in 0.6831 seconds [INFO] CUDA device memory: 97.112064 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... Traceback (most recent call last): File "\Python310\Scripts\tecpg-script.py", line 33, in sys.exit(load_entry_point('tecpg', 'console_scripts', 'tecpg')()) File "\Python310\Scripts\tecpg-script.py", line 25, in importlib_load_entry_point return next(matches).load() File "\Python310\lib\importlib\metadata\__init__.py", line 162, in load module = import_module(match.group('module')) File "\Python310\lib\importlib\__init__.py", line 126, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "", line 1050, in _gcd_import File "", line 1027, in _find_and_load File "", line 1006, in _find_and_load_unlocked File "", line 688, in _load_unlocked File "", line 883, in exec_module File "", line 241, in _call_with_frames_removed File "\tecpg\__main__.py", line 9, in main() File "\tecpg\__main__.py", line 6, in main start() File "\tecpg\cli.py", line 536, in start cli(obj={}) File "\Python310\lib\site-packages\click\core.py", line 1128, in __call__ return self.main(*args, **kwargs) File "\Python310\lib\site-packages\click\core.py", line 1053, in main rv = self.invoke(ctx) File "\Python310\lib\site-packages\click\core.py", line 1659, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "\Python310\lib\site-packages\click\core.py", line 1659, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "\Python310\lib\site-packages\click\core.py", line 1395, in invoke return ctx.invoke(self.callback, **ctx.params) File "\Python310\lib\site-packages\click\core.py", line 754, in invoke return __callback(*args, **kwargs) File "\Python310\lib\site-packages\click\decorators.py", line 26, in new_func return f(get_current_context(), *args, **kwargs) File "\tecpg\cli.py", line 273, in mlr_full output = regression_full(*args, **logger) File "\tecpg\regression_full.py", line 156, in regression_full E = (Y.unsqueeze(1) - X.bmm(B.unsqueeze(2))).squeeze(2) RuntimeError: CUDA out of memory. Tried to allocate 12.00 MiB (GPU 0; 8.00 GiB total capacity; 7.11 GiB already allocated; 0 bytes free; 7.29 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF ```
tecpg run mlr-full -p 0.05 ```python [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.0019 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.4339 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 2.0454 seconds [INFOTIMER] Finished reading 3 dataframes in 2.4813 seconds. [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 296 degrees of freedom [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0005 seconds [INFOTIMER] Converted G to tensor in 0.0109 seconds [INFOTIMER] Created ones in 0.0002 seconds [INFOTIMER] Created X in 0.0004 seconds [INFOTIMER] Transposed X in 0.0001 seconds [INFOTIMER] Calculated XtXi in 0.6711 seconds [INFOTIMER] Calculated XtXi_diag in 0.0001 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0003 seconds [INFOTIMER] Calculated X constants in 0.6835 seconds [INFO] CUDA device memory: 96.630784 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOTIMER] Looped over methylation loci in 92.52 seconds [INFOTIMER] Calculated regression_full in 93.2036 seconds [INFO] Generating dataframe from results... [INFOTIMER] Finished creating indices in 3.2543 seconds [INFOTIMER] Finished creating preliminary dataframe in 1.919 seconds [INFOTIMER] Created output dataframe in 5.1733 total seconds [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Saving 1 dataframes... [INFOTIMER] Saving 1/1: out.csv [INFOTIMER] Saved 1/1 in 74.2389 seconds [INFOTIMER] Finished saving 1 dataframes in 74.2389 seconds. ```
tecpg run mlr-full -p 0.05 --cis ```python [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.0019 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.42 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 1.9897 seconds [INFOTIMER] Finished reading 3 dataframes in 2.4117 seconds. [INFO] No region window provided. Resorting to default. [INFO] Using default window for cis of 1000000 bases [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 296 degrees of freedom [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0066 seconds [INFOTIMER] Converted G to tensor in 0.01 seconds [INFOTIMER] Created ones in 0.0001 seconds [INFOTIMER] Created X in 0.0004 seconds [INFOTIMER] Transposed X in 0.0 seconds [INFOTIMER] Calculated XtXi in 0.61 seconds [INFOTIMER] Calculated XtXi_diag in 0.0001 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0002 seconds [INFOTIMER] Calculated X constants in 0.6276 seconds [INFO] CUDA device memory: 97.112064 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOTIMER] Looped over methylation loci in 92.7993 seconds [INFOTIMER] Calculated regression_full in 93.427 seconds [INFO] Generating dataframe from results... [INFOTIMER] Finished creating indices in 2.4257 seconds [INFOTIMER] Finished creating preliminary dataframe in 0.0475 seconds [INFOTIMER] Created output dataframe in 2.4733 total seconds [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Saving 1 dataframes... [INFOTIMER] Saving 1/1: out.csv [INFOTIMER] Saved 1/1 in 0.0418 seconds [INFOTIMER] Finished saving 1 dataframes in 0.0419 seconds. ```
tecpg run mlr-full -p 0.05 --distal ```python [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.0019 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.4096 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 1.9927 seconds [INFOTIMER] Finished reading 3 dataframes in 2.4043 seconds. [INFO] No region window provided. Resorting to default. [INFO] Using default window for distal of 50000 bases [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 296 degrees of freedom [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0065 seconds [INFOTIMER] Converted G to tensor in 0.0084 seconds [INFOTIMER] Created ones in 0.0001 seconds [INFOTIMER] Created X in 0.0003 seconds [INFOTIMER] Transposed X in 0.0 seconds [INFOTIMER] Calculated XtXi in 0.6044 seconds [INFOTIMER] Calculated XtXi_diag in 0.0001 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0002 seconds [INFOTIMER] Calculated X constants in 0.6202 seconds [INFO] CUDA device memory: 97.112064 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOTIMER] Looped over methylation loci in 92.9034 seconds [INFOTIMER] Calculated regression_full in 93.5236 seconds [INFO] Generating dataframe from results... [INFOTIMER] Finished creating indices in 2.5099 seconds [INFOTIMER] Finished creating preliminary dataframe in 0.1527 seconds [INFOTIMER] Created output dataframe in 2.6626 total seconds [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Saving 1 dataframes... [INFOTIMER] Saving 1/1: out.csv [INFOTIMER] Saved 1/1 in 3.1803 seconds [INFOTIMER] Finished saving 1 dataframes in 3.1804 seconds. ```
tecpg run mlr-full -p 0.05 --trans ```python [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.0027 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.4399 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 2.1111 seconds [INFOTIMER] Finished reading 3 dataframes in 2.5539 seconds. [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 296 degrees of freedom [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0065 seconds [INFOTIMER] Converted G to tensor in 0.0093 seconds [INFOTIMER] Created ones in 0.0002 seconds [INFOTIMER] Created X in 0.0004 seconds [INFOTIMER] Transposed X in 0.0 seconds [INFOTIMER] Calculated XtXi in 0.6216 seconds [INFOTIMER] Calculated XtXi_diag in 0.0001 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0003 seconds [INFOTIMER] Calculated X constants in 0.6384 seconds [INFO] CUDA device memory: 97.112064 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOTIMER] Looped over methylation loci in 92.9569 seconds [INFOTIMER] Calculated regression_full in 93.5954 seconds [INFO] Generating dataframe from results... [INFOTIMER] Finished creating indices in 3.2387 seconds [INFOTIMER] Finished creating preliminary dataframe in 1.8502 seconds [INFOTIMER] Created output dataframe in 5.089 total seconds [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Saving 1 dataframes... [INFOTIMER] Saving 1/1: out.csv [INFOTIMER] Saved 1/1 in 71.7912 seconds [INFOTIMER] Finished saving 1 dataframes in 71.7913 seconds. ```
tecpg run mlr-full -l 2000 ```python [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.0019 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.4278 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 2.1612 seconds [INFOTIMER] Finished reading 3 dataframes in 2.591 seconds. [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 296 degrees of freedom [INFO] Initializing output directory [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0008 seconds [INFOTIMER] Converted G to tensor in 0.0126 seconds [INFOTIMER] Created ones in 0.0002 seconds [INFOTIMER] Created X in 0.0004 seconds [INFOTIMER] Transposed X in 0.0 seconds [INFOTIMER] Calculated XtXi in 0.6352 seconds [INFOTIMER] Calculated XtXi_diag in 0.0001 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0003 seconds [INFOTIMER] Calculated X constants in 0.6497 seconds [INFO] CUDA device memory: 96.630784 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOCOUNT] Saving part 1/25: [INFOTIMER] Completed chunk 1/50000 in 5.161 seconds. Average chunk time: 5.161 seconds [INFOCOUNT] Saving part 2/25: [INFOTIMER] Completed chunk 2/50000 in 5.7265 seconds. Average chunk time: 5.4437 seconds [INFOCOUNT] Saving part 3/25: [INFOTIMER] Completed chunk 3/50000 in 5.8349 seconds. Average chunk time: 5.5741 seconds [INFOCOUNT] Saving part 4/25: [INFOTIMER] Completed chunk 4/50000 in 5.9623 seconds. Average chunk time: 5.6712 seconds [INFOCOUNT] Saving part 5/25: [INFOTIMER] Completed chunk 5/50000 in 6.1881 seconds. Average chunk time: 5.7746 seconds [INFOCOUNT] Saving part 6/25: [INFOTIMER] Completed chunk 6/50000 in 6.5023 seconds. Average chunk time: 5.8959 seconds [INFOCOUNT] Saving part 7/25: [INFOTIMER] Completed chunk 7/50000 in 6.676 seconds. Average chunk time: 6.0073 seconds [INFOCOUNT] Saving part 8/25: [INFOTIMER] Completed chunk 8/50000 in 7.2719 seconds. Average chunk time: 6.1654 seconds [INFOCOUNT] Saving part 9/25: [INFOTIMER] Completed chunk 9/50000 in 7.2993 seconds. Average chunk time: 6.2914 seconds [INFOCOUNT] Saving part 10/25: [INFOTIMER] Completed chunk 10/50000 in 7.9574 seconds. Average chunk time: 6.458 seconds [INFOCOUNT] Saving part 11/25: [INFOTIMER] Completed chunk 11/50000 in 6.5997 seconds. Average chunk time: 6.4709 seconds [INFOCOUNT] Saving part 12/25: [INFOTIMER] Completed chunk 12/50000 in 7.3404 seconds. Average chunk time: 6.5433 seconds [INFOCOUNT] Saving part 13/25: [INFOTIMER] Completed chunk 13/50000 in 7.1029 seconds. Average chunk time: 6.5864 seconds [INFOCOUNT] Saving part 14/25: [INFOTIMER] Completed chunk 14/50000 in 7.1535 seconds. Average chunk time: 6.6269 seconds [INFOCOUNT] Saving part 15/25: [INFOTIMER] Completed chunk 15/50000 in 7.2482 seconds. Average chunk time: 6.6683 seconds [INFOCOUNT] Saving part 16/25: [INFOTIMER] Completed chunk 16/50000 in 7.2193 seconds. Average chunk time: 6.7027 seconds [INFOCOUNT] Saving part 17/25: [INFOTIMER] Completed chunk 17/50000 in 6.9051 seconds. Average chunk time: 6.7146 seconds [INFOCOUNT] Saving part 18/25: [INFOTIMER] Completed chunk 18/50000 in 6.788 seconds. Average chunk time: 6.7187 seconds [INFOCOUNT] Saving part 19/25: [INFOTIMER] Completed chunk 19/50000 in 6.5594 seconds. Average chunk time: 6.7103 seconds [INFOCOUNT] Saving part 20/25: [INFOTIMER] Completed chunk 20/50000 in 6.474 seconds. Average chunk time: 6.6985 seconds [INFOCOUNT] Done saving part 1 [INFOCOUNT] Saving part 21/25: [INFOTIMER] Completed chunk 21/50000 in 7.4175 seconds. Average chunk time: 6.7328 seconds [INFOCOUNT] Done saving part 2 [INFOCOUNT] Saving part 22/25: [INFOTIMER] Completed chunk 22/50000 in 6.9812 seconds. Average chunk time: 6.744 seconds [INFOCOUNT] Saving part 23/25: [INFOTIMER] Completed chunk 23/50000 in 6.5522 seconds. Average chunk time: 6.7357 seconds [INFOCOUNT] Saving part 24/25: [INFOTIMER] Completed chunk 24/50000 in 6.4488 seconds. Average chunk time: 6.7237 seconds [INFOCOUNT] Done saving part 4 [INFOCOUNT] Done saving part 3 [INFOCOUNT] Saving part 25/25: [INFOTIMER] Completed chunk 25/50000 in 7.1832 seconds. Average chunk time: 6.7421 seconds [INFOTIMER] Looped over methylation loci in 168.5589 seconds [INFOTIMER] Calculated regression_full in 169.2086 seconds [INFOTIMER] Waiting for chunks to save... [INFOCOUNT] Done saving part 5 [INFOCOUNT] Done saving part 6 [INFOCOUNT] Done saving part 7 [INFOCOUNT] Done saving part 8 [INFOCOUNT] Done saving part 9 [INFOCOUNT] Done saving part 10 [INFOCOUNT] Done saving part 11 [INFOCOUNT] Done saving part 12 [INFOCOUNT] Done saving part 13 [INFOCOUNT] Done saving part 14 [INFOCOUNT] Done saving part 15 [INFOCOUNT] Done saving part 16 [INFOCOUNT] Done saving part 17 [INFOCOUNT] Done saving part 18 [INFOCOUNT] Done saving part 19 [INFOCOUNT] Done saving part 20 [INFOCOUNT] Done saving part 21 [INFOCOUNT] Done saving part 22 [INFOCOUNT] Done saving part 23 [INFOCOUNT] Done saving part 24 [INFOCOUNT] Done saving part 25 [INFOTIMER] Finished waiting for chunks to save in 314.7567 seconds ```
tecpg run mlr-full -l 2000 --cis ```python [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.0024 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.4143 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 2.042 seconds [INFOTIMER] Finished reading 3 dataframes in 2.4588 seconds. [INFO] No region window provided. Resorting to default. [INFO] Using default window for cis of 1000000 bases [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 296 degrees of freedom [INFO] Initializing output directory [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.013 seconds [INFOTIMER] Converted G to tensor in 0.0122 seconds [INFOTIMER] Created ones in 0.0003 seconds [INFOTIMER] Created X in 0.0004 seconds [INFOTIMER] Transposed X in 0.0001 seconds [INFOTIMER] Calculated XtXi in 3.6884 seconds [INFOTIMER] Calculated XtXi_diag in 0.0002 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0005 seconds [INFOTIMER] Calculated X constants in 3.7152 seconds [INFO] CUDA device memory: 97.112064 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOCOUNT] Saving part 1/25: [INFOTIMER] Completed chunk 1/50000 in 5.8347 seconds. Average chunk time: 5.8347 seconds [INFOCOUNT] Done saving part 1 [INFOCOUNT] Saving part 2/25: [INFOTIMER] Completed chunk 2/50000 in 3.8749 seconds. Average chunk time: 4.8548 seconds [INFOCOUNT] Done saving part 2 [INFOCOUNT] Saving part 3/25: [INFOTIMER] Completed chunk 3/50000 in 3.8629 seconds. Average chunk time: 4.5242 seconds [INFOCOUNT] Done saving part 3 [INFOCOUNT] Saving part 4/25: [INFOTIMER] Completed chunk 4/50000 in 3.8694 seconds. Average chunk time: 4.3605 seconds [INFOCOUNT] Done saving part 4 [INFOCOUNT] Saving part 5/25: [INFOTIMER] Completed chunk 5/50000 in 3.8881 seconds. Average chunk time: 4.266 seconds [INFOCOUNT] Done saving part 5 [INFOCOUNT] Saving part 6/25: [INFOTIMER] Completed chunk 6/50000 in 3.9176 seconds. Average chunk time: 4.2079 seconds [INFOCOUNT] Done saving part 6 [INFOCOUNT] Saving part 7/25: [INFOTIMER] Completed chunk 7/50000 in 3.9043 seconds. Average chunk time: 4.1646 seconds [INFOCOUNT] Done saving part 7 [INFOCOUNT] Saving part 8/25: [INFOTIMER] Completed chunk 8/50000 in 3.8836 seconds. Average chunk time: 4.1294 seconds [INFOCOUNT] Done saving part 8 [INFOCOUNT] Saving part 9/25: [INFOTIMER] Completed chunk 9/50000 in 3.8613 seconds. Average chunk time: 4.0996 seconds [INFOCOUNT] Done saving part 9 [INFOCOUNT] Saving part 10/25: [INFOTIMER] Completed chunk 10/50000 in 3.8454 seconds. Average chunk time: 4.0742 seconds [INFOCOUNT] Done saving part 10 [INFOCOUNT] Saving part 11/25: [INFOTIMER] Completed chunk 11/50000 in 3.8719 seconds. Average chunk time: 4.0558 seconds [INFOCOUNT] Done saving part 11 [INFOCOUNT] Saving part 12/25: [INFOTIMER] Completed chunk 12/50000 in 3.843 seconds. Average chunk time: 4.0381 seconds [INFOCOUNT] Done saving part 12 [INFOCOUNT] Saving part 13/25: [INFOTIMER] Completed chunk 13/50000 in 3.85 seconds. Average chunk time: 4.0236 seconds [INFOCOUNT] Done saving part 13 [INFOCOUNT] Saving part 14/25: [INFOTIMER] Completed chunk 14/50000 in 3.8509 seconds. Average chunk time: 4.0113 seconds [INFOCOUNT] Done saving part 14 [INFOCOUNT] Saving part 15/25: [INFOTIMER] Completed chunk 15/50000 in 3.8524 seconds. Average chunk time: 4.0007 seconds [INFOCOUNT] Done saving part 15 [INFOCOUNT] Saving part 16/25: [INFOTIMER] Completed chunk 16/50000 in 3.8676 seconds. Average chunk time: 3.9924 seconds [INFOCOUNT] Done saving part 16 [INFOCOUNT] Saving part 17/25: [INFOTIMER] Completed chunk 17/50000 in 3.9382 seconds. Average chunk time: 3.9892 seconds [INFOCOUNT] Done saving part 17 [INFOCOUNT] Saving part 18/25: [INFOTIMER] Completed chunk 18/50000 in 3.8898 seconds. Average chunk time: 3.9837 seconds [INFOCOUNT] Done saving part 18 [INFOCOUNT] Saving part 19/25: [INFOTIMER] Completed chunk 19/50000 in 3.8946 seconds. Average chunk time: 3.979 seconds [INFOCOUNT] Done saving part 19 [INFOCOUNT] Saving part 20/25: [INFOTIMER] Completed chunk 20/50000 in 3.8784 seconds. Average chunk time: 3.9739 seconds [INFOCOUNT] Done saving part 20 [INFOCOUNT] Saving part 21/25: [INFOTIMER] Completed chunk 21/50000 in 3.8714 seconds. Average chunk time: 3.9691 seconds [INFOCOUNT] Done saving part 21 [INFOCOUNT] Saving part 22/25: [INFOTIMER] Completed chunk 22/50000 in 3.873 seconds. Average chunk time: 3.9647 seconds [INFOCOUNT] Done saving part 22 [INFOCOUNT] Saving part 23/25: [INFOTIMER] Completed chunk 23/50000 in 3.9117 seconds. Average chunk time: 3.9624 seconds [INFOCOUNT] Done saving part 23 [INFOCOUNT] Saving part 24/25: [INFOTIMER] Completed chunk 24/50000 in 3.9047 seconds. Average chunk time: 3.96 seconds [INFOCOUNT] Done saving part 24 [INFOCOUNT] Saving part 25/25: [INFOTIMER] Completed chunk 25/50000 in 3.9015 seconds. Average chunk time: 3.9576 seconds [INFOTIMER] Looped over methylation loci in 98.9498 seconds [INFOTIMER] Calculated regression_full in 102.665 seconds [INFOTIMER] Waiting for chunks to save... [INFOCOUNT] Done saving part 25 [INFOTIMER] Finished waiting for chunks to save in 0.3457 seconds ```
tecpg run mlr-full -l 2000 --distal ```python [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.0019 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.4375 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 2.0615 seconds [INFOTIMER] Finished reading 3 dataframes in 2.5009 seconds. [INFO] No region window provided. Resorting to default. [INFO] Using default window for distal of 50000 bases [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 296 degrees of freedom [INFO] Initializing output directory [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0065 seconds [INFOTIMER] Converted G to tensor in 0.0112 seconds [INFOTIMER] Created ones in 0.0002 seconds [INFOTIMER] Created X in 0.0004 seconds [INFOTIMER] Transposed X in 0.0001 seconds [INFOTIMER] Calculated XtXi in 0.6833 seconds [INFOTIMER] Calculated XtXi_diag in 0.0002 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0004 seconds [INFOTIMER] Calculated X constants in 0.7022 seconds [INFO] CUDA device memory: 97.112064 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOCOUNT] Saving part 1/25: [INFOTIMER] Completed chunk 1/50000 in 4.0666 seconds. Average chunk time: 4.0666 seconds [INFOCOUNT] Saving part 2/25: [INFOTIMER] Completed chunk 2/50000 in 4.0602 seconds. Average chunk time: 4.0634 seconds [INFOCOUNT] Done saving part 1 [INFOCOUNT] Saving part 3/25: [INFOTIMER] Completed chunk 3/50000 in 4.1207 seconds. Average chunk time: 4.0825 seconds [INFOCOUNT] Done saving part 2 [INFOCOUNT] Saving part 4/25: [INFOTIMER] Completed chunk 4/50000 in 4.1471 seconds. Average chunk time: 4.0987 seconds [INFOCOUNT] Done saving part 3 [INFOCOUNT] Saving part 5/25: [INFOTIMER] Completed chunk 5/50000 in 4.1898 seconds. Average chunk time: 4.1169 seconds [INFOCOUNT] Done saving part 4 [INFOCOUNT] Saving part 6/25: [INFOTIMER] Completed chunk 6/50000 in 4.2449 seconds. Average chunk time: 4.1382 seconds [INFOCOUNT] Done saving part 5 [INFOCOUNT] Saving part 7/25: [INFOTIMER] Completed chunk 7/50000 in 4.1444 seconds. Average chunk time: 4.1391 seconds [INFOCOUNT] Done saving part 6 [INFOCOUNT] Saving part 8/25: [INFOTIMER] Completed chunk 8/50000 in 4.184 seconds. Average chunk time: 4.1447 seconds [INFOCOUNT] Done saving part 7 [INFOCOUNT] Saving part 9/25: [INFOTIMER] Completed chunk 9/50000 in 4.212 seconds. Average chunk time: 4.1522 seconds [INFOCOUNT] Done saving part 8 [INFOCOUNT] Done saving part 9 [INFOCOUNT] Saving part 10/25: [INFOTIMER] Completed chunk 10/50000 in 4.1122 seconds. Average chunk time: 4.1482 seconds [INFOCOUNT] Done saving part 10 [INFOCOUNT] Saving part 11/25: [INFOTIMER] Completed chunk 11/50000 in 4.1542 seconds. Average chunk time: 4.1487 seconds [INFOCOUNT] Done saving part 11 [INFOCOUNT] Saving part 12/25: [INFOTIMER] Completed chunk 12/50000 in 4.1497 seconds. Average chunk time: 4.1488 seconds [INFOCOUNT] Done saving part 12 [INFOCOUNT] Saving part 13/25: [INFOTIMER] Completed chunk 13/50000 in 4.2324 seconds. Average chunk time: 4.1553 seconds [INFOCOUNT] Saving part 14/25: [INFOTIMER] Completed chunk 14/50000 in 4.2206 seconds. Average chunk time: 4.1599 seconds [INFOCOUNT] Done saving part 13 [INFOCOUNT] Done saving part 14 [INFOCOUNT] Saving part 15/25: [INFOTIMER] Completed chunk 15/50000 in 4.1818 seconds. Average chunk time: 4.1614 seconds [INFOCOUNT] Done saving part 15 [INFOCOUNT] Saving part 16/25: [INFOTIMER] Completed chunk 16/50000 in 4.189 seconds. Average chunk time: 4.1631 seconds [INFOCOUNT] Done saving part 16 [INFOCOUNT] Saving part 17/25: [INFOTIMER] Completed chunk 17/50000 in 4.2351 seconds. Average chunk time: 4.1673 seconds [INFOCOUNT] Done saving part 17 [INFOCOUNT] Saving part 18/25: [INFOTIMER] Completed chunk 18/50000 in 4.1678 seconds. Average chunk time: 4.1674 seconds [INFOCOUNT] Done saving part 18 [INFOCOUNT] Saving part 19/25: [INFOTIMER] Completed chunk 19/50000 in 4.1503 seconds. Average chunk time: 4.1665 seconds [INFOCOUNT] Done saving part 19 [INFOCOUNT] Saving part 20/25: [INFOTIMER] Completed chunk 20/50000 in 4.159 seconds. Average chunk time: 4.1661 seconds [INFOCOUNT] Done saving part 20 [INFOCOUNT] Saving part 21/25: [INFOTIMER] Completed chunk 21/50000 in 4.0963 seconds. Average chunk time: 4.1628 seconds [INFOCOUNT] Done saving part 21 [INFOCOUNT] Saving part 22/25: [INFOTIMER] Completed chunk 22/50000 in 4.1498 seconds. Average chunk time: 4.1622 seconds [INFOCOUNT] Done saving part 22 [INFOCOUNT] Saving part 23/25: [INFOTIMER] Completed chunk 23/50000 in 4.1382 seconds. Average chunk time: 4.1611 seconds [INFOCOUNT] Done saving part 23 [INFOCOUNT] Saving part 24/25: [INFOTIMER] Completed chunk 24/50000 in 4.1011 seconds. Average chunk time: 4.1586 seconds [INFOCOUNT] Done saving part 24 [INFOCOUNT] Saving part 25/25: [INFOTIMER] Completed chunk 25/50000 in 4.1426 seconds. Average chunk time: 4.158 seconds [INFOTIMER] Looped over methylation loci in 103.9589 seconds [INFOTIMER] Calculated regression_full in 104.6743 seconds [INFOTIMER] Waiting for chunks to save... [INFOCOUNT] Done saving part 25 [INFOTIMER] Finished waiting for chunks to save in 3.7764 seconds ```
tecpg run mlr-full -l 2000 --trans ```python [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.0019 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.4449 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 2.1944 seconds [INFOTIMER] Finished reading 3 dataframes in 2.6414 seconds. [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 296 degrees of freedom [INFO] Initializing output directory [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0099 seconds [INFOTIMER] Converted G to tensor in 0.0115 seconds [INFOTIMER] Created ones in 0.0001 seconds [INFOTIMER] Created X in 0.0003 seconds [INFOTIMER] Transposed X in 0.0 seconds [INFOTIMER] Calculated XtXi in 0.6809 seconds [INFOTIMER] Calculated XtXi_diag in 0.0001 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0003 seconds [INFOTIMER] Calculated X constants in 0.7032 seconds [INFO] CUDA device memory: 97.112064 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOCOUNT] Saving part 1/25: [INFOTIMER] Completed chunk 1/50000 in 5.5706 seconds. Average chunk time: 5.5706 seconds [INFOCOUNT] Saving part 2/25: [INFOTIMER] Completed chunk 2/50000 in 6.0339 seconds. Average chunk time: 5.8022 seconds [INFOCOUNT] Saving part 3/25: [INFOTIMER] Completed chunk 3/50000 in 6.2667 seconds. Average chunk time: 5.9571 seconds [INFOCOUNT] Saving part 4/25: [INFOTIMER] Completed chunk 4/50000 in 6.6002 seconds. Average chunk time: 6.1178 seconds [INFOCOUNT] Saving part 5/25: [INFOTIMER] Completed chunk 5/50000 in 6.6299 seconds. Average chunk time: 6.2203 seconds [INFOCOUNT] Saving part 6/25: [INFOTIMER] Completed chunk 6/50000 in 6.9148 seconds. Average chunk time: 6.336 seconds [INFOCOUNT] Saving part 7/25: [INFOTIMER] Completed chunk 7/50000 in 7.1398 seconds. Average chunk time: 6.4508 seconds [INFOCOUNT] Saving part 8/25: [INFOTIMER] Completed chunk 8/50000 in 7.7258 seconds. Average chunk time: 6.6102 seconds [INFOCOUNT] Saving part 9/25: [INFOTIMER] Completed chunk 9/50000 in 8.2529 seconds. Average chunk time: 6.7927 seconds [INFOCOUNT] Saving part 10/25: [INFOTIMER] Completed chunk 10/50000 in 8.1461 seconds. Average chunk time: 6.9281 seconds [INFOCOUNT] Saving part 11/25: [INFOTIMER] Completed chunk 11/50000 in 7.8067 seconds. Average chunk time: 7.0079 seconds [INFOCOUNT] Saving part 12/25: [INFOTIMER] Completed chunk 12/50000 in 6.8487 seconds. Average chunk time: 6.9947 seconds [INFOCOUNT] Saving part 13/25: [INFOTIMER] Completed chunk 13/50000 in 8.2471 seconds. Average chunk time: 7.091 seconds [INFOCOUNT] Saving part 14/25: [INFOTIMER] Completed chunk 14/50000 in 7.4319 seconds. Average chunk time: 7.1154 seconds [INFOCOUNT] Saving part 15/25: [INFOTIMER] Completed chunk 15/50000 in 6.8405 seconds. Average chunk time: 7.097 seconds [INFOCOUNT] Saving part 16/25: [INFOTIMER] Completed chunk 16/50000 in 7.3484 seconds. Average chunk time: 7.1127 seconds [INFOCOUNT] Saving part 17/25: [INFOTIMER] Completed chunk 17/50000 in 6.9462 seconds. Average chunk time: 7.103 seconds [INFOCOUNT] Saving part 18/25: [INFOTIMER] Completed chunk 18/50000 in 6.8057 seconds. Average chunk time: 7.0864 seconds [INFOCOUNT] Done saving part 1 [INFOCOUNT] Saving part 19/25: [INFOTIMER] Completed chunk 19/50000 in 7.5005 seconds. Average chunk time: 7.1082 seconds [INFOCOUNT] Done saving part 2 [INFOCOUNT] Saving part 20/25: [INFOTIMER] Completed chunk 20/50000 in 8.1752 seconds. Average chunk time: 7.1616 seconds [INFOCOUNT] Done saving part 3 [INFOCOUNT] Saving part 21/25: [INFOTIMER] Completed chunk 21/50000 in 7.2494 seconds. Average chunk time: 7.1658 seconds [INFOCOUNT] Saving part 22/25: [INFOTIMER] Completed chunk 22/50000 in 7.0164 seconds. Average chunk time: 7.159 seconds [INFOCOUNT] Done saving part 4 [INFOCOUNT] Saving part 23/25: [INFOTIMER] Completed chunk 23/50000 in 7.4953 seconds. Average chunk time: 7.1736 seconds [INFOCOUNT] Done saving part 5 [INFOCOUNT] Saving part 24/25: [INFOTIMER] Completed chunk 24/50000 in 7.0779 seconds. Average chunk time: 7.1696 seconds [INFOCOUNT] Done saving part 6 [INFOCOUNT] Saving part 25/25: [INFOTIMER] Completed chunk 25/50000 in 7.7343 seconds. Average chunk time: 7.1922 seconds [INFOTIMER] Looped over methylation loci in 179.8135 seconds [INFOTIMER] Calculated regression_full in 180.5167 seconds [INFOTIMER] Waiting for chunks to save... [INFOCOUNT] Done saving part 7 [INFOCOUNT] Done saving part 8 [INFOCOUNT] Done saving part 9 [INFOCOUNT] Done saving part 10 [INFOCOUNT] Done saving part 11 [INFOCOUNT] Done saving part 12 [INFOCOUNT] Done saving part 13 [INFOCOUNT] Done saving part 14 [INFOCOUNT] Done saving part 15 [INFOCOUNT] Done saving part 16 [INFOCOUNT] Done saving part 17 [INFOCOUNT] Done saving part 18 [INFOCOUNT] Done saving part 19 [INFOCOUNT] Done saving part 20 [INFOCOUNT] Done saving part 21 [INFOCOUNT] Done saving part 22 [INFOCOUNT] Done saving part 23 [INFOCOUNT] Done saving part 24 [INFOCOUNT] Done saving part 25 [INFOTIMER] Finished waiting for chunks to save in 284.5807 seconds ```
tecpg run mlr-full -l 2000 -p 0.05 ```python [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.0021 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.4493 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 2.3284 seconds [INFOTIMER] Finished reading 3 dataframes in 2.7799 seconds. [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 296 degrees of freedom [INFO] Initializing output directory [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0008 seconds [INFOTIMER] Converted G to tensor in 0.0127 seconds [INFOTIMER] Created ones in 0.0002 seconds [INFOTIMER] Created X in 0.0004 seconds [INFOTIMER] Transposed X in 0.0001 seconds [INFOTIMER] Calculated XtXi in 0.7208 seconds [INFOTIMER] Calculated XtXi_diag in 0.0001 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0002 seconds [INFOTIMER] Calculated X constants in 0.7353 seconds [INFO] CUDA device memory: 96.630784 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOCOUNT] Saving part 1/25: [INFOTIMER] Completed chunk 1/50000 in 4.0094 seconds. Average chunk time: 4.0094 seconds [INFOCOUNT] Done saving part 1 [INFOCOUNT] Saving part 2/25: [INFOTIMER] Completed chunk 2/50000 in 3.98 seconds. Average chunk time: 3.9947 seconds [INFOCOUNT] Saving part 3/25: [INFOTIMER] Completed chunk 3/50000 in 4.0215 seconds. Average chunk time: 4.0037 seconds [INFOCOUNT] Done saving part 2 [INFOCOUNT] Saving part 4/25: [INFOTIMER] Completed chunk 4/50000 in 4.0718 seconds. Average chunk time: 4.0207 seconds [INFOCOUNT] Done saving part 3 [INFOCOUNT] Saving part 5/25: [INFOTIMER] Completed chunk 5/50000 in 4.0592 seconds. Average chunk time: 4.0284 seconds [INFOCOUNT] Done saving part 4 [INFOCOUNT] Saving part 6/25: [INFOTIMER] Completed chunk 6/50000 in 4.0614 seconds. Average chunk time: 4.0339 seconds [INFOCOUNT] Done saving part 5 [INFOCOUNT] Saving part 7/25: [INFOTIMER] Completed chunk 7/50000 in 4.0255 seconds. Average chunk time: 4.0327 seconds [INFOCOUNT] Done saving part 6 [INFOCOUNT] Saving part 8/25: [INFOTIMER] Completed chunk 8/50000 in 4.0927 seconds. Average chunk time: 4.0402 seconds [INFOCOUNT] Done saving part 7 [INFOCOUNT] Saving part 9/25: [INFOTIMER] Completed chunk 9/50000 in 4.0394 seconds. Average chunk time: 4.0401 seconds [INFOCOUNT] Done saving part 8 [INFOCOUNT] Done saving part 9 [INFOCOUNT] Saving part 10/25: [INFOTIMER] Completed chunk 10/50000 in 4.01 seconds. Average chunk time: 4.0371 seconds [INFOCOUNT] Done saving part 10 [INFOCOUNT] Saving part 11/25: [INFOTIMER] Completed chunk 11/50000 in 4.0225 seconds. Average chunk time: 4.0358 seconds [INFOCOUNT] Done saving part 11 [INFOCOUNT] Saving part 12/25: [INFOTIMER] Completed chunk 12/50000 in 4.0248 seconds. Average chunk time: 4.0349 seconds [INFOCOUNT] Done saving part 12 [INFOCOUNT] Saving part 13/25: [INFOTIMER] Completed chunk 13/50000 in 4.0453 seconds. Average chunk time: 4.0357 seconds [INFOCOUNT] Done saving part 13 [INFOCOUNT] Saving part 14/25: [INFOTIMER] Completed chunk 14/50000 in 3.9977 seconds. Average chunk time: 4.033 seconds [INFOCOUNT] Done saving part 14 [INFOCOUNT] Saving part 15/25: [INFOTIMER] Completed chunk 15/50000 in 4.0091 seconds. Average chunk time: 4.0314 seconds [INFOCOUNT] Done saving part 15 [INFOCOUNT] Saving part 16/25: [INFOTIMER] Completed chunk 16/50000 in 4.0571 seconds. Average chunk time: 4.033 seconds [INFOCOUNT] Done saving part 16 [INFOCOUNT] Saving part 17/25: [INFOTIMER] Completed chunk 17/50000 in 3.9957 seconds. Average chunk time: 4.0308 seconds [INFOCOUNT] Done saving part 17 [INFOCOUNT] Saving part 18/25: [INFOTIMER] Completed chunk 18/50000 in 4.0735 seconds. Average chunk time: 4.0332 seconds [INFOCOUNT] Done saving part 18 [INFOCOUNT] Saving part 19/25: [INFOTIMER] Completed chunk 19/50000 in 3.9937 seconds. Average chunk time: 4.0311 seconds [INFOCOUNT] Done saving part 19 [INFOCOUNT] Saving part 20/25: [INFOTIMER] Completed chunk 20/50000 in 4.0337 seconds. Average chunk time: 4.0312 seconds [INFOCOUNT] Done saving part 20 [INFOCOUNT] Saving part 21/25: [INFOTIMER] Completed chunk 21/50000 in 4.038 seconds. Average chunk time: 4.0315 seconds [INFOCOUNT] Done saving part 21 [INFOCOUNT] Saving part 22/25: [INFOTIMER] Completed chunk 22/50000 in 3.9942 seconds. Average chunk time: 4.0298 seconds [INFOCOUNT] Done saving part 22 [INFOCOUNT] Saving part 23/25: [INFOTIMER] Completed chunk 23/50000 in 4.0458 seconds. Average chunk time: 4.0305 seconds [INFOCOUNT] Done saving part 23 [INFOCOUNT] Saving part 24/25: [INFOTIMER] Completed chunk 24/50000 in 4.0341 seconds. Average chunk time: 4.0307 seconds [INFOCOUNT] Done saving part 24 [INFOCOUNT] Saving part 25/25: [INFOTIMER] Completed chunk 25/50000 in 4.0298 seconds. Average chunk time: 4.0306 seconds [INFOTIMER] Looped over methylation loci in 100.7718 seconds [INFOTIMER] Calculated regression_full in 101.5071 seconds [INFOTIMER] Waiting for chunks to save... [INFOCOUNT] Done saving part 25 [INFOTIMER] Finished waiting for chunks to save in 3.4859 seconds ```
tecpg run mlr-full -l 2000 -p 0.05 --cis ```python [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.0019 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.4152 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 2.0889 seconds [INFOTIMER] Finished reading 3 dataframes in 2.5062 seconds. [INFO] No region window provided. Resorting to default. [INFO] Using default window for cis of 1000000 bases [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 296 degrees of freedom [INFO] Initializing output directory [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0065 seconds [INFOTIMER] Converted G to tensor in 0.0088 seconds [INFOTIMER] Created ones in 0.0001 seconds [INFOTIMER] Created X in 0.0003 seconds [INFOTIMER] Transposed X in 0.0 seconds [INFOTIMER] Calculated XtXi in 0.6318 seconds [INFOTIMER] Calculated XtXi_diag in 0.0001 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0003 seconds [INFOTIMER] Calculated X constants in 0.648 seconds [INFO] CUDA device memory: 97.112064 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOCOUNT] Saving part 1/25: [INFOTIMER] Completed chunk 1/50000 in 3.9993 seconds. Average chunk time: 3.9993 seconds [INFOCOUNT] Done saving part 1 [INFOCOUNT] Saving part 2/25: [INFOTIMER] Completed chunk 2/50000 in 3.883 seconds. Average chunk time: 3.9411 seconds [INFOCOUNT] Done saving part 2 [INFOCOUNT] Saving part 3/25: [INFOTIMER] Completed chunk 3/50000 in 3.8792 seconds. Average chunk time: 3.9205 seconds [INFOCOUNT] Done saving part 3 [INFOCOUNT] Saving part 4/25: [INFOTIMER] Completed chunk 4/50000 in 3.8839 seconds. Average chunk time: 3.9114 seconds [INFOCOUNT] Done saving part 4 [INFOCOUNT] Saving part 5/25: [INFOTIMER] Completed chunk 5/50000 in 3.9043 seconds. Average chunk time: 3.9099 seconds [INFOCOUNT] Done saving part 5 [INFOCOUNT] Saving part 6/25: [INFOTIMER] Completed chunk 6/50000 in 3.8986 seconds. Average chunk time: 3.9081 seconds [INFOCOUNT] Done saving part 6 [INFOCOUNT] Saving part 7/25: [INFOTIMER] Completed chunk 7/50000 in 3.9159 seconds. Average chunk time: 3.9092 seconds [INFOCOUNT] Done saving part 7 [INFOCOUNT] Saving part 8/25: [INFOTIMER] Completed chunk 8/50000 in 3.8982 seconds. Average chunk time: 3.9078 seconds [INFOCOUNT] Done saving part 8 [INFOCOUNT] Saving part 9/25: [INFOTIMER] Completed chunk 9/50000 in 3.9417 seconds. Average chunk time: 3.9116 seconds [INFOCOUNT] Done saving part 9 [INFOCOUNT] Saving part 10/25: [INFOTIMER] Completed chunk 10/50000 in 3.8449 seconds. Average chunk time: 3.9049 seconds [INFOCOUNT] Done saving part 10 [INFOCOUNT] Saving part 11/25: [INFOTIMER] Completed chunk 11/50000 in 3.829 seconds. Average chunk time: 3.898 seconds [INFOCOUNT] Done saving part 11 [INFOCOUNT] Saving part 12/25: [INFOTIMER] Completed chunk 12/50000 in 3.8419 seconds. Average chunk time: 3.8933 seconds [INFOCOUNT] Done saving part 12 [INFOCOUNT] Saving part 13/25: [INFOTIMER] Completed chunk 13/50000 in 3.8332 seconds. Average chunk time: 3.8887 seconds [INFOCOUNT] Done saving part 13 [INFOCOUNT] Saving part 14/25: [INFOTIMER] Completed chunk 14/50000 in 3.8579 seconds. Average chunk time: 3.8865 seconds [INFOCOUNT] Done saving part 14 [INFOCOUNT] Saving part 15/25: [INFOTIMER] Completed chunk 15/50000 in 3.8253 seconds. Average chunk time: 3.8824 seconds [INFOCOUNT] Done saving part 15 [INFOCOUNT] Saving part 16/25: [INFOTIMER] Completed chunk 16/50000 in 3.8245 seconds. Average chunk time: 3.8788 seconds [INFOCOUNT] Done saving part 16 [INFOCOUNT] Saving part 17/25: [INFOTIMER] Completed chunk 17/50000 in 3.8404 seconds. Average chunk time: 3.8765 seconds [INFOCOUNT] Done saving part 17 [INFOCOUNT] Saving part 18/25: [INFOTIMER] Completed chunk 18/50000 in 3.8394 seconds. Average chunk time: 3.8745 seconds [INFOCOUNT] Done saving part 18 [INFOCOUNT] Saving part 19/25: [INFOTIMER] Completed chunk 19/50000 in 3.829 seconds. Average chunk time: 3.8721 seconds [INFOCOUNT] Done saving part 19 [INFOCOUNT] Saving part 20/25: [INFOTIMER] Completed chunk 20/50000 in 3.8246 seconds. Average chunk time: 3.8697 seconds [INFOCOUNT] Done saving part 20 [INFOCOUNT] Saving part 21/25: [INFOTIMER] Completed chunk 21/50000 in 3.9286 seconds. Average chunk time: 3.8725 seconds [INFOCOUNT] Done saving part 21 [INFOCOUNT] Saving part 22/25: [INFOTIMER] Completed chunk 22/50000 in 3.8869 seconds. Average chunk time: 3.8732 seconds [INFOCOUNT] Done saving part 22 [INFOCOUNT] Saving part 23/25: [INFOTIMER] Completed chunk 23/50000 in 3.8745 seconds. Average chunk time: 3.8732 seconds [INFOCOUNT] Done saving part 23 [INFOCOUNT] Saving part 24/25: [INFOTIMER] Completed chunk 24/50000 in 3.8759 seconds. Average chunk time: 3.8733 seconds [INFOCOUNT] Done saving part 24 [INFOCOUNT] Saving part 25/25: [INFOTIMER] Completed chunk 25/50000 in 3.8867 seconds. Average chunk time: 3.8739 seconds [INFOTIMER] Looped over methylation loci in 96.8521 seconds [INFOTIMER] Calculated regression_full in 97.5001 seconds [INFOTIMER] Waiting for chunks to save... [INFOCOUNT] Done saving part 25 [INFOTIMER] Finished waiting for chunks to save in 0.3214 seconds ```
tecpg run mlr-full -l 2000 -p 0.05 --distal ```python [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.0028 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.4454 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 2.1451 seconds [INFOTIMER] Finished reading 3 dataframes in 2.5934 seconds. [INFO] No region window provided. Resorting to default. [INFO] Using default window for distal of 50000 bases [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 296 degrees of freedom [INFO] Initializing output directory [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0067 seconds [INFOTIMER] Converted G to tensor in 0.0113 seconds [INFOTIMER] Created ones in 0.0001 seconds [INFOTIMER] Created X in 0.0003 seconds [INFOTIMER] Transposed X in 0.0 seconds [INFOTIMER] Calculated XtXi in 0.6423 seconds [INFOTIMER] Calculated XtXi_diag in 0.0001 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0002 seconds [INFOTIMER] Calculated X constants in 0.661 seconds [INFO] CUDA device memory: 97.112064 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOCOUNT] Saving part 1/25: [INFOTIMER] Completed chunk 1/50000 in 3.9742 seconds. Average chunk time: 3.9742 seconds [INFOCOUNT] Done saving part 1 [INFOCOUNT] Saving part 2/25: [INFOTIMER] Completed chunk 2/50000 in 3.9131 seconds. Average chunk time: 3.9436 seconds [INFOCOUNT] Done saving part 2 [INFOCOUNT] Saving part 3/25: [INFOTIMER] Completed chunk 3/50000 in 3.9238 seconds. Average chunk time: 3.937 seconds [INFOCOUNT] Done saving part 3 [INFOCOUNT] Saving part 4/25: [INFOTIMER] Completed chunk 4/50000 in 3.9276 seconds. Average chunk time: 3.9347 seconds [INFOCOUNT] Done saving part 4 [INFOCOUNT] Saving part 5/25: [INFOTIMER] Completed chunk 5/50000 in 3.9069 seconds. Average chunk time: 3.9291 seconds [INFOCOUNT] Done saving part 5 [INFOCOUNT] Saving part 6/25: [INFOTIMER] Completed chunk 6/50000 in 3.9297 seconds. Average chunk time: 3.9292 seconds [INFOCOUNT] Done saving part 6 [INFOCOUNT] Saving part 7/25: [INFOTIMER] Completed chunk 7/50000 in 3.9792 seconds. Average chunk time: 3.9364 seconds [INFOCOUNT] Done saving part 7 [INFOCOUNT] Saving part 8/25: [INFOTIMER] Completed chunk 8/50000 in 4.0366 seconds. Average chunk time: 3.9489 seconds [INFOCOUNT] Done saving part 8 [INFOCOUNT] Saving part 9/25: [INFOTIMER] Completed chunk 9/50000 in 4.1858 seconds. Average chunk time: 3.9752 seconds [INFOCOUNT] Done saving part 9 [INFOCOUNT] Saving part 10/25: [INFOTIMER] Completed chunk 10/50000 in 3.9697 seconds. Average chunk time: 3.9747 seconds [INFOCOUNT] Done saving part 10 [INFOCOUNT] Saving part 11/25: [INFOTIMER] Completed chunk 11/50000 in 3.8835 seconds. Average chunk time: 3.9664 seconds [INFOCOUNT] Done saving part 11 [INFOCOUNT] Saving part 12/25: [INFOTIMER] Completed chunk 12/50000 in 3.894 seconds. Average chunk time: 3.9603 seconds [INFOCOUNT] Done saving part 12 [INFOCOUNT] Saving part 13/25: [INFOTIMER] Completed chunk 13/50000 in 3.8788 seconds. Average chunk time: 3.9541 seconds [INFOCOUNT] Done saving part 13 [INFOCOUNT] Saving part 14/25: [INFOTIMER] Completed chunk 14/50000 in 3.8913 seconds. Average chunk time: 3.9496 seconds [INFOCOUNT] Done saving part 14 [INFOCOUNT] Saving part 15/25: [INFOTIMER] Completed chunk 15/50000 in 3.8756 seconds. Average chunk time: 3.9447 seconds [INFOCOUNT] Done saving part 15 [INFOCOUNT] Saving part 16/25: [INFOTIMER] Completed chunk 16/50000 in 3.8802 seconds. Average chunk time: 3.9406 seconds [INFOCOUNT] Done saving part 16 [INFOCOUNT] Saving part 17/25: [INFOTIMER] Completed chunk 17/50000 in 3.8966 seconds. Average chunk time: 3.938 seconds [INFOCOUNT] Done saving part 17 [INFOCOUNT] Saving part 18/25: [INFOTIMER] Completed chunk 18/50000 in 3.876 seconds. Average chunk time: 3.9346 seconds [INFOCOUNT] Done saving part 18 [INFOCOUNT] Saving part 19/25: [INFOTIMER] Completed chunk 19/50000 in 3.883 seconds. Average chunk time: 3.9319 seconds [INFOCOUNT] Done saving part 19 [INFOCOUNT] Saving part 20/25: [INFOTIMER] Completed chunk 20/50000 in 3.9026 seconds. Average chunk time: 3.9304 seconds [INFOCOUNT] Done saving part 20 [INFOCOUNT] Saving part 21/25: [INFOTIMER] Completed chunk 21/50000 in 3.9099 seconds. Average chunk time: 3.9294 seconds [INFOCOUNT] Done saving part 21 [INFOCOUNT] Saving part 22/25: [INFOTIMER] Completed chunk 22/50000 in 3.8953 seconds. Average chunk time: 3.9279 seconds [INFOCOUNT] Done saving part 22 [INFOCOUNT] Saving part 23/25: [INFOTIMER] Completed chunk 23/50000 in 3.8844 seconds. Average chunk time: 3.926 seconds [INFOCOUNT] Done saving part 23 [INFOCOUNT] Saving part 24/25: [INFOTIMER] Completed chunk 24/50000 in 3.8801 seconds. Average chunk time: 3.9241 seconds [INFOCOUNT] Done saving part 24 [INFOCOUNT] Saving part 25/25: [INFOTIMER] Completed chunk 25/50000 in 3.9401 seconds. Average chunk time: 3.9247 seconds [INFOTIMER] Looped over methylation loci in 98.1241 seconds [INFOTIMER] Calculated regression_full in 98.7852 seconds [INFOTIMER] Waiting for chunks to save... [INFOCOUNT] Done saving part 25 [INFOTIMER] Finished waiting for chunks to save in 0.4087 seconds ```
tecpg run mlr-full -l 2000 -p 0.05 --trans ```python [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.0018 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.4346 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 1.9731 seconds [INFOTIMER] Finished reading 3 dataframes in 2.4096 seconds. [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 296 degrees of freedom [INFO] Initializing output directory [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0064 seconds [INFOTIMER] Converted G to tensor in 0.0099 seconds [INFOTIMER] Created ones in 0.0002 seconds [INFOTIMER] Created X in 0.0004 seconds [INFOTIMER] Transposed X in 0.0001 seconds [INFOTIMER] Calculated XtXi in 0.5778 seconds [INFOTIMER] Calculated XtXi_diag in 0.0002 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0003 seconds [INFOTIMER] Calculated X constants in 0.5952 seconds [INFO] CUDA device memory: 97.112064 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOCOUNT] Saving part 1/25: [INFOTIMER] Completed chunk 1/50000 in 3.9888 seconds. Average chunk time: 3.9888 seconds [INFOCOUNT] Done saving part 1 [INFOCOUNT] Saving part 2/25: [INFOTIMER] Completed chunk 2/50000 in 3.9847 seconds. Average chunk time: 3.9867 seconds [INFOCOUNT] Saving part 3/25: [INFOTIMER] Completed chunk 3/50000 in 4.0818 seconds. Average chunk time: 4.0184 seconds [INFOCOUNT] Done saving part 2 [INFOCOUNT] Done saving part 3 [INFOCOUNT] Saving part 4/25: [INFOTIMER] Completed chunk 4/50000 in 4.0449 seconds. Average chunk time: 4.025 seconds [INFOCOUNT] Saving part 5/25: [INFOTIMER] Completed chunk 5/50000 in 4.0587 seconds. Average chunk time: 4.0318 seconds [INFOCOUNT] Done saving part 4 [INFOCOUNT] Saving part 6/25: [INFOTIMER] Completed chunk 6/50000 in 4.0583 seconds. Average chunk time: 4.0362 seconds [INFOCOUNT] Done saving part 5 [INFOCOUNT] Done saving part 6 [INFOCOUNT] Saving part 7/25: [INFOTIMER] Completed chunk 7/50000 in 4.0744 seconds. Average chunk time: 4.0416 seconds [INFOCOUNT] Saving part 8/25: [INFOTIMER] Completed chunk 8/50000 in 4.0699 seconds. Average chunk time: 4.0452 seconds [INFOCOUNT] Done saving part 7 [INFOCOUNT] Saving part 9/25: [INFOTIMER] Completed chunk 9/50000 in 4.0466 seconds. Average chunk time: 4.0453 seconds [INFOCOUNT] Done saving part 8 [INFOCOUNT] Done saving part 9 [INFOCOUNT] Saving part 10/25: [INFOTIMER] Completed chunk 10/50000 in 4.014 seconds. Average chunk time: 4.0422 seconds [INFOCOUNT] Done saving part 10 [INFOCOUNT] Saving part 11/25: [INFOTIMER] Completed chunk 11/50000 in 4.0693 seconds. Average chunk time: 4.0447 seconds [INFOCOUNT] Done saving part 11 [INFOCOUNT] Saving part 12/25: [INFOTIMER] Completed chunk 12/50000 in 4.0073 seconds. Average chunk time: 4.0415 seconds [INFOCOUNT] Done saving part 12 [INFOCOUNT] Saving part 13/25: [INFOTIMER] Completed chunk 13/50000 in 4.053 seconds. Average chunk time: 4.0424 seconds [INFOCOUNT] Done saving part 13 [INFOCOUNT] Saving part 14/25: [INFOTIMER] Completed chunk 14/50000 in 4.0255 seconds. Average chunk time: 4.0412 seconds [INFOCOUNT] Done saving part 14 [INFOCOUNT] Saving part 15/25: [INFOTIMER] Completed chunk 15/50000 in 4.0308 seconds. Average chunk time: 4.0405 seconds [INFOCOUNT] Done saving part 15 [INFOCOUNT] Saving part 16/25: [INFOTIMER] Completed chunk 16/50000 in 4.0787 seconds. Average chunk time: 4.0429 seconds [INFOCOUNT] Done saving part 16 [INFOCOUNT] Saving part 17/25: [INFOTIMER] Completed chunk 17/50000 in 4.0301 seconds. Average chunk time: 4.0422 seconds [INFOCOUNT] Done saving part 17 [INFOCOUNT] Saving part 18/25: [INFOTIMER] Completed chunk 18/50000 in 4.0527 seconds. Average chunk time: 4.0427 seconds [INFOCOUNT] Done saving part 18 [INFOCOUNT] Saving part 19/25: [INFOTIMER] Completed chunk 19/50000 in 4.0793 seconds. Average chunk time: 4.0447 seconds [INFOCOUNT] Done saving part 19 [INFOCOUNT] Saving part 20/25: [INFOTIMER] Completed chunk 20/50000 in 4.0253 seconds. Average chunk time: 4.0437 seconds [INFOCOUNT] Done saving part 20 [INFOCOUNT] Saving part 21/25: [INFOTIMER] Completed chunk 21/50000 in 4.0299 seconds. Average chunk time: 4.043 seconds [INFOCOUNT] Done saving part 21 [INFOCOUNT] Saving part 22/25: [INFOTIMER] Completed chunk 22/50000 in 4.054 seconds. Average chunk time: 4.0435 seconds [INFOCOUNT] Done saving part 22 [INFOCOUNT] Saving part 23/25: [INFOTIMER] Completed chunk 23/50000 in 4.0303 seconds. Average chunk time: 4.043 seconds [INFOCOUNT] Done saving part 23 [INFOCOUNT] Saving part 24/25: [INFOTIMER] Completed chunk 24/50000 in 4.0411 seconds. Average chunk time: 4.0429 seconds [INFOCOUNT] Done saving part 24 [INFOCOUNT] Saving part 25/25: [INFOTIMER] Completed chunk 25/50000 in 4.0634 seconds. Average chunk time: 4.0437 seconds [INFOTIMER] Looped over methylation loci in 101.1015 seconds [INFOTIMER] Calculated regression_full in 101.6967 seconds [INFOTIMER] Waiting for chunks to save... [INFOCOUNT] Done saving part 25 [INFOTIMER] Finished waiting for chunks to save in 3.2605 seconds ```
liamgd commented 1 year ago

Times for tests with 300 samples, 50000 methylation loci, and 10000 gene expression loci on a 2070 super with 8 GB VRAM (constant calculation and loop time, saving included only for chunked calculations):

No chunks, no saving:

tecpg run mlr-full: *Too much GPU memory allocation*
tecpg run mlr-full --cis: 93.1929 seconds
tecpg run mlr-full --distal: 94.3363 seconds
tecpg run mlr-full --trans: *Too much GPU memory allocation*

tecpg run mlr-full -p 0.05: 93.2036 seconds
tecpg run mlr-full -p 0.05 --cis: 93.427 seconds
tecpg run mlr-full -p 0.05 --distal: 93.5236 seconds
tecpg run mlr-full -p 0.05 --trans: 93.5954 seconds

Chunks of 2000 methylation loci, saving included:

tecpg run mlr-full -l 2000: 169.2086 seconds
tecpg run mlr-full -l 2000 --cis: 102.665 seconds
tecpg run mlr-full -l 2000 --distal: 104.6743 seconds
tecpg run mlr-full -l 2000 --trans: 180.5167 seconds

tecpg run mlr-full -l 2000 -p 0.05: 101.5071 seconds
tecpg run mlr-full -l 2000 -p 0.05 --cis: 97.5001 seconds
tecpg run mlr-full -l 2000 -p 0.05 --distal: 98.7852 seconds
tecpg run mlr-full -l 2000 -p 0.05 --trans: 101.6967 seconds
liamgd commented 1 year ago

Judging by these speeds, it seems that the current implementation of region filtration slightly lengthens run time but slightly reduces the time it takes to save (judging by the chunked analyses that included saving). It may be possible to reduce the speed further for cis, distal, and trans filtration by filtering the necessary tensors before each computation on a particular methylation locus.

This would entail creating a region filtration mask tensor before computation. The Y (or M_row if it is faster), XtXi_Xt, and X tensors would be filtered with this mask. Then, the p-value filtration mask would be calculated after and would further filter the region mask.

liamgd commented 1 year ago

With the above performance improvements, we obtain the following performance measurements.

Tests with 100 samples, 100 methylation loci, and 100 gene expression loci:
tecpg run mlr-full ```python [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.0048 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.0085 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 0.0111 seconds [INFOTIMER] Finished reading 3 dataframes in 0.0245 seconds. [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 96 degrees of freedom [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0003 seconds [INFOTIMER] Converted G to tensor in 0.0002 seconds [INFOTIMER] Created ones in 0.0001 seconds [INFOTIMER] Created X in 0.0001 seconds [INFOTIMER] Transposed X in 0.0 seconds [INFOTIMER] Calculated XtXi in 0.6043 seconds [INFOTIMER] Calculated XtXi_diag in 0.0001 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0001 seconds [INFOTIMER] Calculated X constants in 0.6052 seconds [INFO] CUDA device memory: 0.324096 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOTIMER] Looped over methylation loci in 0.1041 seconds [INFOTIMER] Calculated regression_full in 0.7093 seconds [INFO] Generating dataframe from results... [INFOTIMER] Finished creating indices in 0.0005 seconds [INFOTIMER] Finished creating preliminary dataframe in 0.0039 seconds [INFOTIMER] Created output dataframe in 0.0044 total seconds [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Saving 1 dataframes... [INFOTIMER] Saving 1/1: out.csv [INFOTIMER] Saved 1/1 in 0.0335 seconds [INFOTIMER] Finished saving 1 dataframes in 0.0336 seconds. ```
tecpg run mlr-full --cis ```python [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.0018 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.0052 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 0.0043 seconds [INFOTIMER] Finished reading 3 dataframes in 0.0114 seconds. [INFO] No region window provided. Resorting to default. [INFO] Using default window for cis of 1000000 bases [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 96 degrees of freedom [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0015 seconds [INFOTIMER] Converted G to tensor in 0.0001 seconds [INFOTIMER] Created ones in 0.0001 seconds [INFOTIMER] Created X in 0.0001 seconds [INFOTIMER] Transposed X in 0.0 seconds [INFOTIMER] Calculated XtXi in 0.5882 seconds [INFOTIMER] Calculated XtXi_diag in 0.0002 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0001 seconds [INFOTIMER] Calculated X constants in 0.5902 seconds [INFO] CUDA device memory: 0.326144 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOTIMER] Looped over methylation loci in 0.1215 seconds [INFOTIMER] Calculated regression_full in 0.7117 seconds [INFO] Generating dataframe from results... [INFOTIMER] Finished creating indices in 0.0003 seconds [INFOTIMER] Finished creating preliminary dataframe in 0.0013 seconds [INFOTIMER] Created output dataframe in 0.0016 total seconds [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Saving 1 dataframes... [INFOTIMER] Saving 1/1: out.csv [INFOTIMER] Saved 1/1 in 0.0016 seconds [INFOTIMER] Finished saving 1 dataframes in 0.0017 seconds. ```
tecpg run mlr-full --distal ```python [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.0019 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.0048 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 0.0045 seconds [INFOTIMER] Finished reading 3 dataframes in 0.0113 seconds. [INFO] No region window provided. Resorting to default. [INFO] Using default window for distal of 50000 bases [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 96 degrees of freedom [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0015 seconds [INFOTIMER] Converted G to tensor in 0.0001 seconds [INFOTIMER] Created ones in 0.0001 seconds [INFOTIMER] Created X in 0.0001 seconds [INFOTIMER] Transposed X in 0.0 seconds [INFOTIMER] Calculated XtXi in 0.5917 seconds [INFOTIMER] Calculated XtXi_diag in 0.0001 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0001 seconds [INFOTIMER] Calculated X constants in 0.5937 seconds [INFO] CUDA device memory: 0.326144 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOTIMER] Looped over methylation loci in 0.1764 seconds [INFOTIMER] Calculated regression_full in 0.7702 seconds [INFO] Generating dataframe from results... [INFOTIMER] Finished creating indices in 0.0003 seconds [INFOTIMER] Finished creating preliminary dataframe in 0.0015 seconds [INFOTIMER] Created output dataframe in 0.0019 total seconds [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Saving 1 dataframes... [INFOTIMER] Saving 1/1: out.csv [INFOTIMER] Saved 1/1 in 0.0033 seconds [INFOTIMER] Finished saving 1 dataframes in 0.0033 seconds. ```
tecpg run mlr-full --trans ```python [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.0017 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.0098 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 0.0043 seconds [INFOTIMER] Finished reading 3 dataframes in 0.016 seconds. [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 96 degrees of freedom [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0018 seconds [INFOTIMER] Converted G to tensor in 0.0001 seconds [INFOTIMER] Created ones in 0.0001 seconds [INFOTIMER] Created X in 0.0001 seconds [INFOTIMER] Transposed X in 0.0 seconds [INFOTIMER] Calculated XtXi in 0.5865 seconds [INFOTIMER] Calculated XtXi_diag in 0.0001 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0001 seconds [INFOTIMER] Calculated X constants in 0.5889 seconds [INFO] CUDA device memory: 0.326144 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOTIMER] Looped over methylation loci in 0.1337 seconds [INFOTIMER] Calculated regression_full in 0.7226 seconds [INFO] Generating dataframe from results... [INFOTIMER] Finished creating indices in 0.0004 seconds [INFOTIMER] Finished creating preliminary dataframe in 0.0026 seconds [INFOTIMER] Created output dataframe in 0.0031 total seconds [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Saving 1 dataframes... [INFOTIMER] Saving 1/1: out.csv [INFOTIMER] Saved 1/1 in 0.0318 seconds [INFOTIMER] Finished saving 1 dataframes in 0.0319 seconds. ```
tecpg run mlr-full -p 0.05 ```python [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.0017 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.0065 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 0.0047 seconds [INFOTIMER] Finished reading 3 dataframes in 0.0129 seconds. [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 96 degrees of freedom [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0003 seconds [INFOTIMER] Converted G to tensor in 0.0001 seconds [INFOTIMER] Created ones in 0.0001 seconds [INFOTIMER] Created X in 0.0001 seconds [INFOTIMER] Transposed X in 0.0 seconds [INFOTIMER] Calculated XtXi in 0.5775 seconds [INFOTIMER] Calculated XtXi_diag in 0.0001 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0001 seconds [INFOTIMER] Calculated X constants in 0.5783 seconds [INFO] CUDA device memory: 0.324096 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOTIMER] Looped over methylation loci in 0.1119 seconds [INFOTIMER] Calculated regression_full in 0.6903 seconds [INFO] Generating dataframe from results... [INFOTIMER] Finished creating indices in 0.0003 seconds [INFOTIMER] Finished creating preliminary dataframe in 0.0016 seconds [INFOTIMER] Created output dataframe in 0.0019 total seconds [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Saving 1 dataframes... [INFOTIMER] Saving 1/1: out.csv [INFOTIMER] Saved 1/1 in 0.003 seconds [INFOTIMER] Finished saving 1 dataframes in 0.0031 seconds. ```
tecpg run mlr-full -p 0.05 --cis ```python [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.0017 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.005 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 0.0044 seconds [INFOTIMER] Finished reading 3 dataframes in 0.0112 seconds. [INFO] No region window provided. Resorting to default. [INFO] Using default window for cis of 1000000 bases [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 96 degrees of freedom [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0015 seconds [INFOTIMER] Converted G to tensor in 0.0001 seconds [INFOTIMER] Created ones in 0.0001 seconds [INFOTIMER] Created X in 0.0001 seconds [INFOTIMER] Transposed X in 0.0 seconds [INFOTIMER] Calculated XtXi in 0.5814 seconds [INFOTIMER] Calculated XtXi_diag in 0.0001 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0001 seconds [INFOTIMER] Calculated X constants in 0.5834 seconds [INFO] CUDA device memory: 0.326144 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOTIMER] Looped over methylation loci in 0.134 seconds [INFOTIMER] Calculated regression_full in 0.7175 seconds [INFO] Generating dataframe from results... [INFOTIMER] Finished creating indices in 0.0004 seconds [INFOTIMER] Finished creating preliminary dataframe in 0.0013 seconds [INFOTIMER] Created output dataframe in 0.0018 total seconds [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Saving 1 dataframes... [INFOTIMER] Saving 1/1: out.csv [INFOTIMER] Saved 1/1 in 0.0011 seconds [INFOTIMER] Finished saving 1 dataframes in 0.0011 seconds. ```
tecpg run mlr-full -p 0.05 --distal ```python [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.0017 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.0047 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 0.0049 seconds [INFOTIMER] Finished reading 3 dataframes in 0.0114 seconds. [INFO] No region window provided. Resorting to default. [INFO] Using default window for distal of 50000 bases [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 96 degrees of freedom [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0015 seconds [INFOTIMER] Converted G to tensor in 0.0002 seconds [INFOTIMER] Created ones in 0.0001 seconds [INFOTIMER] Created X in 0.0001 seconds [INFOTIMER] Transposed X in 0.0 seconds [INFOTIMER] Calculated XtXi in 0.6173 seconds [INFOTIMER] Calculated XtXi_diag in 0.0001 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0001 seconds [INFOTIMER] Calculated X constants in 0.6193 seconds [INFO] CUDA device memory: 0.326144 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOTIMER] Looped over methylation loci in 0.1742 seconds [INFOTIMER] Calculated regression_full in 0.7935 seconds [INFO] Generating dataframe from results... [INFOTIMER] Finished creating indices in 0.0004 seconds [INFOTIMER] Finished creating preliminary dataframe in 0.0012 seconds [INFOTIMER] Created output dataframe in 0.0016 total seconds [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Saving 1 dataframes... [INFOTIMER] Saving 1/1: out.csv [INFOTIMER] Saved 1/1 in 0.0018 seconds [INFOTIMER] Finished saving 1 dataframes in 0.0018 seconds. ```
tecpg run mlr-full -p 0.05 --trans ```python [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.0018 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.005 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 0.0044 seconds [INFOTIMER] Finished reading 3 dataframes in 0.0113 seconds. [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 96 degrees of freedom [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0015 seconds [INFOTIMER] Converted G to tensor in 0.0002 seconds [INFOTIMER] Created ones in 0.0001 seconds [INFOTIMER] Created X in 0.0001 seconds [INFOTIMER] Transposed X in 0.0 seconds [INFOTIMER] Calculated XtXi in 0.5983 seconds [INFOTIMER] Calculated XtXi_diag in 0.0001 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0001 seconds [INFOTIMER] Calculated X constants in 0.6003 seconds [INFO] CUDA device memory: 0.326144 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOTIMER] Looped over methylation loci in 0.154 seconds [INFOTIMER] Calculated regression_full in 0.7544 seconds [INFO] Generating dataframe from results... [INFOTIMER] Finished creating indices in 0.0005 seconds [INFOTIMER] Finished creating preliminary dataframe in 0.0016 seconds [INFOTIMER] Created output dataframe in 0.0021 total seconds [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Saving 1 dataframes... [INFOTIMER] Saving 1/1: out.csv [INFOTIMER] Saved 1/1 in 0.0035 seconds [INFOTIMER] Finished saving 1 dataframes in 0.0035 seconds. ```
tecpg run mlr-full -l 2000 ```python [INFOCOUNT] Done saving part 1 [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.0018 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.0054 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 0.0047 seconds [INFOTIMER] Finished reading 3 dataframes in 0.012 seconds. [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 96 degrees of freedom [INFO] Initializing output directory [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0003 seconds [INFOTIMER] Converted G to tensor in 0.0002 seconds [INFOTIMER] Created ones in 0.0001 seconds [INFOTIMER] Created X in 0.0001 seconds [INFOTIMER] Transposed X in 0.0 seconds [INFOTIMER] Calculated XtXi in 0.5789 seconds [INFOTIMER] Calculated XtXi_diag in 0.0001 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0001 seconds [INFOTIMER] Calculated X constants in 0.5798 seconds [INFO] CUDA device memory: 0.324096 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOCOUNT] Saving part 1/1: [INFOTIMER] Completed chunk 1/100 in 0.0877 seconds. Average chunk time: 0.0877 seconds [INFOTIMER] Looped over methylation loci in 0.088 seconds [INFOTIMER] Calculated regression_full in 0.6678 seconds [INFOTIMER] Waiting for chunks to save... [INFOTIMER] Finished waiting for chunks to save in 1.1379 seconds ```
tecpg run mlr-full -l 2000 --cis ```python [INFOCOUNT] Done saving part 1 [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.0017 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.0047 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 0.0052 seconds [INFOTIMER] Finished reading 3 dataframes in 0.0117 seconds. [INFO] No region window provided. Resorting to default. [INFO] Using default window for cis of 1000000 bases [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 96 degrees of freedom [INFO] Initializing output directory [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0017 seconds [INFOTIMER] Converted G to tensor in 0.0002 seconds [INFOTIMER] Created ones in 0.0001 seconds [INFOTIMER] Created X in 0.0001 seconds [INFOTIMER] Transposed X in 0.0 seconds [INFOTIMER] Calculated XtXi in 0.5897 seconds [INFOTIMER] Calculated XtXi_diag in 0.0001 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0001 seconds [INFOTIMER] Calculated X constants in 0.592 seconds [INFO] CUDA device memory: 0.326144 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOCOUNT] Saving part 1/1: [INFOTIMER] Completed chunk 1/100 in 0.13 seconds. Average chunk time: 0.13 seconds [INFOTIMER] Looped over methylation loci in 0.1305 seconds [INFOTIMER] Calculated regression_full in 0.7225 seconds [INFOTIMER] Waiting for chunks to save... [INFOTIMER] Finished waiting for chunks to save in 1.1288 seconds ```
tecpg run mlr-full -l 2000 --distal ```python [INFOCOUNT] Done saving part 1 [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.0018 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.0047 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 0.0043 seconds [INFOTIMER] Finished reading 3 dataframes in 0.0108 seconds. [INFO] No region window provided. Resorting to default. [INFO] Using default window for distal of 50000 bases [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 96 degrees of freedom [INFO] Initializing output directory [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0015 seconds [INFOTIMER] Converted G to tensor in 0.0001 seconds [INFOTIMER] Created ones in 0.0001 seconds [INFOTIMER] Created X in 0.0001 seconds [INFOTIMER] Transposed X in 0.0 seconds [INFOTIMER] Calculated XtXi in 0.5771 seconds [INFOTIMER] Calculated XtXi_diag in 0.0001 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0001 seconds [INFOTIMER] Calculated X constants in 0.579 seconds [INFO] CUDA device memory: 0.326144 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOCOUNT] Saving part 1/1: [INFOTIMER] Completed chunk 1/100 in 0.165 seconds. Average chunk time: 0.165 seconds [INFOTIMER] Looped over methylation loci in 0.1654 seconds [INFOTIMER] Calculated regression_full in 0.7445 seconds [INFOTIMER] Waiting for chunks to save... [INFOTIMER] Finished waiting for chunks to save in 1.0307 seconds ```
tecpg run mlr-full -l 2000 --trans ```python [INFOCOUNT] Done saving part 1 [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.0017 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.006 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 0.0047 seconds [INFOTIMER] Finished reading 3 dataframes in 0.0125 seconds. [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 96 degrees of freedom [INFO] Initializing output directory [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0015 seconds [INFOTIMER] Converted G to tensor in 0.0001 seconds [INFOTIMER] Created ones in 0.0001 seconds [INFOTIMER] Created X in 0.0001 seconds [INFOTIMER] Transposed X in 0.0 seconds [INFOTIMER] Calculated XtXi in 0.5883 seconds [INFOTIMER] Calculated XtXi_diag in 0.0001 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0001 seconds [INFOTIMER] Calculated X constants in 0.5903 seconds [INFO] CUDA device memory: 0.326144 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOCOUNT] Saving part 1/1: [INFOTIMER] Completed chunk 1/100 in 0.1396 seconds. Average chunk time: 0.1396 seconds [INFOTIMER] Looped over methylation loci in 0.1402 seconds [INFOTIMER] Calculated regression_full in 0.7306 seconds [INFOTIMER] Waiting for chunks to save... [INFOTIMER] Finished waiting for chunks to save in 1.0957 seconds ```
tecpg run mlr-full -l 2000 -p 0.05 ```python [INFOCOUNT] Done saving part 1 [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.0018 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.0048 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 0.0042 seconds [INFOTIMER] Finished reading 3 dataframes in 0.0108 seconds. [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 96 degrees of freedom [INFO] Initializing output directory [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0003 seconds [INFOTIMER] Converted G to tensor in 0.0002 seconds [INFOTIMER] Created ones in 0.0001 seconds [INFOTIMER] Created X in 0.0001 seconds [INFOTIMER] Transposed X in 0.0 seconds [INFOTIMER] Calculated XtXi in 0.5766 seconds [INFOTIMER] Calculated XtXi_diag in 0.0001 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0001 seconds [INFOTIMER] Calculated X constants in 0.5774 seconds [INFO] CUDA device memory: 0.324096 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOCOUNT] Saving part 1/1: [INFOTIMER] Completed chunk 1/100 in 0.1181 seconds. Average chunk time: 0.1181 seconds [INFOTIMER] Looped over methylation loci in 0.1186 seconds [INFOTIMER] Calculated regression_full in 0.696 seconds [INFOTIMER] Waiting for chunks to save... [INFOTIMER] Finished waiting for chunks to save in 1.0613 seconds ```
tecpg run mlr-full -l 2000 -p 0.05 --cis ```python [INFOCOUNT] Done saving part 1 [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.0017 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.0048 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 0.0046 seconds [INFOTIMER] Finished reading 3 dataframes in 0.0112 seconds. [INFO] No region window provided. Resorting to default. [INFO] Using default window for cis of 1000000 bases [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 96 degrees of freedom [INFO] Initializing output directory [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0015 seconds [INFOTIMER] Converted G to tensor in 0.0001 seconds [INFOTIMER] Created ones in 0.0001 seconds [INFOTIMER] Created X in 0.0001 seconds [INFOTIMER] Transposed X in 0.0 seconds [INFOTIMER] Calculated XtXi in 0.5838 seconds [INFOTIMER] Calculated XtXi_diag in 0.0001 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0001 seconds [INFOTIMER] Calculated X constants in 0.5858 seconds [INFO] CUDA device memory: 0.326144 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOCOUNT] Saving part 1/1: [INFOTIMER] Completed chunk 1/100 in 0.1403 seconds. Average chunk time: 0.1403 seconds [INFOTIMER] Looped over methylation loci in 0.1408 seconds [INFOTIMER] Calculated regression_full in 0.7266 seconds [INFOTIMER] Waiting for chunks to save... [INFOTIMER] Finished waiting for chunks to save in 1.0248 seconds ```
tecpg run mlr-full -l 2000 -p 0.05 --distal ```python [INFOCOUNT] Done saving part 1 [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.0021 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.0052 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 0.0045 seconds [INFOTIMER] Finished reading 3 dataframes in 0.0118 seconds. [INFO] No region window provided. Resorting to default. [INFO] Using default window for distal of 50000 bases [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 96 degrees of freedom [INFO] Initializing output directory [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0015 seconds [INFOTIMER] Converted G to tensor in 0.0001 seconds [INFOTIMER] Created ones in 0.0001 seconds [INFOTIMER] Created X in 0.0001 seconds [INFOTIMER] Transposed X in 0.0 seconds [INFOTIMER] Calculated XtXi in 0.579 seconds [INFOTIMER] Calculated XtXi_diag in 0.0001 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0001 seconds [INFOTIMER] Calculated X constants in 0.5811 seconds [INFO] CUDA device memory: 0.326144 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOCOUNT] Saving part 1/1: [INFOTIMER] Completed chunk 1/100 in 0.1904 seconds. Average chunk time: 0.1904 seconds [INFOTIMER] Looped over methylation loci in 0.1909 seconds [INFOTIMER] Calculated regression_full in 0.772 seconds [INFOTIMER] Waiting for chunks to save... [INFOTIMER] Finished waiting for chunks to save in 1.0128 seconds ```
tecpg run mlr-full -l 2000 -p 0.05 --trans ```python [INFOCOUNT] Done saving part 1 [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.0017 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.0063 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 0.0045 seconds [INFOTIMER] Finished reading 3 dataframes in 0.0125 seconds. [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 96 degrees of freedom [INFO] Initializing output directory [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0015 seconds [INFOTIMER] Converted G to tensor in 0.0001 seconds [INFOTIMER] Created ones in 0.0001 seconds [INFOTIMER] Created X in 0.0001 seconds [INFOTIMER] Transposed X in 0.0 seconds [INFOTIMER] Calculated XtXi in 0.5917 seconds [INFOTIMER] Calculated XtXi_diag in 0.0001 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0001 seconds [INFOTIMER] Calculated X constants in 0.5937 seconds [INFO] CUDA device memory: 0.326144 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOCOUNT] Saving part 1/1: [INFOTIMER] Completed chunk 1/100 in 0.1556 seconds. Average chunk time: 0.1556 seconds [INFOTIMER] Looped over methylation loci in 0.1562 seconds [INFOTIMER] Calculated regression_full in 0.7499 seconds [INFOTIMER] Waiting for chunks to save... [INFOTIMER] Finished waiting for chunks to save in 1.0413 seconds ```
Tests with 300 samples, 1000 methylation loci, and 1000 gene expression loci:
tecpg run mlr-full ```python [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.0063 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.0572 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 0.0567 seconds [INFOTIMER] Finished reading 3 dataframes in 0.1203 seconds. [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 296 degrees of freedom [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0004 seconds [INFOTIMER] Converted G to tensor in 0.0022 seconds [INFOTIMER] Created ones in 0.0002 seconds [INFOTIMER] Created X in 0.0001 seconds [INFOTIMER] Transposed X in 0.0 seconds [INFOTIMER] Calculated XtXi in 0.6776 seconds [INFOTIMER] Calculated XtXi_diag in 0.0001 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0001 seconds [INFOTIMER] Calculated X constants in 0.6808 seconds [INFO] CUDA device memory: 9.618432 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOTIMER] Looped over methylation loci in 0.4808 seconds [INFOTIMER] Calculated regression_full in 1.1617 seconds [INFO] Generating dataframe from results... [INFOTIMER] Finished creating indices in 0.0059 seconds [INFOTIMER] Finished creating preliminary dataframe in 0.0757 seconds [INFOTIMER] Created output dataframe in 0.0816 total seconds [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Saving 1 dataframes... [INFOTIMER] Saving 1/1: out.csv [INFOTIMER] Saved 1/1 in 4.0106 seconds [INFOTIMER] Finished saving 1 dataframes in 4.0106 seconds. ```
tecpg run mlr-full --cis ```python [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.0019 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.0576 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 0.0652 seconds [INFOTIMER] Finished reading 3 dataframes in 0.1249 seconds. [INFO] No region window provided. Resorting to default. [INFO] Using default window for cis of 1000000 bases [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 296 degrees of freedom [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0018 seconds [INFOTIMER] Converted G to tensor in 0.0016 seconds [INFOTIMER] Created ones in 0.0001 seconds [INFOTIMER] Created X in 0.0001 seconds [INFOTIMER] Transposed X in 0.0 seconds [INFOTIMER] Calculated XtXi in 0.6821 seconds [INFOTIMER] Calculated XtXi_diag in 0.0001 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0001 seconds [INFOTIMER] Calculated X constants in 0.6859 seconds [INFO] CUDA device memory: 9.634816 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOTIMER] Looped over methylation loci in 0.7974 seconds [INFOTIMER] Calculated regression_full in 1.4834 seconds [INFO] Generating dataframe from results... [INFOTIMER] Finished creating indices in 0.0048 seconds [INFOTIMER] Finished creating preliminary dataframe in 0.0017 seconds [INFOTIMER] Created output dataframe in 0.0066 total seconds [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Saving 1 dataframes... [INFOTIMER] Saving 1/1: out.csv [INFOTIMER] Saved 1/1 in 0.0037 seconds [INFOTIMER] Finished saving 1 dataframes in 0.0037 seconds. ```
tecpg run mlr-full --distal ```python [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.0019 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.0547 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 0.05 seconds [INFOTIMER] Finished reading 3 dataframes in 0.1067 seconds. [INFO] No region window provided. Resorting to default. [INFO] Using default window for distal of 50000 bases [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 296 degrees of freedom [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0029 seconds [INFOTIMER] Converted G to tensor in 0.004 seconds [INFOTIMER] Created ones in 0.0003 seconds [INFOTIMER] Created X in 0.0002 seconds [INFOTIMER] Transposed X in 0.0001 seconds [INFOTIMER] Calculated XtXi in 0.7831 seconds [INFOTIMER] Calculated XtXi_diag in 0.0001 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0001 seconds [INFOTIMER] Calculated X constants in 0.7908 seconds [INFO] CUDA device memory: 9.634816 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOTIMER] Looped over methylation loci in 1.0143 seconds [INFOTIMER] Calculated regression_full in 1.8051 seconds [INFO] Generating dataframe from results... [INFOTIMER] Finished creating indices in 0.007 seconds [INFOTIMER] Finished creating preliminary dataframe in 0.0064 seconds [INFOTIMER] Created output dataframe in 0.0134 total seconds [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Saving 1 dataframes... [INFOTIMER] Saving 1/1: out.csv [INFOTIMER] Saved 1/1 in 0.1512 seconds [INFOTIMER] Finished saving 1 dataframes in 0.1513 seconds. ```
tecpg run mlr-full --trans ```python [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.0018 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.0534 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 0.0575 seconds [INFOTIMER] Finished reading 3 dataframes in 0.1128 seconds. [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 296 degrees of freedom [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.002 seconds [INFOTIMER] Converted G to tensor in 0.002 seconds [INFOTIMER] Created ones in 0.0001 seconds [INFOTIMER] Created X in 0.0001 seconds [INFOTIMER] Transposed X in 0.0 seconds [INFOTIMER] Calculated XtXi in 0.7073 seconds [INFOTIMER] Calculated XtXi_diag in 0.0001 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0001 seconds [INFOTIMER] Calculated X constants in 0.7117 seconds [INFO] CUDA device memory: 9.634816 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOTIMER] Looped over methylation loci in 0.9031 seconds [INFOTIMER] Calculated regression_full in 1.6148 seconds [INFO] Generating dataframe from results... [INFOTIMER] Finished creating indices in 0.0109 seconds [INFOTIMER] Finished creating preliminary dataframe in 0.0754 seconds [INFOTIMER] Created output dataframe in 0.0863 total seconds [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Saving 1 dataframes... [INFOTIMER] Saving 1/1: out.csv [INFOTIMER] Saved 1/1 in 3.7962 seconds [INFOTIMER] Finished saving 1 dataframes in 3.7963 seconds. ```
tecpg run mlr-full -p 0.05 ```python [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.0018 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.054 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 0.0575 seconds [INFOTIMER] Finished reading 3 dataframes in 0.1134 seconds. [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 296 degrees of freedom [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0005 seconds [INFOTIMER] Converted G to tensor in 0.002 seconds [INFOTIMER] Created ones in 0.0001 seconds [INFOTIMER] Created X in 0.0001 seconds [INFOTIMER] Transposed X in 0.0 seconds [INFOTIMER] Calculated XtXi in 0.6579 seconds [INFOTIMER] Calculated XtXi_diag in 0.0001 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0001 seconds [INFOTIMER] Calculated X constants in 0.6609 seconds [INFO] CUDA device memory: 9.618432 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOTIMER] Looped over methylation loci in 0.632 seconds [INFOTIMER] Calculated regression_full in 1.293 seconds [INFO] Generating dataframe from results... [INFOTIMER] Finished creating indices in 0.0058 seconds [INFOTIMER] Finished creating preliminary dataframe in 0.0053 seconds [INFOTIMER] Created output dataframe in 0.0111 total seconds [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Saving 1 dataframes... [INFOTIMER] Saving 1/1: out.csv [INFOTIMER] Saved 1/1 in 0.1458 seconds [INFOTIMER] Finished saving 1 dataframes in 0.1458 seconds. ```
tecpg run mlr-full -p 0.05 --cis ```python [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.0018 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.0508 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 0.0511 seconds [INFOTIMER] Finished reading 3 dataframes in 0.1038 seconds. [INFO] No region window provided. Resorting to default. [INFO] Using default window for cis of 1000000 bases [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 296 degrees of freedom [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0029 seconds [INFOTIMER] Converted G to tensor in 0.0021 seconds [INFOTIMER] Created ones in 0.0001 seconds [INFOTIMER] Created X in 0.0001 seconds [INFOTIMER] Transposed X in 0.0 seconds [INFOTIMER] Calculated XtXi in 0.6169 seconds [INFOTIMER] Calculated XtXi_diag in 0.0001 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0001 seconds [INFOTIMER] Calculated X constants in 0.6224 seconds [INFO] CUDA device memory: 9.634816 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOTIMER] Looped over methylation loci in 0.8851 seconds [INFOTIMER] Calculated regression_full in 1.5075 seconds [INFO] Generating dataframe from results... [INFOTIMER] Finished creating indices in 0.0051 seconds [INFOTIMER] Finished creating preliminary dataframe in 0.0012 seconds [INFOTIMER] Created output dataframe in 0.0064 total seconds [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Saving 1 dataframes... [INFOTIMER] Saving 1/1: out.csv [INFOTIMER] Saved 1/1 in 0.0018 seconds [INFOTIMER] Finished saving 1 dataframes in 0.0018 seconds. ```
tecpg run mlr-full -p 0.05 --distal ```python [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.0022 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.0615 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 0.0498 seconds [INFOTIMER] Finished reading 3 dataframes in 0.1137 seconds. [INFO] No region window provided. Resorting to default. [INFO] Using default window for distal of 50000 bases [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 296 degrees of freedom [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0017 seconds [INFOTIMER] Converted G to tensor in 0.0016 seconds [INFOTIMER] Created ones in 0.0001 seconds [INFOTIMER] Created X in 0.0001 seconds [INFOTIMER] Transposed X in 0.0 seconds [INFOTIMER] Calculated XtXi in 0.6174 seconds [INFOTIMER] Calculated XtXi_diag in 0.0001 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0001 seconds [INFOTIMER] Calculated X constants in 0.6212 seconds [INFO] CUDA device memory: 9.634816 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOTIMER] Looped over methylation loci in 1.0373 seconds [INFOTIMER] Calculated regression_full in 1.6585 seconds [INFO] Generating dataframe from results... [INFOTIMER] Finished creating indices in 0.0057 seconds [INFOTIMER] Finished creating preliminary dataframe in 0.002 seconds [INFOTIMER] Created output dataframe in 0.0077 total seconds [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Saving 1 dataframes... [INFOTIMER] Saving 1/1: out.csv [INFOTIMER] Saved 1/1 in 0.0077 seconds [INFOTIMER] Finished saving 1 dataframes in 0.0078 seconds. ```
tecpg run mlr-full -p 0.05 --trans ```python [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.0018 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.0512 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 0.0505 seconds [INFOTIMER] Finished reading 3 dataframes in 0.1036 seconds. [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 296 degrees of freedom [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0017 seconds [INFOTIMER] Converted G to tensor in 0.0019 seconds [INFOTIMER] Created ones in 0.0002 seconds [INFOTIMER] Created X in 0.0002 seconds [INFOTIMER] Transposed X in 0.0 seconds [INFOTIMER] Calculated XtXi in 0.6295 seconds [INFOTIMER] Calculated XtXi_diag in 0.0001 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0001 seconds [INFOTIMER] Calculated X constants in 0.6337 seconds [INFO] CUDA device memory: 9.634816 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOTIMER] Looped over methylation loci in 0.9403 seconds [INFOTIMER] Calculated regression_full in 1.5741 seconds [INFO] Generating dataframe from results... [INFOTIMER] Finished creating indices in 0.0101 seconds [INFOTIMER] Finished creating preliminary dataframe in 0.0064 seconds [INFOTIMER] Created output dataframe in 0.0166 total seconds [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Saving 1 dataframes... [INFOTIMER] Saving 1/1: out.csv [INFOTIMER] Saved 1/1 in 0.1304 seconds [INFOTIMER] Finished saving 1 dataframes in 0.1304 seconds. ```
tecpg run mlr-full -l 2000 ```python [INFOCOUNT] Done saving part 1 [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.0029 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.0574 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 0.0613 seconds [INFOTIMER] Finished reading 3 dataframes in 0.1216 seconds. [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 296 degrees of freedom [INFO] Initializing output directory [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0006 seconds [INFOTIMER] Converted G to tensor in 0.0017 seconds [INFOTIMER] Created ones in 0.0001 seconds [INFOTIMER] Created X in 0.0001 seconds [INFOTIMER] Transposed X in 0.0 seconds [INFOTIMER] Calculated XtXi in 0.6498 seconds [INFOTIMER] Calculated XtXi_diag in 0.0001 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0001 seconds [INFOTIMER] Calculated X constants in 0.6524 seconds [INFO] CUDA device memory: 9.618432 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOCOUNT] Saving part 1/1: [INFOTIMER] Completed chunk 1/1000 in 0.5424 seconds. Average chunk time: 0.5424 seconds [INFOTIMER] Looped over methylation loci in 0.543 seconds [INFOTIMER] Calculated regression_full in 1.1955 seconds [INFOTIMER] Waiting for chunks to save... [INFOTIMER] Finished waiting for chunks to save in 4.4746 seconds ```
tecpg run mlr-full -l 2000 --cis ```python [INFOCOUNT] Done saving part 1 [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.0019 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.0548 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 0.0557 seconds [INFOTIMER] Finished reading 3 dataframes in 0.1124 seconds. [INFO] No region window provided. Resorting to default. [INFO] Using default window for cis of 1000000 bases [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 296 degrees of freedom [INFO] Initializing output directory [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0023 seconds [INFOTIMER] Converted G to tensor in 0.002 seconds [INFOTIMER] Created ones in 0.0002 seconds [INFOTIMER] Created X in 0.0001 seconds [INFOTIMER] Transposed X in 0.0 seconds [INFOTIMER] Calculated XtXi in 0.6191 seconds [INFOTIMER] Calculated XtXi_diag in 0.0001 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0001 seconds [INFOTIMER] Calculated X constants in 0.6239 seconds [INFO] CUDA device memory: 9.634816 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOCOUNT] Saving part 1/1: [INFOTIMER] Completed chunk 1/1000 in 0.7648 seconds. Average chunk time: 0.7648 seconds [INFOTIMER] Looped over methylation loci in 0.7659 seconds [INFOTIMER] Calculated regression_full in 1.3898 seconds [INFOTIMER] Waiting for chunks to save... [INFOTIMER] Finished waiting for chunks to save in 1.0291 seconds ```
tecpg run mlr-full -l 2000 --distal ```python [INFOCOUNT] Done saving part 1 [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.0018 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.0518 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 0.0505 seconds [INFOTIMER] Finished reading 3 dataframes in 0.1042 seconds. [INFO] No region window provided. Resorting to default. [INFO] Using default window for distal of 50000 bases [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 296 degrees of freedom [INFO] Initializing output directory [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0019 seconds [INFOTIMER] Converted G to tensor in 0.0018 seconds [INFOTIMER] Created ones in 0.0001 seconds [INFOTIMER] Created X in 0.0001 seconds [INFOTIMER] Transposed X in 0.0 seconds [INFOTIMER] Calculated XtXi in 0.6057 seconds [INFOTIMER] Calculated XtXi_diag in 0.0001 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0001 seconds [INFOTIMER] Calculated X constants in 0.6098 seconds [INFO] CUDA device memory: 9.634816 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOCOUNT] Saving part 1/1: [INFOTIMER] Completed chunk 1/1000 in 0.9017 seconds. Average chunk time: 0.9017 seconds [INFOTIMER] Looped over methylation loci in 0.9031 seconds [INFOTIMER] Calculated regression_full in 1.5129 seconds [INFOTIMER] Waiting for chunks to save... [INFOTIMER] Finished waiting for chunks to save in 1.1608 seconds ```
tecpg run mlr-full -l 2000 --trans ```python [INFOCOUNT] Done saving part 1 [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.0019 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.0522 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 0.0515 seconds [INFOTIMER] Finished reading 3 dataframes in 0.1057 seconds. [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 296 degrees of freedom [INFO] Initializing output directory [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0018 seconds [INFOTIMER] Converted G to tensor in 0.0016 seconds [INFOTIMER] Created ones in 0.0001 seconds [INFOTIMER] Created X in 0.0001 seconds [INFOTIMER] Transposed X in 0.0 seconds [INFOTIMER] Calculated XtXi in 0.6136 seconds [INFOTIMER] Calculated XtXi_diag in 0.0001 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0001 seconds [INFOTIMER] Calculated X constants in 0.6175 seconds [INFO] CUDA device memory: 9.634816 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOCOUNT] Saving part 1/1: [INFOTIMER] Completed chunk 1/1000 in 0.8944 seconds. Average chunk time: 0.8944 seconds [INFOTIMER] Looped over methylation loci in 0.8959 seconds [INFOTIMER] Calculated regression_full in 1.5134 seconds [INFOTIMER] Waiting for chunks to save... [INFOTIMER] Finished waiting for chunks to save in 4.3024 seconds ```
tecpg run mlr-full -l 2000 -p 0.05 ```python [INFOCOUNT] Done saving part 1 [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.0019 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.0569 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 0.0593 seconds [INFOTIMER] Finished reading 3 dataframes in 0.1181 seconds. [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 296 degrees of freedom [INFO] Initializing output directory [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0006 seconds [INFOTIMER] Converted G to tensor in 0.0016 seconds [INFOTIMER] Created ones in 0.0001 seconds [INFOTIMER] Created X in 0.0001 seconds [INFOTIMER] Transposed X in 0.0 seconds [INFOTIMER] Calculated XtXi in 0.6055 seconds [INFOTIMER] Calculated XtXi_diag in 0.0001 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0001 seconds [INFOTIMER] Calculated X constants in 0.6081 seconds [INFO] CUDA device memory: 9.618432 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOCOUNT] Saving part 1/1: [INFOTIMER] Completed chunk 1/1000 in 0.5945 seconds. Average chunk time: 0.5945 seconds [INFOTIMER] Looped over methylation loci in 0.596 seconds [INFOTIMER] Calculated regression_full in 1.2041 seconds [INFOTIMER] Waiting for chunks to save... [INFOTIMER] Finished waiting for chunks to save in 1.1852 seconds ```
tecpg run mlr-full -l 2000 -p 0.05 --cis ```python [INFOCOUNT] Done saving part 1 [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.0019 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.0572 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 0.0591 seconds [INFOTIMER] Finished reading 3 dataframes in 0.1182 seconds. [INFO] No region window provided. Resorting to default. [INFO] Using default window for cis of 1000000 bases [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 296 degrees of freedom [INFO] Initializing output directory [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0017 seconds [INFOTIMER] Converted G to tensor in 0.0017 seconds [INFOTIMER] Created ones in 0.0001 seconds [INFOTIMER] Created X in 0.0001 seconds [INFOTIMER] Transposed X in 0.0 seconds [INFOTIMER] Calculated XtXi in 0.6176 seconds [INFOTIMER] Calculated XtXi_diag in 0.0001 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0001 seconds [INFOTIMER] Calculated X constants in 0.6215 seconds [INFO] CUDA device memory: 9.634816 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOCOUNT] Saving part 1/1: [INFOTIMER] Completed chunk 1/1000 in 0.8825 seconds. Average chunk time: 0.8825 seconds [INFOTIMER] Looped over methylation loci in 0.8838 seconds [INFOTIMER] Calculated regression_full in 1.5052 seconds [INFOTIMER] Waiting for chunks to save... [INFOTIMER] Finished waiting for chunks to save in 1.0238 seconds ```
tecpg run mlr-full -l 2000 -p 0.05 --distal ```python [INFOCOUNT] Done saving part 1 [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.002 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.0532 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 0.0531 seconds [INFOTIMER] Finished reading 3 dataframes in 0.1085 seconds. [INFO] No region window provided. Resorting to default. [INFO] Using default window for distal of 50000 bases [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 296 degrees of freedom [INFO] Initializing output directory [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0018 seconds [INFOTIMER] Converted G to tensor in 0.0022 seconds [INFOTIMER] Created ones in 0.0002 seconds [INFOTIMER] Created X in 0.0001 seconds [INFOTIMER] Transposed X in 0.0 seconds [INFOTIMER] Calculated XtXi in 0.6436 seconds [INFOTIMER] Calculated XtXi_diag in 0.0001 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0001 seconds [INFOTIMER] Calculated X constants in 0.6482 seconds [INFO] CUDA device memory: 9.634816 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOCOUNT] Saving part 1/1: [INFOTIMER] Completed chunk 1/1000 in 1.03 seconds. Average chunk time: 1.03 seconds [INFOTIMER] Looped over methylation loci in 1.0321 seconds [INFOTIMER] Calculated regression_full in 1.6803 seconds [INFOTIMER] Waiting for chunks to save... [INFOTIMER] Finished waiting for chunks to save in 1.042 seconds ```
tecpg run mlr-full -l 2000 -p 0.05 --trans ```python [INFOCOUNT] Done saving part 1 [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.002 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.0521 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 0.0525 seconds [INFOTIMER] Finished reading 3 dataframes in 0.1067 seconds. [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 296 degrees of freedom [INFO] Initializing output directory [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0017 seconds [INFOTIMER] Converted G to tensor in 0.0018 seconds [INFOTIMER] Created ones in 0.0002 seconds [INFOTIMER] Created X in 0.0001 seconds [INFOTIMER] Transposed X in 0.0 seconds [INFOTIMER] Calculated XtXi in 0.6157 seconds [INFOTIMER] Calculated XtXi_diag in 0.0001 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0001 seconds [INFOTIMER] Calculated X constants in 0.6198 seconds [INFO] CUDA device memory: 9.634816 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOCOUNT] Saving part 1/1: [INFOTIMER] Completed chunk 1/1000 in 0.9711 seconds. Average chunk time: 0.9711 seconds [INFOTIMER] Looped over methylation loci in 0.9737 seconds [INFOTIMER] Calculated regression_full in 1.5935 seconds [INFOTIMER] Waiting for chunks to save... [INFOTIMER] Finished waiting for chunks to save in 1.1614 seconds ```
Tests with 300 samples, 50000 methylation loci, and 10000 gene expression loci:
tecpg run mlr-full ```python [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.0072 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.4604 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 2.2715 seconds [INFOTIMER] Finished reading 3 dataframes in 2.7392 seconds. [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 296 degrees of freedom [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0007 seconds [INFOTIMER] Converted G to tensor in 0.013 seconds [INFOTIMER] Created ones in 0.0002 seconds [INFOTIMER] Created X in 0.0003 seconds [INFOTIMER] Transposed X in 0.0 seconds [INFOTIMER] Calculated XtXi in 0.7293 seconds [INFOTIMER] Calculated XtXi_diag in 0.0001 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0003 seconds [INFOTIMER] Calculated X constants in 0.7439 seconds [INFO] CUDA device memory: 96.630784 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... Traceback (most recent call last): File "\Python310\Scripts\tecpg-script.py", line 33, in sys.exit(load_entry_point('tecpg', 'console_scripts', 'tecpg')()) File "\Python310\Scripts\tecpg-script.py", line 25, in importlib_load_entry_point return next(matches).load() File "\Python310\lib\importlib\metadata\__init__.py", line 162, in load module = import_module(match.group('module')) File "\Python310\lib\importlib\__init__.py", line 126, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "", line 1050, in _gcd_import File "", line 1027, in _find_and_load File "", line 1006, in _find_and_load_unlocked File "", line 688, in _load_unlocked File "", line 883, in exec_module File "", line 241, in _call_with_frames_removed File "\tecpg\__main__.py", line 9, in main() File "\tecpg\__main__.py", line 6, in main start() File "\tecpg\cli.py", line 533, in start cli(obj={}) File "\Python310\lib\site-packages\click\core.py", line 1128, in __call__ return self.main(*args, **kwargs) File "\Python310\lib\site-packages\click\core.py", line 1053, in main rv = self.invoke(ctx) File "\Python310\lib\site-packages\click\core.py", line 1659, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "\Python310\lib\site-packages\click\core.py", line 1659, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "\Python310\lib\site-packages\click\core.py", line 1395, in invoke return ctx.invoke(self.callback, **ctx.params) File "\Python310\lib\site-packages\click\core.py", line 754, in invoke return __callback(*args, **kwargs) File "\Python310\lib\site-packages\click\decorators.py", line 26, in new_func return f(get_current_context(), *args, **kwargs) File "\tecpg\cli.py", line 270, in mlr_full output = regression_full(*args, **logger) File "\tecpg\regression_full_pre_filtration.py", line 162, in regression_full E = (Y.unsqueeze(1) - X.bmm(B.unsqueeze(2))).squeeze(2) RuntimeError: CUDA out of memory. Tried to allocate 12.00 MiB (GPU 0; 8.00 GiB total capacity; 7.24 GiB already allocated; 0 bytes free; 7.29 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF ```
tecpg run mlr-full --cis ```python [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.0022 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.4233 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 2.0747 seconds [INFOTIMER] Finished reading 3 dataframes in 2.5003 seconds. [INFO] No region window provided. Resorting to default. [INFO] Using default window for cis of 1000000 bases [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 296 degrees of freedom [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0067 seconds [INFOTIMER] Converted G to tensor in 0.0094 seconds [INFOTIMER] Created ones in 0.0001 seconds [INFOTIMER] Created X in 0.0003 seconds [INFOTIMER] Transposed X in 0.0 seconds [INFOTIMER] Calculated XtXi in 0.6202 seconds [INFOTIMER] Calculated XtXi_diag in 0.0001 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0002 seconds [INFOTIMER] Calculated X constants in 0.6371 seconds [INFO] CUDA device memory: 97.112064 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOTIMER] Looped over methylation loci in 40.792 seconds [INFOTIMER] Calculated regression_full in 41.4291 seconds [INFO] Generating dataframe from results... [INFOTIMER] Finished creating indices in 2.5759 seconds [INFOTIMER] Finished creating preliminary dataframe in 0.0964 seconds [INFOTIMER] Created output dataframe in 2.6723 total seconds [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Saving 1 dataframes... [INFOTIMER] Saving 1/1: out.csv [INFOTIMER] Saved 1/1 in 1.2304 seconds [INFOTIMER] Finished saving 1 dataframes in 1.2305 seconds. ```
tecpg run mlr-full --distal ```python [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.0019 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.455 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 2.1997 seconds [INFOTIMER] Finished reading 3 dataframes in 2.6567 seconds. [INFO] No region window provided. Resorting to default. [INFO] Using default window for distal of 50000 bases [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 296 degrees of freedom [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0088 seconds [INFOTIMER] Converted G to tensor in 0.0144 seconds [INFOTIMER] Created ones in 0.0002 seconds [INFOTIMER] Created X in 0.0004 seconds [INFOTIMER] Transposed X in 0.0001 seconds [INFOTIMER] Calculated XtXi in 0.7754 seconds [INFOTIMER] Calculated XtXi_diag in 0.0001 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0002 seconds [INFOTIMER] Calculated X constants in 0.7996 seconds [INFO] CUDA device memory: 97.112064 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOTIMER] Looped over methylation loci in 42.818 seconds [INFOTIMER] Calculated regression_full in 43.6176 seconds [INFO] Generating dataframe from results... [INFOTIMER] Finished creating indices in 3.3126 seconds [INFOTIMER] Finished creating preliminary dataframe in 1.9613 seconds [INFOTIMER] Created output dataframe in 5.2739 total seconds [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Saving 1 dataframes... [INFOTIMER] Saving 1/1: out.csv [INFOTIMER] Saved 1/1 in 108.7834 seconds [INFOTIMER] Finished saving 1 dataframes in 108.7835 seconds. ```
tecpg run mlr-full --trans ```python [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.0032 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.6677 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 3.0834 seconds [INFOTIMER] Finished reading 3 dataframes in 3.7545 seconds. [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 296 degrees of freedom [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0103 seconds [INFOTIMER] Converted G to tensor in 0.0144 seconds [INFOTIMER] Created ones in 0.0002 seconds [INFOTIMER] Created X in 0.0004 seconds [INFOTIMER] Transposed X in 0.0001 seconds [INFOTIMER] Calculated XtXi in 1.0892 seconds [INFOTIMER] Calculated XtXi_diag in 0.0002 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0004 seconds [INFOTIMER] Calculated X constants in 1.1151 seconds [INFO] CUDA device memory: 97.112064 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... Traceback (most recent call last): File "\Python310\Scripts\tecpg-script.py", line 33, in sys.exit(load_entry_point('tecpg', 'console_scripts', 'tecpg')()) File "\Python310\Scripts\tecpg-script.py", line 25, in importlib_load_entry_point return next(matches).load() File "\Python310\lib\importlib\metadata\__init__.py", line 162, in load module = import_module(match.group('module')) File "\Python310\lib\importlib\__init__.py", line 126, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "", line 1050, in _gcd_import File "", line 1027, in _find_and_load File "", line 1006, in _find_and_load_unlocked File "", line 688, in _load_unlocked File "", line 883, in exec_module File "", line 241, in _call_with_frames_removed File "\tecpg\__main__.py", line 9, in main() File "\tecpg\__main__.py", line 6, in main start() File "\tecpg\cli.py", line 533, in start cli(obj={}) File "\Python310\lib\site-packages\click\core.py", line 1128, in __call__ return self.main(*args, **kwargs) File "\Python310\lib\site-packages\click\core.py", line 1053, in main rv = self.invoke(ctx) File "\Python310\lib\site-packages\click\core.py", line 1659, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "\Python310\lib\site-packages\click\core.py", line 1659, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "\Python310\lib\site-packages\click\core.py", line 1395, in invoke return ctx.invoke(self.callback, **ctx.params) File "\Python310\lib\site-packages\click\core.py", line 754, in invoke return __callback(*args, **kwargs) File "\Python310\lib\site-packages\click\decorators.py", line 26, in new_func return f(get_current_context(), *args, **kwargs) File "\tecpg\cli.py", line 270, in mlr_full output = regression_full(*args, **logger) File "\tecpg\regression_full_pre_filtration.py", line 192, in regression_full B = XtXi_Xt[region_indices].matmul(Y) RuntimeError: CUDA out of memory. Tried to allocate 46.00 MiB (GPU 0; 8.00 GiB total capacity; 7.11 GiB already allocated; 0 bytes free; 7.26 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF ```
tecpg run mlr-full -p 0.05 ```python [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.0018 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.4396 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 2.0748 seconds [INFOTIMER] Finished reading 3 dataframes in 2.5163 seconds. [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 296 degrees of freedom [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0004 seconds [INFOTIMER] Converted G to tensor in 0.0125 seconds [INFOTIMER] Created ones in 0.0001 seconds [INFOTIMER] Created X in 0.0004 seconds [INFOTIMER] Transposed X in 0.0 seconds [INFOTIMER] Calculated XtXi in 0.6418 seconds [INFOTIMER] Calculated XtXi_diag in 0.0001 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0002 seconds [INFOTIMER] Calculated X constants in 0.6556 seconds [INFO] CUDA device memory: 96.630784 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOTIMER] Looped over methylation loci in 96.0436 seconds [INFOTIMER] Calculated regression_full in 96.6992 seconds [INFO] Generating dataframe from results... [INFOTIMER] Finished creating indices in 3.0981 seconds [INFOTIMER] Finished creating preliminary dataframe in 1.8539 seconds [INFOTIMER] Created output dataframe in 4.952 total seconds [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Saving 1 dataframes... [INFOTIMER] Saving 1/1: out.csv [INFOTIMER] Saved 1/1 in 102.6513 seconds [INFOTIMER] Finished saving 1 dataframes in 102.6514 seconds. ```
tecpg run mlr-full -p 0.05 --cis ```python [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.0027 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.6713 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 2.7697 seconds [INFOTIMER] Finished reading 3 dataframes in 3.4439 seconds. [INFO] No region window provided. Resorting to default. [INFO] Using default window for cis of 1000000 bases [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 296 degrees of freedom [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0081 seconds [INFOTIMER] Converted G to tensor in 0.012 seconds [INFOTIMER] Created ones in 0.0002 seconds [INFOTIMER] Created X in 0.0004 seconds [INFOTIMER] Transposed X in 0.0 seconds [INFOTIMER] Calculated XtXi in 0.74 seconds [INFOTIMER] Calculated XtXi_diag in 0.0001 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0003 seconds [INFOTIMER] Calculated X constants in 0.7611 seconds [INFO] CUDA device memory: 97.112064 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOTIMER] Looped over methylation loci in 49.739 seconds [INFOTIMER] Calculated regression_full in 50.5002 seconds [INFO] Generating dataframe from results... [INFOTIMER] Finished creating indices in 2.5604 seconds [INFOTIMER] Finished creating preliminary dataframe in 0.0468 seconds [INFOTIMER] Created output dataframe in 2.6072 total seconds [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Saving 1 dataframes... [INFOTIMER] Saving 1/1: out.csv [INFOTIMER] Saved 1/1 in 0.0475 seconds [INFOTIMER] Finished saving 1 dataframes in 0.0476 seconds. ```
tecpg run mlr-full -p 0.05 --distal ```python [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.0019 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.4272 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 2.0767 seconds [INFOTIMER] Finished reading 3 dataframes in 2.5059 seconds. [INFO] No region window provided. Resorting to default. [INFO] Using default window for distal of 50000 bases [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 296 degrees of freedom [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0063 seconds [INFOTIMER] Converted G to tensor in 0.0119 seconds [INFOTIMER] Created ones in 0.0001 seconds [INFOTIMER] Created X in 0.0003 seconds [INFOTIMER] Transposed X in 0.0 seconds [INFOTIMER] Calculated XtXi in 0.6204 seconds [INFOTIMER] Calculated XtXi_diag in 0.0001 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0002 seconds [INFOTIMER] Calculated X constants in 0.6394 seconds [INFO] CUDA device memory: 97.112064 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOTIMER] Looped over methylation loci in 44.5296 seconds [INFOTIMER] Calculated regression_full in 45.169 seconds [INFO] Generating dataframe from results... [INFOTIMER] Finished creating indices in 3.1823 seconds [INFOTIMER] Finished creating preliminary dataframe in 0.1528 seconds [INFOTIMER] Created output dataframe in 3.3351 total seconds [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Saving 1 dataframes... [INFOTIMER] Saving 1/1: out.csv [INFOTIMER] Saved 1/1 in 3.9931 seconds [INFOTIMER] Finished saving 1 dataframes in 3.9931 seconds. ```
tecpg run mlr-full -p 0.05 --trans ```python [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.0019 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.4702 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 2.2532 seconds [INFOTIMER] Finished reading 3 dataframes in 2.7254 seconds. [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 296 degrees of freedom [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0068 seconds [INFOTIMER] Converted G to tensor in 0.012 seconds [INFOTIMER] Created ones in 0.0001 seconds [INFOTIMER] Created X in 0.0003 seconds [INFOTIMER] Transposed X in 0.0 seconds [INFOTIMER] Calculated XtXi in 0.6795 seconds [INFOTIMER] Calculated XtXi_diag in 0.0001 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0003 seconds [INFOTIMER] Calculated X constants in 0.6992 seconds [INFO] CUDA device memory: 97.112064 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOTIMER] Looped over methylation loci in 132.5641 seconds [INFOTIMER] Calculated regression_full in 133.2633 seconds [INFO] Generating dataframe from results... [INFOTIMER] Finished creating indices in 5.5753 seconds [INFOTIMER] Finished creating preliminary dataframe in 1.7423 seconds [INFOTIMER] Created output dataframe in 7.3176 total seconds [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Saving 1 dataframes... [INFOTIMER] Saving 1/1: out.csv [INFOTIMER] Saved 1/1 in 94.6957 seconds [INFOTIMER] Finished saving 1 dataframes in 94.6957 seconds. ```
tecpg run mlr-full -l 2000 ```python [INFOCOUNT] Done saving part 2 [INFOCOUNT] Done saving part 10 [INFOCOUNT] Done saving part 18 [INFOCOUNT] Done saving part 3 [INFOCOUNT] Done saving part 11 [INFOCOUNT] Done saving part 19 [INFOCOUNT] Done saving part 4 [INFOCOUNT] Done saving part 12 [INFOCOUNT] Done saving part 20 [INFOCOUNT] Done saving part 5 [INFOCOUNT] Done saving part 13 [INFOCOUNT] Done saving part 21 [INFOCOUNT] Done saving part 6 [INFOCOUNT] Done saving part 14 [INFOCOUNT] Done saving part 22 [INFOCOUNT] Done saving part 7 [INFOCOUNT] Done saving part 15 [INFOCOUNT] Done saving part 23 [INFOCOUNT] Done saving part 8 [INFOCOUNT] Done saving part 16 [INFOCOUNT] Done saving part 24 [INFOCOUNT] Done saving part 1 [INFOCOUNT] Done saving part 9 [INFOCOUNT] Done saving part 17 [INFOCOUNT] Done saving part 25 [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.0026 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.5683 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 3.4865 seconds [INFOTIMER] Finished reading 3 dataframes in 4.0574 seconds. [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 296 degrees of freedom [INFO] Initializing output directory [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0008 seconds [INFOTIMER] Converted G to tensor in 0.0154 seconds [INFOTIMER] Created ones in 0.0002 seconds [INFOTIMER] Created X in 0.0004 seconds [INFOTIMER] Transposed X in 0.0 seconds [INFOTIMER] Calculated XtXi in 0.8974 seconds [INFOTIMER] Calculated XtXi_diag in 0.0001 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0003 seconds [INFOTIMER] Calculated X constants in 0.9147 seconds [INFO] CUDA device memory: 96.630784 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOCOUNT] Saving part 1/25: [INFOTIMER] Completed chunk 1/50000 in 6.106 seconds. Average chunk time: 6.106 seconds [INFOCOUNT] Saving part 2/25: [INFOTIMER] Completed chunk 2/50000 in 5.9742 seconds. Average chunk time: 6.0401 seconds [INFOCOUNT] Saving part 3/25: [INFOTIMER] Completed chunk 3/50000 in 5.8861 seconds. Average chunk time: 5.9887 seconds [INFOCOUNT] Saving part 4/25: [INFOTIMER] Completed chunk 4/50000 in 6.0758 seconds. Average chunk time: 6.0105 seconds [INFOCOUNT] Saving part 5/25: [INFOTIMER] Completed chunk 5/50000 in 6.3773 seconds. Average chunk time: 6.0839 seconds [INFOCOUNT] Saving part 6/25: [INFOTIMER] Completed chunk 6/50000 in 6.8714 seconds. Average chunk time: 6.2151 seconds [INFOCOUNT] Saving part 7/25: [INFOTIMER] Completed chunk 7/50000 in 7.0202 seconds. Average chunk time: 6.3301 seconds [INFOCOUNT] Saving part 8/25: [INFOTIMER] Completed chunk 8/50000 in 7.4154 seconds. Average chunk time: 6.4658 seconds [INFOCOUNT] Saving part 9/25: [INFOTIMER] Completed chunk 9/50000 in 7.7393 seconds. Average chunk time: 6.6073 seconds [INFOCOUNT] Saving part 10/25: [INFOTIMER] Completed chunk 10/50000 in 8.1196 seconds. Average chunk time: 6.7585 seconds [INFOCOUNT] Saving part 11/25: [INFOTIMER] Completed chunk 11/50000 in 6.9108 seconds. Average chunk time: 6.7723 seconds [INFOCOUNT] Saving part 12/25: [INFOTIMER] Completed chunk 12/50000 in 6.9777 seconds. Average chunk time: 6.7895 seconds [INFOCOUNT] Saving part 13/25: [INFOTIMER] Completed chunk 13/50000 in 6.8766 seconds. Average chunk time: 6.7962 seconds [INFOCOUNT] Saving part 14/25: [INFOTIMER] Completed chunk 14/50000 in 6.7312 seconds. Average chunk time: 6.7915 seconds [INFOCOUNT] Saving part 15/25: [INFOTIMER] Completed chunk 15/50000 in 7.358 seconds. Average chunk time: 6.8293 seconds [INFOCOUNT] Saving part 16/25: [INFOTIMER] Completed chunk 16/50000 in 6.8821 seconds. Average chunk time: 6.8326 seconds [INFOCOUNT] Saving part 17/25: [INFOTIMER] Completed chunk 17/50000 in 7.0373 seconds. Average chunk time: 6.8446 seconds [INFOCOUNT] Saving part 18/25: [INFOTIMER] Completed chunk 18/50000 in 6.7574 seconds. Average chunk time: 6.8398 seconds [INFOCOUNT] Saving part 19/25: [INFOTIMER] Completed chunk 19/50000 in 6.5808 seconds. Average chunk time: 6.8262 seconds [INFOCOUNT] Saving part 20/25: [INFOTIMER] Completed chunk 20/50000 in 7.6851 seconds. Average chunk time: 6.8691 seconds [INFOCOUNT] Saving part 21/25: [INFOTIMER] Completed chunk 21/50000 in 6.9533 seconds. Average chunk time: 6.8731 seconds [INFOCOUNT] Saving part 22/25: [INFOTIMER] Completed chunk 22/50000 in 6.444 seconds. Average chunk time: 6.8536 seconds [INFOCOUNT] Saving part 23/25: [INFOTIMER] Completed chunk 23/50000 in 7.4511 seconds. Average chunk time: 6.8796 seconds [INFOCOUNT] Saving part 24/25: [INFOTIMER] Completed chunk 24/50000 in 7.0741 seconds. Average chunk time: 6.8877 seconds [INFOCOUNT] Saving part 25/25: [INFOTIMER] Completed chunk 25/50000 in 7.085 seconds. Average chunk time: 6.8956 seconds [INFOTIMER] Looped over methylation loci in 172.3946 seconds [INFOTIMER] Calculated regression_full in 173.3093 seconds [INFOTIMER] Waiting for chunks to save... [INFOTIMER] Finished waiting for chunks to save in 329.6891 seconds ```
tecpg run mlr-full -l 2000 --cis ```python [INFOCOUNT] Done saving part 2 [INFOCOUNT] Done saving part 10 [INFOCOUNT] Done saving part 18 [INFOCOUNT] Done saving part 3 [INFOCOUNT] Done saving part 11 [INFOCOUNT] Done saving part 19 [INFOCOUNT] Done saving part 4 [INFOCOUNT] Done saving part 12 [INFOCOUNT] Done saving part 20 [INFOCOUNT] Done saving part 5 [INFOCOUNT] Done saving part 13 [INFOCOUNT] Done saving part 21 [INFOCOUNT] Done saving part 6 [INFOCOUNT] Done saving part 14 [INFOCOUNT] Done saving part 22 [INFOCOUNT] Done saving part 7 [INFOCOUNT] Done saving part 15 [INFOCOUNT] Done saving part 23 [INFOCOUNT] Done saving part 8 [INFOCOUNT] Done saving part 16 [INFOCOUNT] Done saving part 24 [INFOCOUNT] Done saving part 1 [INFOCOUNT] Done saving part 9 [INFOCOUNT] Done saving part 17 [INFOCOUNT] Done saving part 25 [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.002 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.4153 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 2.2086 seconds [INFOTIMER] Finished reading 3 dataframes in 2.6261 seconds. [INFO] No region window provided. Resorting to default. [INFO] Using default window for cis of 1000000 bases [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 296 degrees of freedom [INFO] Initializing output directory [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0109 seconds [INFOTIMER] Converted G to tensor in 0.0109 seconds [INFOTIMER] Created ones in 0.0001 seconds [INFOTIMER] Created X in 0.0003 seconds [INFOTIMER] Transposed X in 0.0 seconds [INFOTIMER] Calculated XtXi in 0.6849 seconds [INFOTIMER] Calculated XtXi_diag in 0.0001 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0003 seconds [INFOTIMER] Calculated X constants in 0.7075 seconds [INFO] CUDA device memory: 97.112064 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOCOUNT] Saving part 1/25: [INFOTIMER] Completed chunk 1/50000 in 1.943 seconds. Average chunk time: 1.943 seconds [INFOCOUNT] Saving part 2/25: [INFOTIMER] Completed chunk 2/50000 in 2.1235 seconds. Average chunk time: 2.0333 seconds [INFOCOUNT] Saving part 3/25: [INFOTIMER] Completed chunk 3/50000 in 1.9675 seconds. Average chunk time: 2.0113 seconds [INFOCOUNT] Saving part 4/25: [INFOTIMER] Completed chunk 4/50000 in 2.0153 seconds. Average chunk time: 2.0123 seconds [INFOCOUNT] Saving part 5/25: [INFOTIMER] Completed chunk 5/50000 in 1.953 seconds. Average chunk time: 2.0005 seconds [INFOCOUNT] Saving part 6/25: [INFOTIMER] Completed chunk 6/50000 in 2.1137 seconds. Average chunk time: 2.0193 seconds [INFOCOUNT] Saving part 7/25: [INFOTIMER] Completed chunk 7/50000 in 1.9211 seconds. Average chunk time: 2.0053 seconds [INFOCOUNT] Saving part 8/25: [INFOTIMER] Completed chunk 8/50000 in 2.0446 seconds. Average chunk time: 2.0102 seconds [INFOCOUNT] Saving part 9/25: [INFOTIMER] Completed chunk 9/50000 in 2.1264 seconds. Average chunk time: 2.0231 seconds [INFOCOUNT] Saving part 10/25: [INFOTIMER] Completed chunk 10/50000 in 2.0149 seconds. Average chunk time: 2.0223 seconds [INFOCOUNT] Saving part 11/25: [INFOTIMER] Completed chunk 11/50000 in 1.8929 seconds. Average chunk time: 2.0105 seconds [INFOCOUNT] Saving part 12/25: [INFOTIMER] Completed chunk 12/50000 in 1.9168 seconds. Average chunk time: 2.0027 seconds [INFOCOUNT] Saving part 13/25: [INFOTIMER] Completed chunk 13/50000 in 1.9336 seconds. Average chunk time: 1.9974 seconds [INFOCOUNT] Saving part 14/25: [INFOTIMER] Completed chunk 14/50000 in 1.9295 seconds. Average chunk time: 1.9926 seconds [INFOCOUNT] Saving part 15/25: [INFOTIMER] Completed chunk 15/50000 in 1.9663 seconds. Average chunk time: 1.9908 seconds [INFOCOUNT] Saving part 16/25: [INFOTIMER] Completed chunk 16/50000 in 1.9138 seconds. Average chunk time: 1.986 seconds [INFOCOUNT] Saving part 17/25: [INFOTIMER] Completed chunk 17/50000 in 1.9838 seconds. Average chunk time: 1.9859 seconds [INFOCOUNT] Saving part 18/25: [INFOTIMER] Completed chunk 18/50000 in 1.8591 seconds. Average chunk time: 1.9788 seconds [INFOCOUNT] Saving part 19/25: [INFOTIMER] Completed chunk 19/50000 in 1.8863 seconds. Average chunk time: 1.9739 seconds [INFOCOUNT] Saving part 20/25: [INFOTIMER] Completed chunk 20/50000 in 1.8658 seconds. Average chunk time: 1.9685 seconds [INFOCOUNT] Saving part 21/25: [INFOTIMER] Completed chunk 21/50000 in 1.8258 seconds. Average chunk time: 1.9617 seconds [INFOCOUNT] Saving part 22/25: [INFOTIMER] Completed chunk 22/50000 in 1.8081 seconds. Average chunk time: 1.9548 seconds [INFOCOUNT] Saving part 23/25: [INFOTIMER] Completed chunk 23/50000 in 1.8192 seconds. Average chunk time: 1.9489 seconds [INFOCOUNT] Saving part 24/25: [INFOTIMER] Completed chunk 24/50000 in 1.8516 seconds. Average chunk time: 1.9448 seconds [INFOCOUNT] Saving part 25/25: [INFOTIMER] Completed chunk 25/50000 in 1.8173 seconds. Average chunk time: 1.9397 seconds [INFOTIMER] Looped over methylation loci in 48.4986 seconds [INFOTIMER] Calculated regression_full in 49.2062 seconds [INFOTIMER] Waiting for chunks to save... [INFOTIMER] Finished waiting for chunks to save in 0.3382 seconds ```
tecpg run mlr-full -l 2000 --distal ```python [INFOCOUNT] Done saving part 2 [INFOCOUNT] Done saving part 10 [INFOCOUNT] Done saving part 18 [INFOCOUNT] Done saving part 3 [INFOCOUNT] Done saving part 11 [INFOCOUNT] Done saving part 19 [INFOCOUNT] Done saving part 4 [INFOCOUNT] Done saving part 12 [INFOCOUNT] Done saving part 20 [INFOCOUNT] Done saving part 5 [INFOCOUNT] Done saving part 13 [INFOCOUNT] Done saving part 21 [INFOCOUNT] Done saving part 6 [INFOCOUNT] Done saving part 14 [INFOCOUNT] Done saving part 22 [INFOCOUNT] Done saving part 7 [INFOCOUNT] Done saving part 15 [INFOCOUNT] Done saving part 23 [INFOCOUNT] Done saving part 8 [INFOCOUNT] Done saving part 16 [INFOCOUNT] Done saving part 24 [INFOCOUNT] Done saving part 1 [INFOCOUNT] Done saving part 9 [INFOCOUNT] Done saving part 17 [INFOCOUNT] Done saving part 25 [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.002 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.4471 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 2.1831 seconds [INFOTIMER] Finished reading 3 dataframes in 2.6324 seconds. [INFO] No region window provided. Resorting to default. [INFO] Using default window for distal of 50000 bases [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 296 degrees of freedom [INFO] Initializing output directory [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0065 seconds [INFOTIMER] Converted G to tensor in 0.0112 seconds [INFOTIMER] Created ones in 0.0001 seconds [INFOTIMER] Created X in 0.0003 seconds [INFOTIMER] Transposed X in 0.0 seconds [INFOTIMER] Calculated XtXi in 0.6286 seconds [INFOTIMER] Calculated XtXi_diag in 0.0001 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0002 seconds [INFOTIMER] Calculated X constants in 0.6471 seconds [INFO] CUDA device memory: 97.112064 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOCOUNT] Saving part 1/25: [INFOTIMER] Completed chunk 1/50000 in 1.9996 seconds. Average chunk time: 1.9996 seconds [INFOCOUNT] Saving part 2/25: [INFOTIMER] Completed chunk 2/50000 in 2.0236 seconds. Average chunk time: 2.0116 seconds [INFOCOUNT] Saving part 3/25: [INFOTIMER] Completed chunk 3/50000 in 2.1836 seconds. Average chunk time: 2.0689 seconds [INFOCOUNT] Saving part 4/25: [INFOTIMER] Completed chunk 4/50000 in 2.314 seconds. Average chunk time: 2.1302 seconds [INFOCOUNT] Saving part 5/25: [INFOTIMER] Completed chunk 5/50000 in 2.4524 seconds. Average chunk time: 2.1946 seconds [INFOCOUNT] Saving part 6/25: [INFOTIMER] Completed chunk 6/50000 in 2.4352 seconds. Average chunk time: 2.2347 seconds [INFOCOUNT] Saving part 7/25: [INFOTIMER] Completed chunk 7/50000 in 2.4618 seconds. Average chunk time: 2.2672 seconds [INFOCOUNT] Saving part 8/25: [INFOTIMER] Completed chunk 8/50000 in 2.4534 seconds. Average chunk time: 2.2904 seconds [INFOCOUNT] Saving part 9/25: [INFOTIMER] Completed chunk 9/50000 in 2.502 seconds. Average chunk time: 2.3139 seconds [INFOCOUNT] Saving part 10/25: [INFOTIMER] Completed chunk 10/50000 in 2.3939 seconds. Average chunk time: 2.3219 seconds [INFOCOUNT] Saving part 11/25: [INFOTIMER] Completed chunk 11/50000 in 2.5834 seconds. Average chunk time: 2.3457 seconds [INFOCOUNT] Saving part 12/25: [INFOTIMER] Completed chunk 12/50000 in 2.4752 seconds. Average chunk time: 2.3565 seconds [INFOCOUNT] Saving part 13/25: [INFOTIMER] Completed chunk 13/50000 in 2.29 seconds. Average chunk time: 2.3514 seconds [INFOCOUNT] Saving part 14/25: [INFOTIMER] Completed chunk 14/50000 in 2.2634 seconds. Average chunk time: 2.3451 seconds [INFOCOUNT] Saving part 15/25: [INFOTIMER] Completed chunk 15/50000 in 2.2847 seconds. Average chunk time: 2.3411 seconds [INFOCOUNT] Saving part 16/25: [INFOTIMER] Completed chunk 16/50000 in 2.2445 seconds. Average chunk time: 2.335 seconds [INFOCOUNT] Saving part 17/25: [INFOTIMER] Completed chunk 17/50000 in 2.4267 seconds. Average chunk time: 2.3404 seconds [INFOCOUNT] Saving part 18/25: [INFOTIMER] Completed chunk 18/50000 in 2.4364 seconds. Average chunk time: 2.3458 seconds [INFOCOUNT] Saving part 19/25: [INFOTIMER] Completed chunk 19/50000 in 2.3153 seconds. Average chunk time: 2.3442 seconds [INFOCOUNT] Saving part 20/25: [INFOTIMER] Completed chunk 20/50000 in 2.2771 seconds. Average chunk time: 2.3408 seconds [INFOCOUNT] Saving part 21/25: [INFOTIMER] Completed chunk 21/50000 in 2.2884 seconds. Average chunk time: 2.3383 seconds [INFOCOUNT] Saving part 22/25: [INFOTIMER] Completed chunk 22/50000 in 2.2763 seconds. Average chunk time: 2.3355 seconds [INFOCOUNT] Saving part 23/25: [INFOTIMER] Completed chunk 23/50000 in 2.4931 seconds. Average chunk time: 2.3424 seconds [INFOCOUNT] Saving part 24/25: [INFOTIMER] Completed chunk 24/50000 in 2.4454 seconds. Average chunk time: 2.3466 seconds [INFOCOUNT] Saving part 25/25: [INFOTIMER] Completed chunk 25/50000 in 2.3147 seconds. Average chunk time: 2.3454 seconds [INFOTIMER] Looped over methylation loci in 58.6429 seconds [INFOTIMER] Calculated regression_full in 59.2901 seconds [INFOTIMER] Waiting for chunks to save... [INFOTIMER] Finished waiting for chunks to save in 3.6264 seconds ```
tecpg run mlr-full -l 2000 --trans ```python [INFOCOUNT] Done saving part 2 [INFOCOUNT] Done saving part 10 [INFOCOUNT] Done saving part 18 [INFOCOUNT] Done saving part 3 [INFOCOUNT] Done saving part 11 [INFOCOUNT] Done saving part 19 [INFOCOUNT] Done saving part 4 [INFOCOUNT] Done saving part 12 [INFOCOUNT] Done saving part 20 [INFOCOUNT] Done saving part 5 [INFOCOUNT] Done saving part 13 [INFOCOUNT] Done saving part 21 [INFOCOUNT] Done saving part 6 [INFOCOUNT] Done saving part 14 [INFOCOUNT] Done saving part 22 [INFOCOUNT] Done saving part 7 [INFOCOUNT] Done saving part 15 [INFOCOUNT] Done saving part 23 [INFOCOUNT] Done saving part 8 [INFOCOUNT] Done saving part 16 [INFOCOUNT] Done saving part 24 [INFOCOUNT] Done saving part 1 [INFOCOUNT] Done saving part 9 [INFOCOUNT] Done saving part 17 [INFOCOUNT] Done saving part 25 [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.002 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.4489 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 2.4462 seconds [INFOTIMER] Finished reading 3 dataframes in 2.8973 seconds. [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 296 degrees of freedom [INFO] Initializing output directory [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0108 seconds [INFOTIMER] Converted G to tensor in 0.0119 seconds [INFOTIMER] Created ones in 0.0001 seconds [INFOTIMER] Created X in 0.0004 seconds [INFOTIMER] Transposed X in 0.0 seconds [INFOTIMER] Calculated XtXi in 0.6605 seconds [INFOTIMER] Calculated XtXi_diag in 0.0001 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0002 seconds [INFOTIMER] Calculated X constants in 0.684 seconds [INFO] CUDA device memory: 97.112064 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOCOUNT] Saving part 1/25: [INFOTIMER] Completed chunk 1/50000 in 7.2635 seconds. Average chunk time: 7.2635 seconds [INFOCOUNT] Saving part 2/25: [INFOTIMER] Completed chunk 2/50000 in 7.7676 seconds. Average chunk time: 7.5155 seconds [INFOCOUNT] Saving part 3/25: [INFOTIMER] Completed chunk 3/50000 in 8.073 seconds. Average chunk time: 7.7014 seconds [INFOCOUNT] Saving part 4/25: [INFOTIMER] Completed chunk 4/50000 in 8.2605 seconds. Average chunk time: 7.8411 seconds [INFOCOUNT] Saving part 5/25: [INFOTIMER] Completed chunk 5/50000 in 8.5518 seconds. Average chunk time: 7.9833 seconds [INFOCOUNT] Saving part 6/25: [INFOTIMER] Completed chunk 6/50000 in 8.8975 seconds. Average chunk time: 8.1356 seconds [INFOCOUNT] Saving part 7/25: [INFOTIMER] Completed chunk 7/50000 in 9.1796 seconds. Average chunk time: 8.2848 seconds [INFOCOUNT] Saving part 8/25: [INFOTIMER] Completed chunk 8/50000 in 9.7433 seconds. Average chunk time: 8.4671 seconds [INFOCOUNT] Saving part 9/25: [INFOTIMER] Completed chunk 9/50000 in 10.1813 seconds. Average chunk time: 8.6575 seconds [INFOCOUNT] Saving part 10/25: [INFOTIMER] Completed chunk 10/50000 in 9.8229 seconds. Average chunk time: 8.7741 seconds [INFOCOUNT] Saving part 11/25: [INFOTIMER] Completed chunk 11/50000 in 9.6587 seconds. Average chunk time: 8.8545 seconds [INFOCOUNT] Saving part 12/25: [INFOTIMER] Completed chunk 12/50000 in 9.2803 seconds. Average chunk time: 8.89 seconds [INFOCOUNT] Saving part 13/25: [INFOTIMER] Completed chunk 13/50000 in 9.3697 seconds. Average chunk time: 8.9269 seconds [INFOCOUNT] Saving part 14/25: [INFOTIMER] Completed chunk 14/50000 in 9.5714 seconds. Average chunk time: 8.9729 seconds [INFOCOUNT] Saving part 15/25: [INFOTIMER] Completed chunk 15/50000 in 9.3783 seconds. Average chunk time: 8.9999 seconds [INFOCOUNT] Saving part 16/25: [INFOTIMER] Completed chunk 16/50000 in 10.0043 seconds. Average chunk time: 9.0627 seconds [INFOCOUNT] Saving part 17/25: [INFOTIMER] Completed chunk 17/50000 in 9.531 seconds. Average chunk time: 9.0903 seconds [INFOCOUNT] Saving part 18/25: [INFOTIMER] Completed chunk 18/50000 in 9.9382 seconds. Average chunk time: 9.1374 seconds [INFOCOUNT] Saving part 19/25: [INFOTIMER] Completed chunk 19/50000 in 10.028 seconds. Average chunk time: 9.1843 seconds [INFOCOUNT] Saving part 20/25: [INFOTIMER] Completed chunk 20/50000 in 9.8453 seconds. Average chunk time: 9.2173 seconds [INFOCOUNT] Saving part 21/25: [INFOTIMER] Completed chunk 21/50000 in 9.3107 seconds. Average chunk time: 9.2218 seconds [INFOCOUNT] Saving part 22/25: [INFOTIMER] Completed chunk 22/50000 in 9.8294 seconds. Average chunk time: 9.2494 seconds [INFOCOUNT] Saving part 23/25: [INFOTIMER] Completed chunk 23/50000 in 10.2024 seconds. Average chunk time: 9.2908 seconds [INFOCOUNT] Saving part 24/25: [INFOTIMER] Completed chunk 24/50000 in 10.1936 seconds. Average chunk time: 9.3284 seconds [INFOCOUNT] Saving part 25/25: [INFOTIMER] Completed chunk 25/50000 in 10.5823 seconds. Average chunk time: 9.3786 seconds [INFOTIMER] Looped over methylation loci in 234.4737 seconds [INFOTIMER] Calculated regression_full in 235.1578 seconds [INFOTIMER] Waiting for chunks to save... [INFOTIMER] Finished waiting for chunks to save in 250.8231 seconds ```
tecpg run mlr-full -l 2000 -p 0.05 ```python [INFOCOUNT] Done saving part 2 [INFOCOUNT] Done saving part 10 [INFOCOUNT] Done saving part 18 [INFOCOUNT] Done saving part 3 [INFOCOUNT] Done saving part 11 [INFOCOUNT] Done saving part 19 [INFOCOUNT] Done saving part 4 [INFOCOUNT] Done saving part 12 [INFOCOUNT] Done saving part 20 [INFOCOUNT] Done saving part 5 [INFOCOUNT] Done saving part 13 [INFOCOUNT] Done saving part 21 [INFOCOUNT] Done saving part 6 [INFOCOUNT] Done saving part 14 [INFOCOUNT] Done saving part 22 [INFOCOUNT] Done saving part 7 [INFOCOUNT] Done saving part 15 [INFOCOUNT] Done saving part 23 [INFOCOUNT] Done saving part 8 [INFOCOUNT] Done saving part 16 [INFOCOUNT] Done saving part 24 [INFOCOUNT] Done saving part 1 [INFOCOUNT] Done saving part 9 [INFOCOUNT] Done saving part 17 [INFOCOUNT] Done saving part 25 [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.0019 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.4201 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 2.265 seconds [INFOTIMER] Finished reading 3 dataframes in 2.6871 seconds. [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 296 degrees of freedom [INFO] Initializing output directory [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0007 seconds [INFOTIMER] Converted G to tensor in 0.0104 seconds [INFOTIMER] Created ones in 0.0001 seconds [INFOTIMER] Created X in 0.0003 seconds [INFOTIMER] Transposed X in 0.0 seconds [INFOTIMER] Calculated XtXi in 0.6347 seconds [INFOTIMER] Calculated XtXi_diag in 0.0001 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0002 seconds [INFOTIMER] Calculated X constants in 0.6466 seconds [INFO] CUDA device memory: 96.630784 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOCOUNT] Saving part 1/25: [INFOTIMER] Completed chunk 1/50000 in 3.9761 seconds. Average chunk time: 3.9761 seconds [INFOCOUNT] Saving part 2/25: [INFOTIMER] Completed chunk 2/50000 in 4.0063 seconds. Average chunk time: 3.9912 seconds [INFOCOUNT] Saving part 3/25: [INFOTIMER] Completed chunk 3/50000 in 4.025 seconds. Average chunk time: 4.0025 seconds [INFOCOUNT] Saving part 4/25: [INFOTIMER] Completed chunk 4/50000 in 4.0468 seconds. Average chunk time: 4.0135 seconds [INFOCOUNT] Saving part 5/25: [INFOTIMER] Completed chunk 5/50000 in 4.0523 seconds. Average chunk time: 4.0213 seconds [INFOCOUNT] Saving part 6/25: [INFOTIMER] Completed chunk 6/50000 in 4.0977 seconds. Average chunk time: 4.034 seconds [INFOCOUNT] Saving part 7/25: [INFOTIMER] Completed chunk 7/50000 in 4.0338 seconds. Average chunk time: 4.034 seconds [INFOCOUNT] Saving part 8/25: [INFOTIMER] Completed chunk 8/50000 in 4.0688 seconds. Average chunk time: 4.0384 seconds [INFOCOUNT] Saving part 9/25: [INFOTIMER] Completed chunk 9/50000 in 4.0896 seconds. Average chunk time: 4.0441 seconds [INFOCOUNT] Saving part 10/25: [INFOTIMER] Completed chunk 10/50000 in 4.0212 seconds. Average chunk time: 4.0418 seconds [INFOCOUNT] Saving part 11/25: [INFOTIMER] Completed chunk 11/50000 in 4.0376 seconds. Average chunk time: 4.0414 seconds [INFOCOUNT] Saving part 12/25: [INFOTIMER] Completed chunk 12/50000 in 4.0143 seconds. Average chunk time: 4.0391 seconds [INFOCOUNT] Saving part 13/25: [INFOTIMER] Completed chunk 13/50000 in 4.0466 seconds. Average chunk time: 4.0397 seconds [INFOCOUNT] Saving part 14/25: [INFOTIMER] Completed chunk 14/50000 in 4.02 seconds. Average chunk time: 4.0383 seconds [INFOCOUNT] Saving part 15/25: [INFOTIMER] Completed chunk 15/50000 in 4.0153 seconds. Average chunk time: 4.0368 seconds [INFOCOUNT] Saving part 16/25: [INFOTIMER] Completed chunk 16/50000 in 4.053 seconds. Average chunk time: 4.0378 seconds [INFOCOUNT] Saving part 17/25: [INFOTIMER] Completed chunk 17/50000 in 4.0399 seconds. Average chunk time: 4.0379 seconds [INFOCOUNT] Saving part 18/25: [INFOTIMER] Completed chunk 18/50000 in 3.9946 seconds. Average chunk time: 4.0355 seconds [INFOCOUNT] Saving part 19/25: [INFOTIMER] Completed chunk 19/50000 in 4.0437 seconds. Average chunk time: 4.0359 seconds [INFOCOUNT] Saving part 20/25: [INFOTIMER] Completed chunk 20/50000 in 4.0477 seconds. Average chunk time: 4.0365 seconds [INFOCOUNT] Saving part 21/25: [INFOTIMER] Completed chunk 21/50000 in 4.0568 seconds. Average chunk time: 4.0375 seconds [INFOCOUNT] Saving part 22/25: [INFOTIMER] Completed chunk 22/50000 in 4.076 seconds. Average chunk time: 4.0392 seconds [INFOCOUNT] Saving part 23/25: [INFOTIMER] Completed chunk 23/50000 in 4.0182 seconds. Average chunk time: 4.0383 seconds [INFOCOUNT] Saving part 24/25: [INFOTIMER] Completed chunk 24/50000 in 4.0339 seconds. Average chunk time: 4.0381 seconds [INFOCOUNT] Saving part 25/25: [INFOTIMER] Completed chunk 25/50000 in 4.0585 seconds. Average chunk time: 4.0389 seconds [INFOTIMER] Looped over methylation loci in 100.9796 seconds [INFOTIMER] Calculated regression_full in 101.6262 seconds [INFOTIMER] Waiting for chunks to save... [INFOTIMER] Finished waiting for chunks to save in 3.4593 seconds ```
tecpg run mlr-full -l 2000 -p 0.05 --cis ```python [INFOCOUNT] Done saving part 2 [INFOCOUNT] Done saving part 10 [INFOCOUNT] Done saving part 18 [INFOCOUNT] Done saving part 3 [INFOCOUNT] Done saving part 11 [INFOCOUNT] Done saving part 19 [INFOCOUNT] Done saving part 4 [INFOCOUNT] Done saving part 12 [INFOCOUNT] Done saving part 20 [INFOCOUNT] Done saving part 5 [INFOCOUNT] Done saving part 13 [INFOCOUNT] Done saving part 21 [INFOCOUNT] Done saving part 6 [INFOCOUNT] Done saving part 14 [INFOCOUNT] Done saving part 22 [INFOCOUNT] Done saving part 7 [INFOCOUNT] Done saving part 15 [INFOCOUNT] Done saving part 23 [INFOCOUNT] Done saving part 8 [INFOCOUNT] Done saving part 16 [INFOCOUNT] Done saving part 24 [INFOCOUNT] Done saving part 1 [INFOCOUNT] Done saving part 9 [INFOCOUNT] Done saving part 17 [INFOCOUNT] Done saving part 25 [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.0019 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.4175 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 2.3297 seconds [INFOTIMER] Finished reading 3 dataframes in 2.7491 seconds. [INFO] No region window provided. Resorting to default. [INFO] Using default window for cis of 1000000 bases [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 296 degrees of freedom [INFO] Initializing output directory [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0092 seconds [INFOTIMER] Converted G to tensor in 0.0115 seconds [INFOTIMER] Created ones in 0.0001 seconds [INFOTIMER] Created X in 0.0004 seconds [INFOTIMER] Transposed X in 0.0 seconds [INFOTIMER] Calculated XtXi in 0.5976 seconds [INFOTIMER] Calculated XtXi_diag in 0.0001 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0002 seconds [INFOTIMER] Calculated X constants in 0.6192 seconds [INFO] CUDA device memory: 97.112064 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOCOUNT] Saving part 1/25: [INFOTIMER] Completed chunk 1/50000 in 1.8543 seconds. Average chunk time: 1.8543 seconds [INFOCOUNT] Saving part 2/25: [INFOTIMER] Completed chunk 2/50000 in 1.7947 seconds. Average chunk time: 1.8245 seconds [INFOCOUNT] Saving part 3/25: [INFOTIMER] Completed chunk 3/50000 in 1.835 seconds. Average chunk time: 1.828 seconds [INFOCOUNT] Saving part 4/25: [INFOTIMER] Completed chunk 4/50000 in 1.9236 seconds. Average chunk time: 1.8519 seconds [INFOCOUNT] Saving part 5/25: [INFOTIMER] Completed chunk 5/50000 in 1.9113 seconds. Average chunk time: 1.8638 seconds [INFOCOUNT] Saving part 6/25: [INFOTIMER] Completed chunk 6/50000 in 1.9358 seconds. Average chunk time: 1.8758 seconds [INFOCOUNT] Saving part 7/25: [INFOTIMER] Completed chunk 7/50000 in 1.9119 seconds. Average chunk time: 1.881 seconds [INFOCOUNT] Saving part 8/25: [INFOTIMER] Completed chunk 8/50000 in 1.9635 seconds. Average chunk time: 1.8913 seconds [INFOCOUNT] Saving part 9/25: [INFOTIMER] Completed chunk 9/50000 in 1.9213 seconds. Average chunk time: 1.8946 seconds [INFOCOUNT] Saving part 10/25: [INFOTIMER] Completed chunk 10/50000 in 1.8665 seconds. Average chunk time: 1.8918 seconds [INFOCOUNT] Saving part 11/25: [INFOTIMER] Completed chunk 11/50000 in 1.8834 seconds. Average chunk time: 1.891 seconds [INFOCOUNT] Saving part 12/25: [INFOTIMER] Completed chunk 12/50000 in 1.8265 seconds. Average chunk time: 1.8857 seconds [INFOCOUNT] Saving part 13/25: [INFOTIMER] Completed chunk 13/50000 in 1.8175 seconds. Average chunk time: 1.8804 seconds [INFOCOUNT] Saving part 14/25: [INFOTIMER] Completed chunk 14/50000 in 1.8106 seconds. Average chunk time: 1.8754 seconds [INFOCOUNT] Saving part 15/25: [INFOTIMER] Completed chunk 15/50000 in 1.8231 seconds. Average chunk time: 1.8719 seconds [INFOCOUNT] Saving part 16/25: [INFOTIMER] Completed chunk 16/50000 in 1.8239 seconds. Average chunk time: 1.8689 seconds [INFOCOUNT] Saving part 17/25: [INFOTIMER] Completed chunk 17/50000 in 1.8264 seconds. Average chunk time: 1.8664 seconds [INFOCOUNT] Saving part 18/25: [INFOTIMER] Completed chunk 18/50000 in 1.8713 seconds. Average chunk time: 1.8667 seconds [INFOCOUNT] Saving part 19/25: [INFOTIMER] Completed chunk 19/50000 in 1.8446 seconds. Average chunk time: 1.8655 seconds [INFOCOUNT] Saving part 20/25: [INFOTIMER] Completed chunk 20/50000 in 1.8697 seconds. Average chunk time: 1.8657 seconds [INFOCOUNT] Saving part 21/25: [INFOTIMER] Completed chunk 21/50000 in 1.8398 seconds. Average chunk time: 1.8645 seconds [INFOCOUNT] Saving part 22/25: [INFOTIMER] Completed chunk 22/50000 in 1.8648 seconds. Average chunk time: 1.8645 seconds [INFOCOUNT] Saving part 23/25: [INFOTIMER] Completed chunk 23/50000 in 1.8803 seconds. Average chunk time: 1.8652 seconds [INFOCOUNT] Saving part 24/25: [INFOTIMER] Completed chunk 24/50000 in 1.9122 seconds. Average chunk time: 1.8672 seconds [INFOCOUNT] Saving part 25/25: [INFOTIMER] Completed chunk 25/50000 in 1.8473 seconds. Average chunk time: 1.8664 seconds [INFOTIMER] Looped over methylation loci in 46.6664 seconds [INFOTIMER] Calculated regression_full in 47.2857 seconds [INFOTIMER] Waiting for chunks to save... [INFOTIMER] Finished waiting for chunks to save in 0.3018 seconds ```
tecpg run mlr-full -l 2000 -p 0.05 --distal ```python [INFOCOUNT] Done saving part 2 [INFOCOUNT] Done saving part 10 [INFOCOUNT] Done saving part 18 [INFOCOUNT] Done saving part 3 [INFOCOUNT] Done saving part 11 [INFOCOUNT] Done saving part 19 [INFOCOUNT] Done saving part 4 [INFOCOUNT] Done saving part 12 [INFOCOUNT] Done saving part 20 [INFOCOUNT] Done saving part 5 [INFOCOUNT] Done saving part 13 [INFOCOUNT] Done saving part 21 [INFOCOUNT] Done saving part 6 [INFOCOUNT] Done saving part 14 [INFOCOUNT] Done saving part 22 [INFOCOUNT] Done saving part 7 [INFOCOUNT] Done saving part 15 [INFOCOUNT] Done saving part 23 [INFOCOUNT] Done saving part 8 [INFOCOUNT] Done saving part 16 [INFOCOUNT] Done saving part 24 [INFOCOUNT] Done saving part 1 [INFOCOUNT] Done saving part 9 [INFOCOUNT] Done saving part 17 [INFOCOUNT] Done saving part 25 [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.0019 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.4196 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 2.0035 seconds [INFOTIMER] Finished reading 3 dataframes in 2.4251 seconds. [INFO] No region window provided. Resorting to default. [INFO] Using default window for distal of 50000 bases [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 296 degrees of freedom [INFO] Initializing output directory [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0066 seconds [INFOTIMER] Converted G to tensor in 0.009 seconds [INFOTIMER] Created ones in 0.0001 seconds [INFOTIMER] Created X in 0.0003 seconds [INFOTIMER] Transposed X in 0.0 seconds [INFOTIMER] Calculated XtXi in 0.6012 seconds [INFOTIMER] Calculated XtXi_diag in 0.0001 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0002 seconds [INFOTIMER] Calculated X constants in 0.6176 seconds [INFO] CUDA device memory: 97.112064 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOCOUNT] Saving part 1/25: [INFOTIMER] Completed chunk 1/50000 in 1.9097 seconds. Average chunk time: 1.9097 seconds [INFOCOUNT] Saving part 2/25: [INFOTIMER] Completed chunk 2/50000 in 1.8088 seconds. Average chunk time: 1.8593 seconds [INFOCOUNT] Saving part 3/25: [INFOTIMER] Completed chunk 3/50000 in 2.0572 seconds. Average chunk time: 1.9252 seconds [INFOCOUNT] Saving part 4/25: [INFOTIMER] Completed chunk 4/50000 in 1.9811 seconds. Average chunk time: 1.9392 seconds [INFOCOUNT] Saving part 5/25: [INFOTIMER] Completed chunk 5/50000 in 2.0422 seconds. Average chunk time: 1.9598 seconds [INFOCOUNT] Saving part 6/25: [INFOTIMER] Completed chunk 6/50000 in 2.0127 seconds. Average chunk time: 1.9686 seconds [INFOCOUNT] Saving part 7/25: [INFOTIMER] Completed chunk 7/50000 in 2.0572 seconds. Average chunk time: 1.9813 seconds [INFOCOUNT] Saving part 8/25: [INFOTIMER] Completed chunk 8/50000 in 2.0535 seconds. Average chunk time: 1.9903 seconds [INFOCOUNT] Saving part 9/25: [INFOTIMER] Completed chunk 9/50000 in 2.1087 seconds. Average chunk time: 2.0034 seconds [INFOCOUNT] Saving part 10/25: [INFOTIMER] Completed chunk 10/50000 in 2.0046 seconds. Average chunk time: 2.0036 seconds [INFOCOUNT] Saving part 11/25: [INFOTIMER] Completed chunk 11/50000 in 1.9632 seconds. Average chunk time: 1.9999 seconds [INFOCOUNT] Saving part 12/25: [INFOTIMER] Completed chunk 12/50000 in 1.8918 seconds. Average chunk time: 1.9909 seconds [INFOCOUNT] Saving part 13/25: [INFOTIMER] Completed chunk 13/50000 in 2.0037 seconds. Average chunk time: 1.9919 seconds [INFOCOUNT] Saving part 14/25: [INFOTIMER] Completed chunk 14/50000 in 1.9792 seconds. Average chunk time: 1.991 seconds [INFOCOUNT] Saving part 15/25: [INFOTIMER] Completed chunk 15/50000 in 1.9628 seconds. Average chunk time: 1.9891 seconds [INFOCOUNT] Saving part 16/25: [INFOTIMER] Completed chunk 16/50000 in 1.94 seconds. Average chunk time: 1.986 seconds [INFOCOUNT] Saving part 17/25: [INFOTIMER] Completed chunk 17/50000 in 2.025 seconds. Average chunk time: 1.9883 seconds [INFOCOUNT] Saving part 18/25: [INFOTIMER] Completed chunk 18/50000 in 1.9617 seconds. Average chunk time: 1.9868 seconds [INFOCOUNT] Saving part 19/25: [INFOTIMER] Completed chunk 19/50000 in 1.9649 seconds. Average chunk time: 1.9857 seconds [INFOCOUNT] Saving part 20/25: [INFOTIMER] Completed chunk 20/50000 in 1.9708 seconds. Average chunk time: 1.9849 seconds [INFOCOUNT] Saving part 21/25: [INFOTIMER] Completed chunk 21/50000 in 1.9359 seconds. Average chunk time: 1.9826 seconds [INFOCOUNT] Saving part 22/25: [INFOTIMER] Completed chunk 22/50000 in 1.9913 seconds. Average chunk time: 1.983 seconds [INFOCOUNT] Saving part 23/25: [INFOTIMER] Completed chunk 23/50000 in 1.9558 seconds. Average chunk time: 1.9818 seconds [INFOCOUNT] Saving part 24/25: [INFOTIMER] Completed chunk 24/50000 in 1.9413 seconds. Average chunk time: 1.9801 seconds [INFOCOUNT] Saving part 25/25: [INFOTIMER] Completed chunk 25/50000 in 1.9533 seconds. Average chunk time: 1.9791 seconds [INFOTIMER] Looped over methylation loci in 49.4837 seconds [INFOTIMER] Calculated regression_full in 50.1013 seconds [INFOTIMER] Waiting for chunks to save... [INFOTIMER] Finished waiting for chunks to save in 0.4985 seconds ```
tecpg run mlr-full -l 2000 -p 0.05 --trans ```python [INFOCOUNT] Done saving part 2 [INFOCOUNT] Done saving part 10 [INFOCOUNT] Done saving part 18 [INFOCOUNT] Done saving part 3 [INFOCOUNT] Done saving part 11 [INFOCOUNT] Done saving part 19 [INFOCOUNT] Done saving part 5 [INFOCOUNT] Done saving part 13 [INFOCOUNT] Done saving part 21 [INFOCOUNT] Done saving part 4 [INFOCOUNT] Done saving part 12 [INFOCOUNT] Done saving part 20 [INFOCOUNT] Done saving part 6 [INFOCOUNT] Done saving part 14 [INFOCOUNT] Done saving part 22 [INFOCOUNT] Done saving part 7 [INFOCOUNT] Done saving part 15 [INFOCOUNT] Done saving part 23 [INFOCOUNT] Done saving part 8 [INFOCOUNT] Done saving part 16 [INFOCOUNT] Done saving part 24 [INFOCOUNT] Done saving part 1 [INFOCOUNT] Done saving part 9 [INFOCOUNT] Done saving part 17 [INFOCOUNT] Done saving part 25 [INFO] CUDA GPU detected. This device supports CUDA. [INFO] Reading 3 dataframes... [INFOTIMER] Reading 1/3: C.csv [INFO] Reading csv file \data\C.csv with separator , [INFOTIMER] Read 1/3 in 0.0019 seconds [INFOTIMER] Reading 2/3: G.csv [INFO] Reading csv file \data\G.csv with separator , [INFOTIMER] Read 2/3 in 0.4489 seconds [INFOTIMER] Reading 3/3: M.csv [INFO] Reading csv file \data\M.csv with separator , [INFOTIMER] Read 3/3 in 2.0877 seconds [INFOTIMER] Finished reading 3 dataframes in 2.5386 seconds. [INFO] Initializing regression variables [INFO] Use CPU not supplied. Checking if CUDA is available. [INFO] Using CUDA [INFO] Running with 296 degrees of freedom [INFO] Initializing output directory [INFO] Removing directory \output... [INFO] Creating directory \output... [INFO] Running regression_full... [INFOTIMER] Converted C to tensor in 0.0066 seconds [INFOTIMER] Converted G to tensor in 0.0102 seconds [INFOTIMER] Created ones in 0.0001 seconds [INFOTIMER] Created X in 0.0004 seconds [INFOTIMER] Transposed X in 0.0 seconds [INFOTIMER] Calculated XtXi in 0.6099 seconds [INFOTIMER] Calculated XtXi_diag in 0.0001 seconds [INFOTIMER] Calculated XtXi_Xt in 0.0003 seconds [INFOTIMER] Calculated X constants in 0.6277 seconds [INFO] CUDA device memory: 97.112064 MB allocated by constants out of 8589.47584 MB total [INFO] Calculating regression... [INFOCOUNT] Saving part 1/25: [INFOTIMER] Completed chunk 1/50000 in 5.6669 seconds. Average chunk time: 5.6669 seconds [INFOCOUNT] Saving part 2/25: [INFOTIMER] Completed chunk 2/50000 in 5.7196 seconds. Average chunk time: 5.6932 seconds [INFOCOUNT] Saving part 3/25: [INFOTIMER] Completed chunk 3/50000 in 5.7899 seconds. Average chunk time: 5.7254 seconds [INFOCOUNT] Saving part 4/25: [INFOTIMER] Completed chunk 4/50000 in 5.912 seconds. Average chunk time: 5.7721 seconds [INFOCOUNT] Saving part 5/25: [INFOTIMER] Completed chunk 5/50000 in 5.8627 seconds. Average chunk time: 5.7902 seconds [INFOCOUNT] Saving part 6/25: [INFOTIMER] Completed chunk 6/50000 in 5.9721 seconds. Average chunk time: 5.8205 seconds [INFOCOUNT] Saving part 7/25: [INFOTIMER] Completed chunk 7/50000 in 5.825 seconds. Average chunk time: 5.8211 seconds [INFOCOUNT] Saving part 8/25: [INFOTIMER] Completed chunk 8/50000 in 5.8622 seconds. Average chunk time: 5.8263 seconds [INFOCOUNT] Saving part 9/25: [INFOTIMER] Completed chunk 9/50000 in 5.7841 seconds. Average chunk time: 5.8216 seconds [INFOCOUNT] Saving part 10/25: [INFOTIMER] Completed chunk 10/50000 in 5.8557 seconds. Average chunk time: 5.825 seconds [INFOCOUNT] Saving part 11/25: [INFOTIMER] Completed chunk 11/50000 in 5.7874 seconds. Average chunk time: 5.8216 seconds [INFOCOUNT] Saving part 12/25: [INFOTIMER] Completed chunk 12/50000 in 5.83 seconds. Average chunk time: 5.8223 seconds [INFOCOUNT] Saving part 13/25: [INFOTIMER] Completed chunk 13/50000 in 5.7962 seconds. Average chunk time: 5.8203 seconds [INFOCOUNT] Saving part 14/25: [INFOTIMER] Completed chunk 14/50000 in 5.7537 seconds. Average chunk time: 5.8155 seconds [INFOCOUNT] Saving part 15/25: [INFOTIMER] Completed chunk 15/50000 in 5.8191 seconds. Average chunk time: 5.8158 seconds [INFOCOUNT] Saving part 16/25: [INFOTIMER] Completed chunk 16/50000 in 5.774 seconds. Average chunk time: 5.8132 seconds [INFOCOUNT] Saving part 17/25: [INFOTIMER] Completed chunk 17/50000 in 5.8549 seconds. Average chunk time: 5.8156 seconds [INFOCOUNT] Saving part 18/25: [INFOTIMER] Completed chunk 18/50000 in 5.7711 seconds. Average chunk time: 5.8131 seconds [INFOCOUNT] Saving part 19/25: [INFOTIMER] Completed chunk 19/50000 in 5.7972 seconds. Average chunk time: 5.8123 seconds [INFOCOUNT] Saving part 20/25: [INFOTIMER] Completed chunk 20/50000 in 5.8192 seconds. Average chunk time: 5.8126 seconds [INFOCOUNT] Saving part 21/25: [INFOTIMER] Completed chunk 21/50000 in 5.7369 seconds. Average chunk time: 5.809 seconds [INFOCOUNT] Saving part 22/25: [INFOTIMER] Completed chunk 22/50000 in 5.8447 seconds. Average chunk time: 5.8107 seconds [INFOCOUNT] Saving part 23/25: [INFOTIMER] Completed chunk 23/50000 in 5.7217 seconds. Average chunk time: 5.8068 seconds [INFOCOUNT] Saving part 24/25: [INFOTIMER] Completed chunk 24/50000 in 5.7916 seconds. Average chunk time: 5.8062 seconds [INFOCOUNT] Saving part 25/25: [INFOTIMER] Completed chunk 25/50000 in 5.8295 seconds. Average chunk time: 5.8071 seconds [INFOTIMER] Looped over methylation loci in 145.1862 seconds [INFOTIMER] Calculated regression_full in 145.8139 seconds [INFOTIMER] Waiting for chunks to save... [INFOTIMER] Finished waiting for chunks to save in 3.7706 seconds ```
liamgd commented 1 year ago

Comparison to the previous region filtration implementation to the new one. The first value in seconds is the previous time, and the second value is the new time.

No chunks, no saving:

tecpg run mlr-full: *Too much GPU memory allocation*
tecpg run mlr-full --cis: 93.1929 seconds to 41.4291 seconds
tecpg run mlr-full --distal: 94.3363 seconds to 43.6176 seconds
tecpg run mlr-full --trans: *Too much GPU memory allocation*

tecpg run mlr-full -p 0.05: 93.2036 seconds to 96.6992 seconds
tecpg run mlr-full -p 0.05 --cis: 93.427 seconds to 50.5002 seconds
tecpg run mlr-full -p 0.05 --distal: 93.5236 seconds to 45.169 seconds
tecpg run mlr-full -p 0.05 --trans: 93.5954 seconds to 133.2633 seconds

Chunks of 2000 methylation loci, saving included

tecpg run mlr-full -l 2000: 169.2086 seconds to 173.3093 seconds
tecpg run mlr-full -l 2000 --cis: 102.665 seconds to 49.2062 seconds
tecpg run mlr-full -l 2000 --distal: 104.6743 seconds to 59.2901 seconds
tecpg run mlr-full -l 2000 --trans: 180.5167 seconds to 235.1578 seconds

tecpg run mlr-full -l 2000 -p 0.05: 101.5071 seconds to 101.6262 seconds
tecpg run mlr-full -l 2000 -p 0.05 --cis: 97.5001 seconds to 47.2857 seconds
tecpg run mlr-full -l 2000 -p 0.05 --distal: 98.7852 seconds to 50.1013 seconds
tecpg run mlr-full -l 2000 -p 0.05 --trans: 101.6967 seconds to 145.8139 seconds
liamgd commented 1 year ago
liamgd commented 1 year ago

In my opinion, for large input matrices, the new way is faster. I believe this algorithm scales better with larger inputs because with respect to the gene expression count, the time complexity with the previous algorithm is superlinear for the computation section and the region mask creation and filtration runs in linear time. With the new algorithm, the linear mask creation and filtration happen before the superlinear computation. Assuming g gene expression loci, the number of gene expression loci for the computation would be a factor between 0 and 1 multiplied by that number. The superlinear function in which the computation runs would then scale that factor even lower, which accounts for the theoretical speed improvement of the new function for large inputs.

liamgd commented 1 year ago

Moved filtration before computation in e8fc49a.

Further optimization warrants a new issue.