scikit compatible learners

[apoorvalal]

Hi James,

great work on this. Was wondering if you were planning to provide a scikit-learn style API for this package? Would largely involve writing init/fit/predict/score methods, which would then allow adelie to be used in a number of useful downstream tasks (e.g. what i'm interested in - nuisance function estimation for AIPW / Rlearner for ATE/CATE estimation in econml / doubleml). Celer is a closely related package that provides a scikit-style API.

Totally understand if this isn't a priority; I might take a crack at it and send you a PR in case you're open to contributions.

[JamesYang007]

Hi Apoorva,

Thanks so much for your interest! Yes, that was on my list of TODOs but it is indeed a low priority right now. I'd be happy to accept any contributions. If you submit a PR with a sketch of what you're looking for, I can flesh out more of the detail if necessary.

[darior]

Hi James, thanks a lot for building and sharing this package! I work in genomics and found it useful for problems where other tools were prohibitively slow.

It took a bit of work to implement the training and evaluation workflow, but code examples examples were very helpful. I wanted to add a couple suggestions to improve the API and noticed this issue so I'll just comment here as they're related.

Couple suggestions below:

• working with matrix indices when preparing inputs (with interactions) required a lot of care to do correctly. This could be abstracted (creating interaction matrices, handling group and penalty specification for different variable types) using a higher level API where used would just provide X, y together with a dictionary of interactions to evaluate, list of variable types (categorical vs continuous) and other custom parameters.
• similarly, extracting active interactions from results and figuring out how their indices relate to original input matrix was a bit challenging and could be automated with a higher level method.

[JamesYang007]

Hi Darek, thanks for your interest as well! I'm very glad to hear that adelie has been suitable for your purpose. Indeed, the initial motivation for my developing adelie was to run group lasso on GWAS datasets as no existing tools could support such a scale of data.

• I'm not sure if you saw this but ad.matrix.interaction provides extra properties that outputs "canonical" grouping: https://jamesyang007.github.io/adelie/generated/adelie.adelie_core.matrix.MatrixNaiveInteractionDense64F.html#adelie.adelie_core.matrix.MatrixNaiveInteractionDense64F. Specifically, it has .groups and .group_sizes. The grouping is in terms of the interaction block. For example, continuous-continuous interaction is a group of 3, continuous-categorical is a group of C (number of classes), and categorical-categorical is a group of C_1 * C_2 where C_i (i=1,2) is the number of classes for the respective categorical feature. I had hoped this would be sufficient.

  I'm a bit hesitant on creating a higher level function/class to encapsulate this logic. I actually had the exact same discussion with my advisor on this exact application, so I'll just explain what led to this design choice of letting the user do all the input prep. As mentioned in the documentation, my advisor and a past student of his worked on an R package called glinternet, which solves this interaction matrix + group lasso. It was quite slow and he wondered if adelie can do the job faster. That was the reason why I exposed this interaction matrix. When I showed him that glinternet can be reproduced with adelie (faster too!), he also remarked that there's a lot of input preparation happening and asked if I could provide a higher level function similar to glinternet(). While I understood the value of such a function, I fought against it for two reasons: 1) this type of interaction matrix is an intuitive one but not a "standard" one and 2) this function doesn't fit into the current organization/package structure. I instead suggested glinternet should come out with a new version that uses adelie internally and all the input preparation can be encapsulated there and this is what we agreed on. I am open to adding a submodule with this sugary function, but I guess I'm just waiting for more examples before I can figure out how exactly I should structure this potentially new submodule.

  I imagine the function you're looking for is different from Apoorva's suggestion. I think he's looking for a general-purpose sklearn-type class that does exactly the same thing as ad.grpnet and not anything specific to like interaction matrices etc.

• This I can definitely expose! I imagine this would be something like ad.diagnostic.active(groups, betas) which provides the indices to groups for which the corresponding coefficients given by betas is non-zero.

Curious to hear your thoughts!

[darior]

Hi James,

• I get the reasoning. I had tried glinternet in R before testing adelie and found the API pretty intuitive and easy to use, but had to switch to adelie due to performance issues. Reimplementing glinternet internals with adelie seems like a really good idea! You could also consider creating a simplified interface to adelie in Python which would provide glinternet like API via separate package if it does not fit well with the current one, but having a fast version of R glinternet running on adelie would be totally sufficient.

• This could help a bit, but the user would still need to create interaction matrices, combine them with main effects and keep track of groups so I would not spend too much time on this.