[PROJECT IDEA]Add Double Lasso to Python Ecosystem

[jaredDlewis]

Main Project Goal

Add Double Lasso functionality for feature selection to the Python ecosystem. This could be done by creating a standalone library, but it would make a much bigger impact on the Python community if incorporated into the open source statsmodels library.

Why This Project is Interesting

Using double lasso for feature selection is a statistically sound way to produce a parsimonious linear regression model for statistical inference, but it is not currently implemented in any easy-to-find publicly available python libraries. It is currently available for Stata users, but it would be a great addition to the Python ecosystem to open-source the functionality for statisticians and researchers who don't have a Stata license. There are also some interesting technical challenges (listed below) that would allow the developer of this functionality to dive into the front lines of statistical methods.

Brief Description of Work Involved

Add a tool to the Python ecosystem that implements Double Lasso similar to Stata's current implementation. You should also look at the options provided in Stata's Lasso implementation and compare it to statsmodels's options to see what kind of functionality you can add. Some notable challenges:

• Implementing Stata's default lambda selection method for single lasso regression, which is the plugin iterative formula. Stata has information on how they implement this in their documentation. This method of lambda selection results in one of the most conservative/robust models for statistical inference, but it is currently not implemented in Scikit-Learn or statsmodels.
• Handling categorical variables that have been one hot encoded. Double Lasso should choose either all or none of the dummy variables associated with the original categorical variable since the dummy variables outputted from one hot encoding depend on one another for meaning in the final model. You may find Celer's Group Lasso or a similar tool helpful for this.

First Steps

• Develop an understanding of Double Lasso and its purpose. Columbia University Professor Gerard Torrats-Espinosa has a good youtube video about applying Double Lasso in his research.
• Familiarize yourself with statsmodels's Contribution process.
• Gerard's research paper is a great candidate to try to reproduce with your python implementation to because his dataset and code are available and the dataset requires tackling the notable challenges for double lasso listed above. You'll want to reproduce the results of using Stata's double lasso to ensure that your Python implementation is working as expected. Here is a youtube video about the research that prompted this Project Idea.

[JonathanBechtel]

@jaredDlewis Thanks. Question: is the Group Lasso segment in addition to Double LASSO or a part of it? I ask since there's already a Group LASSO implementation in python: https://group-lasso.readthedocs.io/en/latest/