Ability to choose a custom filter

[ilyakatz]

Hi @jgillick, you're doing an amazing job with this project!

I think it was be awesome if a filter created on the website could be used to select loans. Basically, there are two steps

1. https://www.lendingclub.com/browse/getSavedFiltersAj.action returns a json of my filters
2. https://www.lendingclub.com/browse/getSavedFilterAj.action?id=FILTER_ID return json that, I think, can be used on this line https://github.com/jgillick/LendingClubAutoInvestor/blob/master/LendingClubInvestor/__init__.py#L294

I'm not a python guy, but I'm willing to help out if you think you want to add this feature (or anything else really, I'm very interested in this project)

[ghost]

That's a great idea. So if I understand your request, the end user could create custom filters (like it works today) or choose to use a saved one? This would probably work similarly to the portfolio picker that lets the user choose an existing portfolio to assign the investments to. Or perhaps they could just pass a filter ID on the command line when they start lcinvestor. What are your thoughts?

[TobyDai]

Have you looked into the soap API the lending club is currently working on? here is the link:

https://www.lendingclub.com/ws/1.3

there are 92 total attributes. To make this thing truly great, user should be able to leverage all those attributes and build custom searches.

Ideally I can import models I built in excel based on these 92 attributes, and automatically fund those that matches my model.

[ilyakatz]

So if I understand your request, the end user could create custom filters (like it works today) or choose to use a saved one?

I guess that depends on how far you want to take this project. For me, I would be ok with just reusing the filters I create on the site and not overburden this utility with the task of creating new filters. Whatever is easier for you - either having user to pass the filter id or actually retrieve all current filters. I'll leave that up to you.

[ghost]

@15L

Have you looked into the soap API the lending club is currently working on?

That's cool. I had heard rumor about a soap API, but this is the first I've actually seen it. Currently the tool is using LendingClub's JSON web services. I'm working on extracting the API parts of the tool into their own standalone python module that can be used in other projects. Maybe many of the requests can be converted to the SOAP API.

[ghost]

@ilyakatz

That's a good point. I don't think it would be too hard to add a command line flag that lets a user set the saved filter to use. This will probably get wrapped into the API module I'm working on.

[ilyakatz]

off topic - i wonder why they don't just make their json api more available/documented, why create soap API when the current one is pretty good :)

[TobyDai]

Going through json, you are limited by the web interface and the filters it offers. With the soap API you can customize your searches way beyond what the web interface offers. Should be much easier to work with as well.

[ghost]

True. The JSON API is also incomplete and my code has to compensate for it in a number of places.

@15L It's been a loooooong time since I've worked in SOAP, good thing there number of python modules to help me with that. But from a high level, I forget how to dig into the WSDL more. For example, how do I get more information on what options LoanBrowseLoans operation has?

[TobyDai]

You will have to contact lending club to have them enable api for your account before you can test. Once your account is enabled, soapui is a great tool to use to poke around with.

[ilyakatz]

@15L What I meant, I wish they expanded the json api instead of creating a completely separate one.

Do you know how we request developer key? Is it even public?

[ghost]

Cool. Thanks for the resources. Unless I'm wrong, it looks like their SOAP API is currently pretty limited. The browse loans operation takes no parameters and seems to simply return all available notes on their platform:

This operation provides a listing of all loans that are in funding on the platform.... The operation provides a complete listing available for this user and does not provide any way of filtering the loans.

So the filtering would have to be done on the python side manually. Also, having to enable API access per account seems to mean that anyone who would want to use lcinvestor would have to contact LendingClub first.

That said, I'm really encouraged that they're working on a formal public API and hope that it'll be more accessible in the future. Where'd you find these resources? When I googled for "Lending Club SOAP API" I didn't get anything.

[TobyDai]

Ya, browse loans returns a complete list, then you can filter the results locally.

As far as the support docs, they aren't available for the public yet, everything is still in beta stage. From the looks of it, they will probably be in beta for a while.

If you download the LC data set, it currently has 92 attributes for each loan. Are you able to filter the data with all those attributes, or only whats available to you on LC web interface?

[ghost]

True, the filtering could be done on the python side. Could possibly do this with the project to extract the LC calls into a dedicated API module.

It's pretty cool that you got access to the beta docs.

[ghost]

Good news. I have effectively extracted the part of the tool that interfaces with LendingClub into it's own standalone API module, added new features (like using saved filters) and am now starting to integrate this new module back into the investor tool (git branch v2.0). The API module itself is still in alpha but my tests so far come out pretty solid. Hopefully I'll have the 2.0 release packaged and stable by the end of the month.

Here's the new API module: https://github.com/jgillick/LendingClub

[jgillick]

Your feature request has now been officially added to the LendingClub tools. The latest release uses two packages, lcinvestor and lendingclub. The lcinvestor package contains the same app that you've been using. The lendingclub package is the API module for Lending Club.

The only change you should notice is the additional prompt for saved filters. If you enter 'Y' when it asks "Would you like to select one of your saved filters...", you will be presented with a list of your filters from the site that you can choose from.

I've been running this version of the tool for a couple weeks and have made several investments with it successfully. Try it and let me know how it works for you.

[ilyakatz]

That's really cool. Thanks @jgillick. I will give it a spin and will report back

[ilyakatz]

I tried to install lendingclub

sudo pip install lendingclub

And I got this

➜  LendingClubAutoInvestor git:(master) sudo pip install lendingclub
Downloading/unpacking lendingclub
  Running setup.py egg_info for package lendingclub
    Traceback (most recent call last):
      File "<string>", line 16, in <module>
      File "/private/tmp/pip-build-root/lendingclub/setup.py", line 5, in <module>
        version=open('lendingclub/VERSION').read(),
    IOError: [Errno 2] No such file or directory: 'lendingclub/VERSION'
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):

  File "<string>", line 16, in <module>

  File "/private/tmp/pip-build-root/lendingclub/setup.py", line 5, in <module>

    version=open('lendingclub/VERSION').read(),

IOError: [Errno 2] No such file or directory: 'lendingclub/VERSION'

----------------------------------------
Command python setup.py egg_info failed with error code 1 in /private/tmp/pip-build-root/lendingclub
Storing complete log in /Users/ilyakatz/.pip/pip.log

[jgillick]

Thanks for the heads up. It should be fixed now. For good measure, start by uninstalling lendingclub before installing lcinvestor:

sudo pip uninstall lendingclub

Then

sudo pip install lcinvestor

If it says something like "Requirement already satisfied...", use the --upgrade flag:

sudo pip install lcinvestor --upgrade

[jgillick]

(FYI: I also just found and fixed a small bug with saved filters)

[ilyakatz]

I'm still having the same error

IOError: [Errno 2] No such file or directory: 'lendingclub/VERSION'

I do see it in the git repo, is there anything that I need to do with pip to let it know to use the latest version?

[jgillick]

The lendingclub directory no longer exists in the lcinvestor project, so pip is probably pulling the package from cache.

Remove the cache with this command:

rm -r /tmp/pip-build-root/

Then uninstall lcinvestor and try again

sudo pip uninstall lcinvestor
sudo pip install lcinvestor

If that doesn't work, install manually from git:

# First install the new lending club API module
sudo pip install lendingclub

# Now install lcinvestor
git clone https://github.com/jgillick/LendingClubAutoInvestor.git
cd LendingClubAutoInvestor
sudo python setup.py install

[ilyakatz]

ok, it worked now (i had to reboot my machine so it probably cleaned up temp directories). Anyways, I was able to run and it did select my portfolio. Thanks very much for adding this feature!

[jgillick]

Great. I'm glad that it worked. I just fixed some small things with the lendingclub API module, so if you see any errors, upgrade it:

sudo pip install lendingclub --upgrade

[ilyakatz]

Hey @jgillick, actually, I think my previous attempts did't work. I updated the api module and tries to run it

INFO: 2013-07-30 10:13 - Auto investing $200 at 10.95%...
ERROR: 2013-07-30 10:13 - Failed trying to invest: The loans property must be a list

P.S. Looks like LC has an interesting bug if you have more cash than what can be invested into available loans (eg. I have $100 but with my filters there are only $75 worth of loans), nothing will show up

[jgillick]

Oh, you're right, I was able to reproduce your bug. Such a silly little one too. I have updated both lcinvestor and lendingclub to fix this. Update them both and you should be good to go.

sudo pip install lcinvestor --upgrade
sudo pip install lendingclub --upgrade

As for the interesting bug of $100 vs $75, that's a very good use case. Currently the lc.com site doesn't provide any indicator that there are $75 worth of loans if you tried to invest $100. Like the site, you say how much you want to invest and it tells you if there are enough matching loans.

However, there is a possible way to fix this in the program... Let's say that you have setup you program to invest when your account has at least $50, and when the program runs you actually have $125. First it will try to invest the entire $125. If nothing is found, it could search a few more times at decreasing values, as long as they're greater than $50. So, for example: $125 -- nothing, $100 -- nothing, $75 -- LOANS FOUND! If nothing was found at $75, it will try once more at $50.

How does that sound? This will, of course all happen behind the scenes without your direct input.

[ilyakatz]

Nice, it works now!

I would have hope that LC will change this behavior fix it - it looks a bit misleading, for a while I thought that my filters were so specific that it couldn't find anything. If you do want to tackle in step approach, I would recommend a percentage based back-out (eg, 15%), so if I have $1000 it would go to to $850, then to $637, then to $478 so this way it could handle large amounts.

[jgillick]

A feature enhancement ticket has been created for this: https://github.com/jgillick/LendingClubAutoInvestor/issues/9

[jgillick]

Your request has been implemented. Now in version 2.1.2, if there are no loan portfolios available for your entire balance, it will attempt up to 6 investments amount between your available balance and your minimum amount set. Try the latest version by upgrading with:

pip install lcinvestor -U

Let me know if it works for you.