jgillick / LendingClubAutoInvestor

A simple command line tool that will automatically invest all cash that becomes available
MIT License
47 stars 21 forks source link

note amount over limit #13

Open snikolaev opened 10 years ago

snikolaev commented 10 years ago

Hi Jeremy,

I have experienced this issue twice now. My investment limit is $125 per note. However, twice now, a $175 note has been invested in. I have captured the logs for the last occurrence. See below.

$ cat .lcinvestor/daemon.log started with pid 12604 INFO: 2013-09-07 06:04 - Authenticated INFO: 2013-09-07 06:04 - Checking for funds to invest... INFO: 2013-09-07 06:04 - $ $ $ $ $ $ $ $ $ $ INFO: 2013-09-07 06:04 - Searching for a portfolio for $175 INFO: 2013-09-07 06:04 - Auto investing $175 at 19.52%... INFO: 2013-09-07 06:05 - Investment portfolio summary: 1 loan notes (D:100%) INFO: 2013-09-07 06:05 - Done

INFO: 2013-09-07 07:05 - Authenticated INFO: 2013-09-07 07:05 - Checking for funds to invest... INFO: 2013-09-07 07:05 - Only $0 available for investing (of your $11.21 balance) INFO: 2013-09-07 08:05 - Authenticated INFO: 2013-09-07 08:05 - Checking for funds to invest... INFO: 2013-09-07 08:05 - Only $0 available for investing (of your $11.21 balance) $ cat .lcinvestor/last_investment.json {"order_id": true, "timestamp": 1378559101, "cash": 175, "investment": {"aa": 0.0, "percent_of_year3_loans": 0.0, "width_of_collateral_loans": "0%", "number_of_personal_loans": 1, "percent_of_personal_loans": 1.0, "percent_of_year5_loans": 1.0, "projectedReturns": 0.10260000000000002, "loan_fractions": [{"loanRate": "19.52", "isWholeLoan": 0, "wholeLoanTimeRemaining": -216245, "invest_amount": 175, "alreadyInvestedIn": false, "loanGrade": "D4", "loanType": "Personal", "title": "debt consolidation", "loanExpirationDate": "2013-09-20 18:16:11.0", "loan_id": 7091656, "loanLength": 60, "uncrunch": "NONE", "purpose": "debt_consolidation", "loanFractionAmount": 175, "loan_status": "INFUNDING", "noFee": 0, "loanFractionGUID": 39347880, "loanAmountRequested": 35000, "loanTimeRemaining": 1167110755, "loanUnfundedAmount": 13175}], "numberOfLoans": 1, "number_of_aa_loans": null, "risk": 0.0, "number_of_year3_loans": null, "serviceFee": 0.0046, "number_of_c_loans": 0, "number_of_e_loans": 0, "number_of_year5_loans": 1, "number_of_a_loans": 0, "width_of_personal_loans": "100%", "width_of_year3_loans": "0%", "percentage": 19.52, "historicalAnnualDefaultRate": 0.088, "number_of_f_loans": 0, "number_of_g_loans": 0, "number_of_b_loans": 0, "number_of_d_loans": 1, "percent_of_collateral_loans": 0.0, "a": 0.0, "c": 0.0, "b": 0.0, "e": 0.0, "d": 100.0, "g": 0.0, "f": 0.0, "number_of_collateral_loans": null, "width_of_year5_loans": "100%"}, "portfolio": "Autoinvested"} $ cat .lcinvestor/investing.json {"max_per_note": 125, "portfolio": "Autoinvested", "filters": 8071248, "filter_id": 8071248, "min_percent": 15.0, "min_cash": 125, "max_percent": 30.0, "email": "email@gmail.com"} $ $ cat etc/investing.json { "min_cash": 125, "min_percent": 15, "max_percent": 30, "max_per_note": 125, "portfolio": "Autoinvested", "filter_id": 8071248, "filters": { "exclude_existing": true, "funding_progress": 90, "term60month": true, "term36month": true, "grades": { "All": false, "A": true, "B": false, "C": false, "D": false, "E": false, "F": false, "G": false } } }

Thanks, Sergey

eli88fine commented 10 years ago

Hello, I just started using this tool (great idea!) but I am also noticing this issue. I have $25 set as the "max per note" amount in the JSON files, but I have seen several notes invested in with $50. Sorry I don't know how to pull the detailed logs like Seregy does, but just wanted to let you know that this problem is also occurring for other people.

Thanks, Eli

popolr commented 10 years ago

Hi Jeremy, Thanks a lot for developing this tool. Great idea and execution!

I have started using this since yesterday and wondering if there is a way to specify the Max amount that can invested through this. Since I am trying out the tool and it's features, I don't want to investment large sums in the beginning till I get comfortable.

Thanks in advance, Satish

jgillick commented 10 years ago

Regarding the over-limit bug. Thanks for the report, I'll try to track this down. What version of the tool are you using?

lcinvestor --version

I haven't been able to reproduce this myself, yet, but I'll keep trying.

Satish, Currently there isn't a way to set a maximum amount to invest since the tool was originally developed to passively invest all available cash in the account. I'd suggest investing your balance manually and then try the tool when the account only has the amount that you'd like it to invest.

eli88fine commented 10 years ago

I downloaded it just this last weekend, so I believe the version is the most recent. It says version 2.1.4

thanks, Eli

ghost commented 10 years ago

Cool, I'll keep trying to figure out why it's doing that for you.

snikolaev commented 10 years ago

Hi Jeremy,

I have two accounts. One invests in $25 notes. The seconds one in $125 notes. This issue happens only with the second account, investing in $125 notes. More than half of all notes gets invested in incorrect amounts. One note is $325. I have to stop auto-investing in this account as it distorts my safety posture.

I hope this helps.

Thanks, Sergey

snikolaev commented 10 years ago

Actually, the first ($25) account also gets incorrect amounts invested. A note of whooping $450 have been invested in.

jgillick commented 10 years ago

Whoa, that's not good at all. I've been trying to get to the bottom of this. It seems like Lending Clib might be ignoring the max_per_note value I'm sending. I'll let you know as soon as I have a solution. Keep the info coming, if you run into anything else that might help me piece this together.

popolr commented 10 years ago

Thanks for the earlier reply! Somehow missed it.... Since I can not control how much to invest through auto investor, I tried to use your APIs (build, start_order, execute) directly to invest the amount. I am getting error for (TypeError: 'bool' object is not subscriptable). I have never done Python programming before and wrote my first Python using the example in your code....Appreciate if you could let me know what kind of stupid mistake I am making :-)

from lendingclub import LendingClub from lendingclub.filters import SavedFilter lc = LendingClub(email='abcd@xyz.com', password='abcde') lc.authenticate() filter = lc.get_saved_filter(8351065)

filter = lc.get_saved_filters()

portfolio = lc.build_portfolio(500, min_percent=13.0,max_percent=20.0,max_per_note=100, filters=filter) print "Portfolio:", portfolio len(portfolio['loan_fractions']) loans_notes = portfolio['loan_fractions'] order = lc.start_order() order.add_batch(loans_notes) order.execute()

vpatchi-mac:LendingClubAutoInvestor vpatchi$ python submitOrder.py Portfolio: False Traceback (most recent call last): File "submitOrder.py", line 9, in len(portfolio['loan_fractions']) TypeError: 'bool' object is not subscriptable

jthermiz commented 10 years ago

Hi,

Great tool! Thanks so much for developing it. I just wanted to add to the voices that have experienced a "note amount over limit." I just experienced this bug for the first time today and have been using the tool for at least a month. $50 was contributed to 1 note and only $25 should have. I've probably made a dozen orders without experiencing this bug, so it seems rare. I'm not sure why this bug happened on this order versus the others... I will probably stop using this program till a fixed has been made just to be safe. Again, thanks for developing the tool and I will check back periodically to see if there's a fix.

ghost commented 10 years ago

@satpat The build_portfolio() function returns either a boolean False or dict value. A boolean False value means that it did not find any portfolios. If it found a portfolio it would return a dictionary object (dict). Since in your case it didn't find anything, calling portfolio['loan_fractions'] fails since you're trying to get a key on a boolean object. That code only works with dictionaries. So what you want to do is put those lines in a conditional:

from lendingclub import LendingClub
from lendingclub.filters import SavedFilter
lc = LendingClub(email='abcd@xyz.com', password='abcde')
lc.authenticate()
filter = lc.get_saved_filter(8351065)
#filter = lc.get_saved_filters()
portfolio = lc.build_portfolio(500, min_percent=13.0,max_percent=20.0,max_per_note=100, filters=filter)
print "Portfolio:", portfolio

if portfolio is not False:
    len(portfolio['loan_fractions']) 
    loans_notes = portfolio['loan_fractions']
    order = lc.start_order()
    order.add_batch(loans_notes)
    order.execute()
else:
  print 'No portfolios found at this time'

Try that and let me know.

jgillick commented 10 years ago

@whitepine, @snikolaev, @eli88fine, @satpat

I'm still trying to reproduce this bug, but haven't myself. Here are my settings

{
  "max_per_note": 25,
  "portfolio": "Reinvested",
  "filters": 7651538,
  "filter_id": 7651538,
  "min_percent": 15.0,
  "min_cash": 200,
  "max_percent": 19.5
}

The saved filter I'm using has these settings:

How are you determining that it's investing in notes above your "max per note" value? What are the values in your saved filters?

eli88fine commented 10 years ago

Hello, I think this may be a difference between our filters: Your "max_percent" is below the maximum interest rate for your allowed note grades (i.e. D and E have higher rates than that), so if there are several notes available (which seems likely given the pretty un-restrictive filtering you set up), Lending club can usually always (or I guess always always if you haven't noticed this bug yet) find more notes than [available cash]/$25 that would have an average interest rate within your filter so it just splits up your investment accordingly.

I want to invest in the highest interest rate notes possible by my filter, so I set the "max_percent" to an arbitrarily large value (I think 27 or something else above all available interest rates). Since I think I remember reading that your program worked by trying to give the highest rate possible (within "max_percent"), that it is presumably just investing however much I have available in the highest interest rate note that met my filter. I often have <$50 in cash, so it usually isn't an issue as it can only autoinvest in 1 note regardless. Other cases, there are two notes with identical interest rate that presumably are the highest that were available at the time, so autoinvest puts half of my available cash into both. But if there is a note in my filter that has higher interest than any other note, autoinvest is putting all of my available cash into it. I had hoped that what it would do would be to put $25 into the highest interest note, then another $25 into the 2nd highest interest note, etc. But as you said earlier, it appears that Lending Club may possibly be ignoring the parameter you send that limits the max_amount_per_note.

Hopefully this is helpful,

Eli

jgillick commented 10 years ago

Thank you for this. I'll increase my max percent value and see what happens. The lcinvestor is actually doing less processing that it might appear. It doesn't look for individual notes, it simply looks for portfolios with your settings (same as going to https://www.lendingclub.com/portfolio/autoInvest.action) and invests in the whole portfolio that is the highest match. I tried to keep all of the logic around breaking up your available balance into notes on the LendingClub side. The logic in lcinvestor is more around handling their unofficial APIs to do this.

Since I can get a list of notes in a returned portfolio search, I'll add some code in the next release to look through the portfolios and raise an error if LendingClub included a note above the max_per_note value.

jthermiz commented 10 years ago

My setting's are pretty restrictive so that is one possible difference between us (jgillick). However my maxPercent=20% is than the allowed highest possible interest rate loan I can invest in (D's max is 20.20%). eli88fine are suggesting I should lower my maxPercent so I can avoid the max_per_note bug?

Also, I think it's great that you are adding some code that will prevent this error from happening!

eli88fine commented 10 years ago

Hello, I don't know if lending club recently changed its API to better handle your "max_per_note" input, but yesterday I noticed for the first time that even though I had $50 available cash, the autoinvestor successfully split it into two different notes (with different interest rates) of $25 each! Maybe this was just a fluke, but I'm hoping it is a good sign. I'll let you know if more notes of >$25 happen again.

Eli

Robbo67 commented 10 years ago

I am trying to use this application to build out my portfolio (rather than reinvest small dollars). I have a LC filter that I want to use that showed 3 available notes. I set my max_note in the JSON to $200 and then modified the code in init.py so that it would start checking for portfolios with a 4x my maximum per note (total $800) size and then decrement by $200. At $800 it could not find a portfolio, but when it decremented to $600, it autoinvested all $600 in just one of the notes (instead of spreading it out between the three available notes). Output is as follows:

///--------------------------- $$$ ---------------------------\ | Welcome to the unofficial Lending Club investment tool |


Authenticated successfully!

========= Summary ========= Invest ALL available funds...

With at LEAST $400 available to invest Select a portfolio with an average interest rate between 5.0% - 20.0% Invest as much as $200 per loan note Add investments to: "Autoinvest"

Using saved filter "Base Screen -- B-C Only" (id:xxxxxx)

INFO: 2013-11-02 13:36 - Authenticated INFO: 2013-11-02 13:36 - Checking for funds to invest... INFO: 2013-11-02 13:36 - $ $ $ $ $ $ $ $ $ $ INFO: 2013-11-02 13:36 - Searching for a portfolio for $800 INFO: 2013-11-02 13:36 - Could not find any matching portfolios for $800 INFO: 2013-11-02 13:36 - Searching for a portfolio for $600 INFO: 2013-11-02 13:36 - Auto investing $600 at 12.99%... INFO: 2013-11-02 13:36 - Investment portfolio summary: 1 loan notes (B:100%) INFO: 2013-11-02 13:36 - Done

After it was executed, I checked and the other two notes were still available. Can the API retrieve the loan IDs of everything that meets the filter and then invest a fixed amount in each one directly (rather than using the buy portfolio function)?

eli88fine commented 10 years ago

Woohoo! It worked for a 2nd time! There was $100 available cash in my account and autoinvest successfully put it into 4x$25 notes instead of what it would have done previously of putting it all into 1x$100 note!

jgillick commented 10 years ago

@Robbo67 It's hard for me to answer your initial question without seeing the actual code change you made.

To your other question, the lcinvestor is only built to access the portfolio search APIs of LendingClub, not to search for individual notes. You can, however, use the LendingClub python API module to do this yourself, if you'd like. Look at the "Using Saved Filters" section in the README for an example on how to do this.

jgillick commented 10 years ago

Good news everyone! I have updated the LendingClub API module that lcinvestor uses to stop a request if it contains a note over the max per note value. It sounds like LendingClub sometimes returns these notes when returning portfolios in their API response -- even though the tool explicitly tells the site what the maximum per note should be.

Update your lending club API on your system with PIP:

sudo pip install --upgrade lendingclub==0.1.6
jgillick commented 10 years ago

Also, run the following command to make sure that pip has installed the 0.1.6 version (or newer):

$ sudo pip list | grep lendingclub

If it hasn't, you might have to uninstall lendingclub and install it again. You will not loose any of your settings.

jthermiz commented 10 years ago

jgillick: Thanks a lot for doing that!

I didn't realize that you had a separate LendingClub API module. I found your documentation on (https://python-lendingclub.readthedocs.org/en/latest/lendingclub.html) to be very helpful! I decided to make my own script instead of using the autoinvestor because what I wanted to do was simpler. I don't use build_portofolio() in my script, so the only chance I have of getting a max_per_note error, is if there is an error in the order() class (or I did something stupid in my script). I will test this out and if there is an error (that I'm confident is not because of my code), I will report it.

Best.