Squad is incomplete thrown

Command

airsenal_run_pipeline --num_thread 10 --weeks_ahead 3

Environment: Linux Fedora, python venv, 3.7

$ which airsenal_run_pipeline
~/Green/AIrsenal/venv3.7/bin/airsenal_run_pipeline

$ python --version
Python 3.7.11

Error stack trace

 File "/usr/lib64/python3.7/multiprocessing/process.py", line 297, in _bootstrap██████████████████████████▊                                                                        | 54.28571428571421/100 [01:45<00:42,  1.08it/s]
    self.run()1:  93%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▍          | 93.33333333333334/100 [01:45<00:00, 19.25it/s]
  File "/usr/lib64/python3.7/multiprocessing/process.py", line 99, in run██████████████████████████████████████████████████████████████████████████████████████████████████████████▉| 99.99999999999976/100 [01:16<00:00,  1.05it/s]
    self._target(*self._args, **self._kwargs)████████████████████████████████████████████████████████████████████████████████████████████████████████▉                              | 80.95238095238078/100 [01:44<00:00, 19.12it/s]
  File "/home/xxx/Green/AIrsenal/venv3.7/lib/python3.7/site-packages/airsenal/scripts/fill_transfersuggestion_table.py", line 188, in optimize███████████████████████████████▉| 99.99999999999976/100 [01:26<00:00,  2.11it/s]
    (updater, increment, pid),█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 100.00000000000001/100 [00:51<00:00,  2.21it/s]
  File "/home/xxx/Green/AIrsenal/venv3.7/lib/python3.7/site-packages/airsenal/framework/optimization_transfers.py", line 359, in make_best_transfers█████████████            | 92.38095238095217/100 [01:44<00:00, 357.80it/s]
    update_func_and_args=update_func_and_args,███████████████████████████████████████████████████████████████████████████████████████████████████████████████                                       | 21/27 [01:44<00:29,  5.00s/it]
  File "/home/xxx/Green/AIrsenal/venv3.7/lib/python3.7/site-packages/airsenal/framework/optimization_transfers.py", line 87, in make_optimum_single_transfer
    gw, tag
  File "/home/xxx/Green/AIrsenal/venv3.7/lib/python3.7/site-packages/airsenal/framework/squad.py", line 361, in get_expected_points
    self.optimize_lineup(gameweek, tag)
  File "/home/xxx/Green/AIrsenal/venv3.7/lib/python3.7/site-packages/airsenal/framework/squad.py", line 348, in optimize_lineup
    raise RuntimeError("Squad is incomplete")

Vars

FPL_TEAM_ID = 5549784
FPL_LEAGUE_ID = 121

Hi @pentatonicfunk , sorry to hear about this error and apologies for not replying before last weekend's deadline... I'm not 100% sure what could have happened, but I think incomplete squads can happen in the optimization if the estimate of a player's "sell price" is wrong, which means that it thinks it can't afford to add the players it wants to. This can happen if the initial database is filled one day and then the prices change, but I'm a bit surprised to see it when running the whole pipeline, as all information should be self-consistent.

In the develop branch, there is a new feature that asks users if they would like to login to the FPL API when running optimization (or the full pipeline). This means that it will get "buy price" and "sell price" for players in your squad directly from the API, which should then always be correct. I would be fairly (but not 100%) confident that this would fix this issue... We should merge develop into main in the near future, so everyone will get this feature by default, but if you'd like to test it out on develop, we'd be grateful for any feedback!

So for the last week workaround, i am able to get results when --weeks-ahead less than 3. both --weeks_ahead 1 and --weeks_ahead 2 yield a results. tried --weeks_ahead 4 but its just somehow take too much to finish on my laptop, so i just quit it and use --weeks_ahead 2

However, this week, first try of using --weeks_ahead 3 it returns results, no error!!. i guess it was CR7 effect .... :smile: jokes aside, means i can't replicate my own issue anymore. @nbarlowATI, you mentioning "sell price" it sounds made sense to me, looking at the codes, thats the only plausible reason the squad unable to be completed.

Anyway, out of curiosity, check-ed out develop branch and run it with airsenal_run_pipeline --num_thread 12 --weeks_ahead 3 --clean But i didn't get prompt for this:

there is a new feature that asks users if they would like to login to the FPL API when running optimization (or the full pipeline) not in the flag either
--num_thread INTEGER   No. of threads to use for pipeline run
--weeks_ahead INTEGER  No of weeks to use for pipeline run
--fpl_team_id INTEGER  fpl team id for pipeline run
--clean                If set, delete and recreate the AIrsenal database
--apply_transfers      If set, go ahead and make the transfers via the API.
--help                 Show this message and exit.
or i misunderstood this ? Though i have FPL_LOGIN and FPL_PASSWORD, maybe because of that ?

Result wise, both develop and main branch give same recommendations, only different in Vice Captain selection.

Hi @pentatonicfunk , yep, that explains it - if you have FPL_LOGIN and FPL_PASSWORD, it will use those rather than prompting you to login. (Though thinking about it now, I guess it should, if you request a team_id that is different to what's in the FPL_TEAM_ID file.... But that's a fairly rare case, for another time :) )

Anyway, great that it's working now!
There is another new change that could account for the different vice-captain recommendation in develop - we now make use of the fact we have a conjugate prior for the multinomial PDF in the player model, so calculate the score/assist/neither probabilities directly rather than running MCMC. This change should make prediction quicker to run, and does seem to lead to a small boost to the predicted scores of the top players.

alan-turing-institute / AIrsenal

Squad is incomplete thrown #421