numerai / numerai-cli

Automated submission workflows in the cloud.
MIT License
91 stars 28 forks source link

Example applications in numerai cli no longer compatible with the daily tournaments. #58

Closed Raynos closed 1 year ago

Raynos commented 1 year ago

It looks like the code in the examples uses a deprecated method and the V1 dataset instead of the V4 dataset

image

When I try to test the example tournament python3 code I get


Task status: PENDING. Waiting for log file to be created............
Task status: PENDING. Log file created: ecs/numerai-raynos/9802ff0a41c94766b7b3ad2402fe859e
[ecs/numerai-raynos/9802ff0a41c94766b7b3ad2402fe859e] 2023-01-31 17:29:30.005000: 2023-01-31 22:29:30,005 INFO root: downloading tournament data files
[ecs/numerai-raynos/9802ff0a41c94766b7b3ad2402fe859e] 2023-01-31 17:29:34.225000: 2023-01-31 22:29:34,225 INFO numerapi.utils: starting download
./numerai_dataset_410.zip: 491MB [00:11, 41.5MB/s]                            00: 
[ecs/numerai-raynos/9802ff0a41c94766b7b3ad2402fe859e] 2023-01-31 17:29:46.050000: 2023-01-31 22:29:46,050 INFO numerapi.base_api: unzipping file...
[ecs/numerai-raynos/9802ff0a41c94766b7b3ad2402fe859e] 2023-01-31 17:30:12.305000: 2023-01-31 22:30:12,305 INFO root: extracting features and targets
[ecs/numerai-raynos/9802ff0a41c94766b7b3ad2402fe859e] 2023-01-31 17:30:12.665000: 2023-01-31 22:30:12,665 INFO root: training model
[ecs/numerai-raynos/9802ff0a41c94766b7b3ad2402fe859e] 2023-01-31 17:30:28.582000: 2023-01-31 22:30:28,582 INFO root: saving model
[ecs/numerai-raynos/9802ff0a41c94766b7b3ad2402fe859e] 2023-01-31 17:30:28.640000: 2023-01-31 22:30:28,639 INFO root: reading prediction data

[ecs/numerai-raynos/9802ff0a41c94766b7b3ad2402fe859e] 2023-01-31 17:31:22.408000: 2023-01-31 22:31:22,408 INFO root: extracting features
[ecs/numerai-raynos/9802ff0a41c94766b7b3ad2402fe859e] 2023-01-31 17:31:24.425000: 2023-01-31 22:31:24,424 INFO root: generating predictions
[ecs/numerai-raynos/9802ff0a41c94766b7b3ad2402fe859e] 2023-01-31 17:31:25.378000: 2023-01-31 22:31:25,378 INFO root: writing predictions to file
[ecs/numerai-raynos/9802ff0a41c94766b7b3ad2402fe859e] 2023-01-31 17:31:30.425000: 2023-01-31 22:31:30,425 INFO root: submitting
[ecs/numerai-raynos/9802ff0a41c94766b7b3ad2402fe859e] 2023-01-31 17:31:30.425000: 2023-01-31 22:31:30,425 INFO numerapi.base_api: uploading predictions...
[ecs/numerai-raynos/9802ff0a41c94766b7b3ad2402fe859e] 2023-01-31 17:31:36.860000: 2023-01-31 22:31:36,860 ERROR numerapi.base_api: You must provide predictions for the current live IDs. Make sure you are using the latest live data.
[ecs/numerai-raynos/9802ff0a41c94766b7b3ad2402fe859e] 2023-01-31 17:31:36.860000: Traceback (most recent call last):
[ecs/numerai-raynos/9802ff0a41c94766b7b3ad2402fe859e] 2023-01-31 17:31:36.860000:   File "//./predict.py", line 119, in <module>
[ecs/numerai-raynos/9802ff0a41c94766b7b3ad2402fe859e] 2023-01-31 17:31:36.860000:     main()
[ecs/numerai-raynos/9802ff0a41c94766b7b3ad2402fe859e] 2023-01-31 17:31:36.860000:   File "//./predict.py", line 115, in main
[ecs/numerai-raynos/9802ff0a41c94766b7b3ad2402fe859e] 2023-01-31 17:31:36.860000:     submit(predictions, predict_output_path, MODEL_ID)
[ecs/numerai-raynos/9802ff0a41c94766b7b3ad2402fe859e] 2023-01-31 17:31:36.860000:   File "//./predict.py", line 106, in submit
[ecs/numerai-raynos/9802ff0a41c94766b7b3ad2402fe859e] 2023-01-31 17:31:36.860000:     napi.upload_predictions(predict_output_path, model_id=model_id)
[ecs/numerai-raynos/9802ff0a41c94766b7b3ad2402fe859e] 2023-01-31 17:31:36.860000:   File "/usr/local/lib/python3.9/site-packages/numerapi/numerapi.py", line 618, in upload_predictions
[ecs/numerai-raynos/9802ff0a41c94766b7b3ad2402fe859e] 2023-01-31 17:31:36.860000:     create = self.raw_query(create_query, arguments, authorization=True)
[ecs/numerai-raynos/9802ff0a41c94766b7b3ad2402fe859e] 2023-01-31 17:31:36.860000:   File "/usr/local/lib/python3.9/site-packages/numerapi/base_api.py", line 126, in raw_query
[ecs/numerai-raynos/9802ff0a41c94766b7b3ad2402fe859e] 2023-01-31 17:31:36.860000:     raise ValueError(err)
[ecs/numerai-raynos/9802ff0a41c94766b7b3ad2402fe859e] 2023-01-31 17:31:36.860000: ValueError: You must provide predictions for the current live IDs. Make sure you are using the latest live data.

Task is stopping...
checking for submission...
No submission found for current round, test failed
ndharasz commented 1 year ago

The example has been updated in v0.3.5 to use modern data, feature sets, and APIs.

Raynos commented 1 year ago

Thanks, that's a great help.