Closed hideojoho closed 2 years ago
Thanks for raising the issue! It should be fixed in the code right now. Please let us know if it still happens.
@QingyaoAi , @anhtran1010 Thank you for your support. The original error seems solved but produced another one as follows.
(venv) me@DESKTOP-FCVGD1U:~/workspace/github/ULTRA_pytorch$ python main.py --max_train_iteration=10
Reading data in ./tests/data/
Read data from ./tests/data//train in ULTRA format.
Feature reading finish.
List reading finish.
Label reading finish.
Remove 7 invalid queries.
Data reading finish!
Finished reading 20 queries with lists.
Read data from ./tests/data//valid in ULTRA format.
Feature reading finish.
List reading finish.
Label reading finish.
Remove 3 invalid queries.
Data reading finish!
Finished reading 6 queries with lists.
Train Rank list size 9
Valid Rank list size 9
Users can only see the top 9 documents for each query in training.
Creating model...
Build DLA
Loss Function is softmax_loss
Created model with fresh parameters.
Create simluated clicks feed
Create direct label feed with list size 9 with feature size 136
max_train_iter: 10
Loss 4.440492 at Global Step 0:
Loss 6.424829 at Global Step 1:
Loss 4.830202 at Global Step 2:
Loss 10.693382 at Global Step 3:
Loss 5.427406 at Global Step 4:
Loss 5.200952 at Global Step 5:
Loss 4.584223 at Global Step 6:
Loss 4.241126 at Global Step 7:
Loss 4.160506 at Global Step 8:
Loss 4.166048 at Global Step 9:
Loss 4.358397 at Global Step 10:
Loss 4.206917 at Global Step 11:
Loss 4.137994 at Global Step 12:
Loss 4.561807 at Global Step 13:
Loss 4.151238 at Global Step 14:
Loss 4.385555 at Global Step 15:
Loss 4.136350 at Global Step 16:
Loss 4.306522 at Global Step 17:
Loss 4.081818 at Global Step 18:
Loss 4.461146 at Global Step 19:
Loss 4.122521 at Global Step 20:
Loss 4.049473 at Global Step 21:
Loss 4.759341 at Global Step 22:
Loss 4.263655 at Global Step 23:
Loss 4.331538 at Global Step 24:
Loss 3.980463 at Global Step 25:
Loss 4.498557 at Global Step 26:
Loss 4.095665 at Global Step 27:
Loss 4.836687 at Global Step 28:
Loss 4.038417 at Global Step 29:
Loss 4.069120 at Global Step 30:
Loss 4.188335 at Global Step 31:
Loss 4.101030 at Global Step 32:
Loss 3.871989 at Global Step 33:
Loss 4.743153 at Global Step 34:
Loss 4.104199 at Global Step 35:
Loss 4.333329 at Global Step 36:
Loss 4.276215 at Global Step 37:
Loss 4.049036 at Global Step 38:
Loss 4.262655 at Global Step 39:
Loss 4.079478 at Global Step 40:
Loss 4.245237 at Global Step 41:
Loss 4.023198 at Global Step 42:
Loss 4.118076 at Global Step 43:
Loss 4.011585 at Global Step 44:
Loss 4.106015 at Global Step 45:
Loss 4.046475 at Global Step 46:
Loss 4.240688 at Global Step 47:
Loss 3.965695 at Global Step 48:
Loss 4.201066 at Global Step 49:
global step 50 learning rate 0.0500 step-time 0.12 loss 4.4594
Traceback (most recent call last):
File "main.py", line 301, in <module>
main(argv)
File "main.py", line 296, in main
train(exp_settings)
File "main.py", line 184, in train
valid_summary = validate_model(valid_set, valid_input_feed)
File "main.py", line 176, in validate_model
_, _, summary = model.validation(input_feed)
File "/home/me/workspace/github/ULTRA_pytorch/ultra/learning_algorithm/dla.py", line 280, in validation
metric_values = ultra.utils.make_ranking_metric_fn(
File "/home/me/workspace/github/ULTRA_pytorch/ultra/utils/metrics.py", line 91, in _mean_reciprocal_rank_fn
return mean_reciprocal_rank(
File "/home/me/workspace/github/ULTRA_pytorch/ultra/utils/metrics.py", line 284, in mean_reciprocal_rank
labels, predictions, weights, topn = _prepare_and_validate_params(
File "/home/me/workspace/github/ULTRA_pytorch/ultra/utils/metrics.py", line 249, in _prepare_and_validate_params
topn = [min(n, list_size) for n in topn]
TypeError: 'int' object is not iterable
As far as I examined, def prepare_and_validate_params
in ultra/utils/metrics.py
is defined at Line 224, where topn
is expected to receive an array or similar type.
def _prepare_and_validate_params(labels, predictions, weights=None, topn=None):
"""Prepares and validates the parameters.
Args:
labels: A `Tensor` of the same shape as `predictions`. A value >= 1 means a
relevant example.
predictions: A `Tensor` with shape [batch_size, list_size]. Each value is
the ranking score of the corresponding example.
weights: A `Tensor` of the same shape of predictions or [batch_size, 1]. The
former case is per-example and the latter case is per-list.
topn: A list of cutoff for how many examples to consider for this metric.
Returns:
(labels, predictions, weights, topn) ready to be used for metric
calculation.
"""
However, the function is called at Line 283 where topn
seems an integer.
list_size = predictions.size()[-1]
labels, predictions, weights, topn = _prepare_and_validate_params(
labels, predictions, weights, list_size)
Hope this helps.
Hi,
First, thank you so much for bringing this to our attention. We have patched the bug and updated our Github repository. As you may already know, we are currently updating ULTRA_pytorch by adding more learning algorithms, ranking models as well as improving the execution time of the toolbox. As a result, bugs like this may occur even with our testing. I sincerely apologize for any inconvenience.
Feel free to reach out to us with any questions, or possible issues that need to be addressed.
Thank you so much
Anh Tran
On Thu, Oct 28, 2021 at 3:15 AM Hideo Joho @.***> wrote:
@QingyaoAi https://github.com/QingyaoAi , @anhtran1010 https://github.com/anhtran1010 Thank you for your support. The original error seems solved but produced another one as follows.
(venv) @.***:~/workspace/github/ULTRA_pytorch$ python main.py --max_train_iteration=10 Reading data in ./tests/data/ Read data from ./tests/data//train in ULTRA format. Feature reading finish. List reading finish. Label reading finish. Remove 7 invalid queries. Data reading finish! Finished reading 20 queries with lists. Read data from ./tests/data//valid in ULTRA format. Feature reading finish. List reading finish. Label reading finish. Remove 3 invalid queries. Data reading finish! Finished reading 6 queries with lists. Train Rank list size 9 Valid Rank list size 9 Users can only see the top 9 documents for each query in training. Creating model... Build DLA
Loss Function is softmax_loss Created model with fresh parameters. Create simluated clicks feed
Create direct label feed with list size 9 with feature size 136 max_train_iter: 10 Loss 4.440492 at Global Step 0: Loss 6.424829 at Global Step 1: Loss 4.830202 at Global Step 2: Loss 10.693382 at Global Step 3: Loss 5.427406 at Global Step 4: Loss 5.200952 at Global Step 5: Loss 4.584223 at Global Step 6: Loss 4.241126 at Global Step 7: Loss 4.160506 at Global Step 8: Loss 4.166048 at Global Step 9: Loss 4.358397 at Global Step 10: Loss 4.206917 at Global Step 11: Loss 4.137994 at Global Step 12: Loss 4.561807 at Global Step 13: Loss 4.151238 at Global Step 14: Loss 4.385555 at Global Step 15: Loss 4.136350 at Global Step 16: Loss 4.306522 at Global Step 17: Loss 4.081818 at Global Step 18: Loss 4.461146 at Global Step 19: Loss 4.122521 at Global Step 20: Loss 4.049473 at Global Step 21: Loss 4.759341 at Global Step 22: Loss 4.263655 at Global Step 23: Loss 4.331538 at Global Step 24: Loss 3.980463 at Global Step 25: Loss 4.498557 at Global Step 26: Loss 4.095665 at Global Step 27: Loss 4.836687 at Global Step 28: Loss 4.038417 at Global Step 29: Loss 4.069120 at Global Step 30: Loss 4.188335 at Global Step 31: Loss 4.101030 at Global Step 32: Loss 3.871989 at Global Step 33: Loss 4.743153 at Global Step 34: Loss 4.104199 at Global Step 35: Loss 4.333329 at Global Step 36: Loss 4.276215 at Global Step 37: Loss 4.049036 at Global Step 38: Loss 4.262655 at Global Step 39: Loss 4.079478 at Global Step 40: Loss 4.245237 at Global Step 41: Loss 4.023198 at Global Step 42: Loss 4.118076 at Global Step 43: Loss 4.011585 at Global Step 44: Loss 4.106015 at Global Step 45: Loss 4.046475 at Global Step 46: Loss 4.240688 at Global Step 47: Loss 3.965695 at Global Step 48: Loss 4.201066 at Global Step 49: global step 50 learning rate 0.0500 step-time 0.12 loss 4.4594 Traceback (most recent call last): File "main.py", line 301, in
main(argv) File "main.py", line 296, in main train(exp_settings) File "main.py", line 184, in train valid_summary = validate_model(valid_set, valid_input_feed) File "main.py", line 176, in validatemodel , _, summary = model.validation(input_feed) File "/home/me/workspace/github/ULTRA_pytorch/ultra/learning_algorithm/dla.py", line 280, in validation metric_values = ultra.utils.make_ranking_metric_fn( File "/home/me/workspace/github/ULTRA_pytorch/ultra/utils/metrics.py", line 91, in _mean_reciprocal_rank_fn return mean_reciprocal_rank( File "/home/me/workspace/github/ULTRA_pytorch/ultra/utils/metrics.py", line 284, in mean_reciprocal_rank labels, predictions, weights, topn = _prepare_and_validate_params( File "/home/me/workspace/github/ULTRA_pytorch/ultra/utils/metrics.py", line 249, in _prepare_and_validate_params topn = [min(n, list_size) for n in topn] TypeError: 'int' object is not iterable As far as I examined, def prepare_and_validate_params in ultra/utils/metrics.py is defined at Line 224, where topn is expected to receive an array or similar type.
def _prepare_and_validate_params(labels, predictions, weights=None, topn=None): """Prepares and validates the parameters. Args: labels: A
Tensor
of the same shape aspredictions
. A value >= 1 means a relevant example. predictions: ATensor
with shape [batch_size, list_size]. Each value is the ranking score of the corresponding example. weights: ATensor
of the same shape of predictions or [batch_size, 1]. The former case is per-example and the latter case is per-list. topn: A list of cutoff for how many examples to consider for this metric. Returns: (labels, predictions, weights, topn) ready to be used for metric calculation. """However, the function is called at Line 283 where topn seems an integer.
list_size = predictions.size()[-1] labels, predictions, weights, topn = _prepare_and_validate_params( labels, predictions, weights, list_size)
Hope this helps.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ULTR-Community/ULTRA_pytorch/issues/2#issuecomment-953662438, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGMWVLWNXUNACHKYJWRRUSLUJEICBANCNFSM5GLNTLEQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
Thanks! Both training and testing went fine.
Hi. Thanks for sharing the great project. I'm new to this and tried the first example script. Training seems fine but testing fails as follows. Any idea?
Thanks