Closed sangkeun00 closed 7 months ago
Since PR #87, metadata is supposed to have only one entry per data_id. However, in the presence of data duplication, which is pretty common in real-world ML, there may also be duplicate data_id in metadata. This causes an error in the below code:
data_id
https://github.com/sangkeun00/analog/blob/21a51683d0a91afc4efe0bf569c2c5c655ab528c/analog/logging/log_loader_util.py#L71
@eatpk Can you look into this bug? My guess is that we can simply remove this if statement and simply override it for data with same id.
https://github.com/sangkeun00/analog/pull/92 PTAL!
eatpk
commented
7 months ago
Since PR #87, metadata is supposed to have only one entry per
data_id. However, in the presence of data duplication, which is pretty common in real-world ML, there may also be duplicatedata_idin metadata. This causes an error in the below code:https://github.com/sangkeun00/analog/blob/21a51683d0a91afc4efe0bf569c2c5c655ab528c/analog/logging/log_loader_util.py#L71
@eatpk Can you look into this bug? My guess is that we can simply remove this if statement and simply override it for data with same id.