modelfoxdotdev / modelfox

ModelFox makes it easy to train, deploy, and monitor machine learning models.
Other
1.46k stars 63 forks source link

App identifier dosent match identifier in log_prediction #87

Closed komatded closed 2 years ago

komatded commented 2 years ago

Good evening!

I recently tried to push model predictions into an application and discovered that identifiers I specified in log_prediction function did not match identifiers that appeared in the application. I use python3.9 and tangram==0.7.0.

Here are screenshots:

Python code

Снимок экрана 2022-01-11 в 23 28 51

Event in the app. Here it says that prediction identifier is the same as in python code, but actual identifier for this prediction is 3237cbe3097c87f5e73b7ce796f38be7, as stated in the its link. And if I want to use log_true_value for this prediction, I have to use 3237cbe3097c87f5e73b7ce796f38be7 but not the original id

Снимок экрана 2022-01-11 в 23 28 58
nitsky commented 2 years ago

Hi @komatded, what you see in the URL for a prediction is what we call the prediction's id, which is unique and automatically generated. It is intentionally different from the identifier for that prediction. The id is always unique, but it's possible, though discouraged, for you to log multiple predictions with the same identifier. Does that make sense?

I just tried the example in languages/python/examples/advanced which logs a single prediction and true value to the app running locally, and was able to successfully associate the true value with the prediction using the identifier. Perhaps there is something different in your setup? Can you provide some more detailed steps to reproduce your problem? Thank you! :)

komatded commented 2 years ago

Hi @nitsky, thank you for your response!

I apologize for disturbing in vain :) Looks like I found what the problem was - in my train and test datasets target's column data type was integer, so I thought that I should also pass integer as true_value in log_true_value... now when I've passed it as a string everything worked fine!

Sorry, for bothering! I guess now this issue can be closed

nitsky commented 2 years ago

@komatded if the value is an integer you should not be required to pass it as a string. Is your model doing regression or classification?

komatded commented 2 years ago

Classification; I've trained it with command tangram train --file web_train_tangram.csv --target target --output web.tangram, and as you can see target columns data type is int64

Снимок экрана 2022-01-13 в 12 03 36