Closed janakrajchadha closed 4 years ago
can you please assign me?
@janakrajchadha can you discuss @abhishekgupta368's approach and then assign this issue to him?
ok
we can train data and dump it into pickle format and store for later training.
later, we will load it and use it, as per our usage
@abhishekgupta368 Your approach makes sense. However, that is only part of the solution. You'll have to modify the Docker config and set up a retraining job as well. Would you be able to pick up those things as well?
i don't know about docker? where i can study about it?
@abhishekgupta368 please refrain from sending individual statements in issue discussions. This is not a chat. Try to collate your thoughts and put them together, before sending your conversation. @janakrajchadha does it make sense to split this issue into two parts?
@abhishekgupta368 you can refer to the official documentation here. Also try setting up the project the Docker way.
@vishakha-lall We can work on the retraining part in another issue. However, persisting the ML model and the configuration in Docker should be handled in one issue.
Interested
@preeti13456 can you discuss your approach here before we assign this to you
We can use tensorflow to retain and store the train model. MlOps Model management, deployment, and monitoring for configuring and retraining the model. Can use a function or CNN fr verifying the accuracy.
@preeti13456 Can you please elaborate? I want to understand why tensorflow is specifically needed to store the trained model. MLOps is a good option for overall monitoring and retraining. As far as verifying performance metrics is concerned, I'm not quite sure how a CNN would help to verify the accuracy of a tree-based classifier. Can you elaborate on this as well?
With tensorflow we can use a trained model without having to retrain it, or pick-up training where we left off—in case the training process was interrupted. This way checkpoints created and automatically saved reduces out time. And CNN can help us in determining accuracy by creating a function epoch and including the specific error like sigmoid according to dataset.
Get Outlook for Androidhttps://aka.ms/ghei36
From: Janak Raj Chadha notifications@github.com Sent: Monday, March 16, 2020 12:47:40 PM To: vishakha-lall/MapBot MapBot@noreply.github.com Cc: preeti13456 preetijecrc99@outlook.com; Mention mention@noreply.github.com Subject: Re: [vishakha-lall/MapBot] Add functionality to persist and retrain ML model (#59)
@preeti13456https://github.com/preeti13456 Can you please elaborate? I want to understand why tensorflow is specifically needed to store the trained model. MLOps is a good option for overall monitoring and retraining. As far as verifying performance metrics is concerned, I'm not quite sure how a CNN would help to verify the accuracy of a tree-based classifier. Can you elaborate on this as well?
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/vishakha-lall/MapBot/issues/59#issuecomment-599380911, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AITUUQ2DEXGIAGD4OJG2ZPDRHXHBJANCNFSM4LCMORVA.
@preeti13456 I understand that tensorflow offers these capabilities as a library. Our current system is not using tensorflow models, is it possible to store models created using other libraries with tensorflow functions? Since this application is being containerized using docker, does tensorflow have implicit capabilities which help with persisting model states and re-training as necessary? If yes, can you please share the sources which support your suggestion?
I'm not sure if you've gone through the current code, I'm not aware of a method of using CNN to determine the accuracy of models unless the model itself is a CNN which we have neither discussed as a possible model for text classification needs nor as part of this issue.
sure
Sent from Mailhttps://go.microsoft.com/fwlink/?LinkId=550986 for Windows 10
From: Vishakha Lall notifications@github.com Sent: Friday, March 13, 2020 8:31:12 AM To: vishakha-lall/MapBot MapBot@noreply.github.com Cc: preeti13456 preetijecrc99@outlook.com; Mention mention@noreply.github.com Subject: Re: [vishakha-lall/MapBot] Add functionality to persist and retrain ML model (#59)
@preeti13456https://github.com/preeti13456 can you discuss your approach here before we assign this to you
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/vishakha-lall/MapBot/issues/59#issuecomment-598524956, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AITUUQ36YXUM7IASYYC6R6TRHGOXRANCNFSM4LCMORVA.
@preeti13456 what is the progress on this issue? When will you be ready to raise a PR?
Yes
Sent from Mailhttps://go.microsoft.com/fwlink/?LinkId=550986 for Windows 10
From: Vishakha Lall notifications@github.com Sent: Friday, March 20, 2020 11:24:13 AM To: vishakha-lall/MapBot MapBot@noreply.github.com Cc: preeti13456 preetijecrc99@outlook.com; Mention mention@noreply.github.com Subject: Re: [vishakha-lall/MapBot] Add functionality to persist and retrain ML model (#59)
@preeti13456https://github.com/preeti13456 what is the progress on this issue? When will you be ready to raise a PR?
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/vishakha-lall/MapBot/issues/59#issuecomment-601547978, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AITUUQYLPNYXJMCTWZBSP6TRIMAILANCNFSM4LCMORVA.
@preeti13456 I'm not receiving updates from your end. I'm unassigning this issue.
NO Wait I have been busy today push the PR tomorrow fr sure.
Sent from Mailhttps://go.microsoft.com/fwlink/?LinkId=550986 for Windows 10
From: Vishakha Lallmailto:notifications@github.com Sent: Friday, March 20, 2020 8:01 PM To: vishakha-lall/MapBotmailto:MapBot@noreply.github.com Cc: preeti13456mailto:preetijecrc99@outlook.com; Mentionmailto:mention@noreply.github.com Subject: Re: [vishakha-lall/MapBot] Add functionality to persist and retrain ML model (#59)
@preeti13456https://github.com/preeti13456 I'm not receiving updates from your end. I'm unassigning this issue.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/vishakha-lall/MapBot/issues/59#issuecomment-601730578, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AITUUQ6MARDSYCFUGUEUVODRIN43VANCNFSM4LCMORVA.
The model needs to be saved in a file (pkl, h5 etc.) from which it can be read again whenever the application is restarted instead of complete retraining (not a major issue as of now but can become one once the dataset size increases significantly).
I found a pretty simple way of how a model can be persisted across various runtimes. Persisting it in docker wouldn't be much of trouble either.
Also, the model needs to be retrained from time to time, depending on either the number of times the bot wasn't able to respond correctly since the last run (a counter would be required for this) or a timed job (every week or two weeks).
Implementing regularly timed retraining is also achievable where the program would check for the last modified time of the pickle
file and compare it to the present time during the program runtime. If the time difference exceeds a threshold, we can go for retraining and rewriting(?) the pickle
file.
@janakrajchadha I'd be willing to work on this.
Requirement Currently, the ML model is trained from scratch whenever the application is setup. We need a way to persist the trained state of the model and retrain it based on certain conditions.
Pre-requisite Intermediate knowledge of Python Elementary understanding of Docker
Dependencies
57, since the logic to pick the model to persist will come from the previous issue
Description As the database state is persisted (issue #14), we need to persist the state of the trained classifier as well. The model needs to be saved in a file (pkl, h5 etc.) from which it can be read again whenever the application is restarted instead of complete retraining (not a major issue as of now but can become one once the dataset size increases significantly). Also, the model needs to be retrained from time to time, depending on either the number of times the bot wasn't able to respond correctly since the last run (a counter would be required for this) or a timed job (every week or two weeks). I'd suggest that we include both ways in the system and then pick one based on which method seems more appropriate to use for a particular scenario.