S-1 (B-16) Movie-Success-Rate-Prediction ( pickle file issue )

[venkat-2811]

Subject: Issue with Pickle File Compatibility During Model Deployment

Hello Sir,

I am writing to seek assistance regarding a technical issue I am facing while deploying our Movie Success Rate Prediction project. Despite following the necessary steps to create and load the machine learning models, I am encountering a compatibility error related to the pickle files. Below is a detailed description of the problem and the steps I've taken to address it.

Description of the Issue

Error Message:

ValueError: node array from the pickle has an incompatible dtype:

• expected: {'names': ['left_child', 'right_child', 'feature', 'threshold', 'impurity', 'n_node_samples', 'weighted_n_node_samples', 'missing_go_to_left'], 'formats': ['<i8', '<i8', '<i8', '<f8', '<f8', '<i8', '<f8', 'u1'], 'offsets': [0, 8, 16, 24, 32, 40, 48, 56], 'itemsize': 64}
• got : [('left_child', '<i8'), ('right_child', '<i8'), ('feature', '<i8'), ('threshold', '<f8'), ('impurity', '<f8'), ('n_node_samples', '<i8'), ('weighted_n_node_samples', '<f8')] Traceback: File "C:\path\to\your\project\env\Lib\site-packages\streamlit\runtime\scriptrunner\script_runner.py", line 600, in _run_script exec(code, module.dict) File "C:\path\to\your\project\app.py", line 20, in adaboost_model = load_model('Adaboost.pkl') ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\path\to\your\project\app.py", line 10, in load_model model = pickle.load(file) ^^^^^^^^^^^^^^^^^ File "_tree.pyx", line 867, in sklearn.tree._tree.Tree.setstate File "_tree.pyx", line 1573, in sklearn.tree._tree._check_node_ndarray

### Steps Taken to Address the Issue Environment Setup:

Installed the necessary libraries (scikit-learn, pandas, numpy) in a controlled environment (Google Colab and local machine). Ensured that both the development and deployment environments have matching library versions. Model Training and Pickle File Creation:

Trained the models (Adaboost, RandomForest, KNeighbors, LogisticRegression, NaiveBayes, SVM) in Google Colab. Saved the trained models as pickle files using the same environment to ensure compatibility. Downloaded the pickle files from Colab and placed them in the project directory for deployment. Loading Models in Streamlit Application:

Implemented the model loading logic in app.py using pickle. Ensured the application code attempts to load the correct pickle files and make predictions based on user input. Despite these efforts, the application throws a dtype incompatibility error when trying to load the pickled models. This suggests a possible mismatch in the data types expected by the model's Tree structure during loading.

**Request for Assistance :

-Assistance in checking the files and their Content. -Debugging Tips:**

### ScreenShots for reference : -ERRORS and Files included in My Project Repo:

### -How I Download My Pickle Files :

Files to be inspected :

https://drive.google.com/drive/folders/1h62auQuxtf2_Tq4WYdsplfC10eRD6lf3

Any insights or suggestions on debugging dtype incompatibility issues in scikit-learn models? Your expertise and advice on resolving this issue would be greatly appreciated. Thank you for your time and support.

[charann29]

Loading Problem with Pickle File

It appears that there is an issue with the pickle file – it may not exist. If you have already created a pickle file from your Colab session and added it but are still encountering issues (possibly with the path), there are a few things to consider:

1. Path Issues:

   • Double-check the path to ensure it is correct.
   • Verify that the pickle file exists at the specified path.

2. Model Training:

   • The issue might be related to the model training process. Recheck each line in the training part of your code.
   • Pay special attention to the library versions you are using. Ensuring compatibility can resolve many issues. GPT can assist with verifying library versions and compatibility.

If you are unable to fix the problem by the end of the day, consider finding a similar project on GitHub and replicating it. This can provide insights and potentially resolve the issue.

Here is a good starting point for finding similar projects: GitHub Machine Learning Projects

[venkat-2811]

Hello Sir,

I appreciate your suggestion regarding the pickle file issue. After some investigation, it seems the problem might have been related to path inaccuracies or compatibility issues with the library versions used during the model training process.

To address this, I double-checked the path to ensure it was correct and confirmed that the pickle file existed at the specified location. I also revisited the model training code, verifying each line to ensure accuracy and compatibility with the library versions in use.

Despite these efforts, the issue persisted. Therefore, I decided to explore an alternative approach to deploying my model. Instead of focusing solely on resolving the pickle file issue, I searched for similar projects on GitHub and different approach to deploy it from GPT.

By following the methodology ,I was able to get code and can deploy my model. This approach resolved the immediate issue.

Thank you for your support and advice.

[charann29]

Try to find a alternative code repository if there are any in the github. If are there other options let’s figure out a way. Please check the PRs raised one of your classmates done the same project but he could able to do it.

On Fri, 21 Jun 2024 at 5:36 AM, Sunkara Venkata Karthik Sai < @.***> wrote:

Hello Sir,

I appreciate your suggestion regarding the pickle file issue. After some investigation, it seems the problem might have been related to path inaccuracies or compatibility issues with the library versions used during the model training process.

To address this, I double-checked the path to ensure it was correct and confirmed that the pickle file existed at the specified location. I also revisited the model training code, verifying each line to ensure accuracy and compatibility with the library versions in use.

Despite these efforts, the issue persisted. Therefore, I decided to explore an alternative approach to deploying my model. Instead of focusing solely on resolving the pickle file issue, I searched for similar projects on GitHub and different approach to deploy it from GPT.

By following the methodology ,I was able to get code and can deploy my model. This approach resolved the immediate issue.

Thank you for your support and advice.

— Reply to this email directly, view it on GitHub https://github.com/charann29/cmr_opensource/issues/410#issuecomment-2181752624, or unsubscribe https://github.com/notifications/unsubscribe-auth/AVXEZGL6UDPNE4DEONA4QDDZINVB5AVCNFSM6AAAAABJUBV4BOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCOBRG42TENRSGQ . You are receiving this because you commented.Message ID: @.***>

[venkat-2811]

Hello Sir,

I found an alternative approach to generate code and it worked. it is deploying successfully using streamlit and will surely raise PR soon sir.

Thank you for the valuable time and will be waiting for you to review my pr sir.