Heroku deployment of NLP model Nagisa Tokenizer showing error

[Pranjal-bisht]

Hi, I deployed my Flask App ( NLP model ) on Heroku. I was basically a price prediction model where some columns were in Japanese where I applied NLP + Nagisa Library for tokenization and some columns were numerical data. I pickled vectorizers and the model and Finally added them to my Flask API. But after deployment when I added the values in the frontend and clicked on Predict button, the result is not getting displayed. This is the exact error I am facing. The exact code of Tokenizer_jp is : def tokenize_jp(doc): doc = nagisa.tagging(doc) return doc.words

I am not able to figure out how to fix this? does Nagisa work in Heroku deployment? PS: I am not really sure if the problem is with Heroku or Nagisa, please help me with this.

[taishi-i]

Hi, @Pranjal-bisht. Thank you for using nagisa! First of all, I have confirmed that nagisa works on Heroku.

As far as the error is concerned, you are getting a Heroku memory overflow error. Nagisa uses about 270 MiB of memory. If you are using the free Heroku, then only 500MiB of memory is available. So, to avoid errors, it is necessary to conserve memory usage with other libraries.

Do you use libraries other than nagisa on Heroku? Let me first check your situation. Thanks

[Pranjal-bisht]

Hi @taishi-i San, Thank you for your reply. Yes I am using following libraries in my Flask Rest API.

from random import shuffle
from flask import Flask,render_template,request
import pandas as pd
from scipy.sparse import csr_matrix, hstack
import pickle
import os
import nagisa
import xgboost as xgb

Also I am using some saved pickle files in my directory which consist of my model and encoding pickle file for tokenizers and countvectorizers.

FYI : Here is some more info of Heroku log :

PS: The Applications Runs fine on my local host. Problem is being caused in Heroku. Thankyou

[taishi-i]

Hi @Pranjal-bisht. Thank you for the Python libraries' information. I used the following code to check memory usage. As a result, the Python libraries in your configuration use at least 345.8 MiB of memory. In addition, loading xgb models and pickle files will use additional memory.

from memory_profiler import profile

@profile
def main():
    import os
    import pickle

    import nagisa
    import pandas as pd
    import xgboost as xgb

    from random import shuffle
    from flask import Flask, render_template, request
    from scipy.sparse import csr_matrix, hstack

    text = "これはサンプルの文です。"
    words = nagisa.tagging(text)
    print(words)

if __name__ == '__main__':
    main()

Line #    Mem usage    Increment  Occurrences   Line Contents
=============================================================
     4     39.1 MiB     39.1 MiB           1   @profile
     5                                         def main():
     6     39.1 MiB      0.0 MiB           1       import os
     7     39.1 MiB      0.0 MiB           1       import pickle
     8                                         
     9    270.2 MiB    231.1 MiB           1       import nagisa
    10    293.5 MiB     23.2 MiB           1       import pandas as pd
    11    339.4 MiB     45.9 MiB           1       import xgboost as xgb
    12                                         
    13    339.4 MiB      0.0 MiB           1       from random import shuffle
    14    345.3 MiB      5.9 MiB           1       from flask import Flask, render_template, request
    15    345.3 MiB      0.0 MiB           1       from scipy.sparse import csr_matrix, hstack
    16                                         
    17    345.3 MiB      0.0 MiB           1       text = "これはサンプルの文です。"
    18    345.3 MiB      0.0 MiB           1       words = nagisa.tagging(text)
    19    345.8 MiB      0.5 MiB           1       print(words)

I think the 512MB of memory in the Free plan of heroku will not be enough. Your local environment has enough memory, so it works fine.

This is not a problem with the nagisa library itself. It is a problem of how memory is used in Heroku. As a solution, the free plan Heroku allows you to use two processes. So how about separating the API for tokenizing and the API for xgb models to separate the memory usage?

[Pranjal-bisht]

Hi @taishi-i Thank you for your response. Sure I can try making API separately. Or, Shall I buy Heroku premium for this maybe? if, Yes Which one would you suggest ? thanks once again.

[taishi-i]

Hi @Pranjal-bisht. No problem! I think it would be better to try several methods before buying Premium. I think the purchase of Premium is the last resort. There must be a better way.

For example, you might consider using other Japanese tokenizers (SudachiPy, Janome, fugashi). I have checked and it appears that these libraries use less memory than nagisa. Please check my repository (awesome-japanese-nlp-resources) where I maintain a list of Japanese tokenizers.

If you are going to use a model trained by nagisa for tokenizing, it would be better to try a method to separate the APIs first.

[Pranjal-bisht]

Hi @taishi-i . I got your point. But I am thinking to buy premium because I will scale this application for the company after some time. I will train different models and build their pickle files and use them in my Flask API. So, in parallel there will be several models, which will shoot up memory space, So, Even if we optimize Time and space complexity now, we won't be sure of how much memory the app will take after scaling. So I am thinking to buy premium and then test the above method as you suggested. Thankyou once again.

[taishi-i]

Hi @Pranjal-bisht. I understand your situation. Once again, I think that if you adjust the memory management in Heroku properly, the program will work without any problems. I hope it works well. If you encounter any other issues, please let me know. I think I can help you. Thanks!

[Pranjal-bisht]

ok @taishi-i San. We can try it out. So you said to build API seperately. Can you suggest how can build it? Actually I am new to flask. Here Is the code snipped of the flask ( app.py file ).

from flask import Flask,render_template,request
import pandas as pd
from scipy.sparse import csr_matrix, hstack
import pickle
import os
import nagisa

app = Flask(__name__)

picFolder = os.path.join('static','icons')
app.config['UPLOAD_FOLDER'] = picFolder
def tokenize_jp(doc):
    doc = nagisa.tagging(doc)
    return doc.words

@app.route('/')
def index():
    nohara1 = os.path.join(app.config['UPLOAD_FOLDER'],'nohara1.png')
    photo = os.path.join(app.config['UPLOAD_FOLDER'],'photo.jpg')
    return render_template('index.html', user_image = [nohara1,photo])

@app.route('/predict',methods=['POST'])
def predict():
    if request.method == 'POST':
        nohara1 = os.path.join(app.config['UPLOAD_FOLDER'],'nohara1.png')
        photo = os.path.join(app.config['UPLOAD_FOLDER'],'photo.jpg')
        list_of_data = [x for x in request.form.values()]

        X =pd.DataFrame([list_of_data], columns=["buyers name","branch name","person in charge","construction site","Product name","specification","quantity","NewOrOldsite"])
        # Takes in a document, returns the list of words:
        trans_buyers_name = pickle.load(open("en_buyers_name.pkl",'rb'))
        trans_val_buyers_name = trans_buyers_name.transform(X['buyers name'].values)

        trans_branch_name = pickle.load(open("en_branch_name.pkl",'rb'))
        trans_val_branch_name = trans_branch_name.transform(X['branch name'].values)

        trans_person_in_charge = pickle.load(open("en_person_in_charge.pkl",'rb'))
        trans_val_person_in_charge = trans_person_in_charge.transform(X['person in charge'].values)
        print("3",trans_val_person_in_charge.shape)

        test_trans = csr_matrix(pd.get_dummies(X[['quantity','NewOrOldsite']], sparse=True).values)

        trans_construction_site = pickle.load(open("en_construction_site.pkl",'rb'))
        trans_val_construction_site = trans_construction_site.transform(X['construction site'].values)

        trans_Product_name = pickle.load(open("en_Product_name.pkl",'rb'))
        trans_val_Product_name = trans_Product_name.transform(X['Product name'].values)

        trans_specification = pickle.load(open("en_specification.pkl",'rb'))
        trans_val_specification = trans_specification.transform(X['specification'].values)

        X = hstack((trans_val_buyers_name,trans_val_branch_name, trans_val_person_in_charge,test_trans, trans_val_construction_site,
                            trans_val_Product_name, trans_val_specification)).tocsr().astype('float32')
        df = X
        model = pickle.load(open("xgb_model.pkl",'rb'))
        prediction = model.predict(df)
        print(prediction[0])

        return render_template('index.html', user_image = [nohara1,photo],price='Estimated Price : ¥ {:.2f}'.format(prediction[0]))

if __name__ == '__main__':
    # app.run(host='0.0.0.0',port=8080)
    app.run(debug=True)

I am taking data from frontend in the list_of_data and converting it to the dataframe. Then I am applying the tokenizers which I had pickled Earlier and In the API, I am opening and fitting to the dataframe and finally predicting using the model.

FYI : This is the code for count vectorizer using Nagisa tokenizer.

cnt_vect_specification = CountVectorizer(tokenizer=tokenize_jp, ngram_range = (1, 2), max_features = 1000000)
cnt_vect_specification.fit(X['specification'].values)

and I am using this tokenizer for last three columns => [ 'Construction site','Product Name','specifications']

import nagisa
# Takes in a document, returns the list of words
def tokenize_jp(doc):
    doc = nagisa.tagging(doc)
    return doc.words

[taishi-i]

Hi @Pranjal-bisht. If I create an API with only Heroku's free plan, do the following: Note that this is a way to sacrifice simplicity. This will create two API servers (the feature_extraction API and the prediction API) on Heroku. Also, since it is a pseudo-code, it cannot be operated as it is.

First, we use nagisa to create an API to extract XGB features. You don't need to import xgm models here. This is the feature_extraction API.

from flask import Flask, jsonify
import pandas as pd
from scipy.sparse import csr_matrix, hstack
import pickle
import os
import nagisa

app = Flask(__name__)

@app.route('/feature_extraction', methods=['POST'])
def feature_extraction():
    if request.method == 'POST':
        nohara1 = os.path.join(app.config['UPLOAD_FOLDER'],'nohara1.png')
        photo = os.path.join(app.config['UPLOAD_FOLDER'],'photo.jpg')
        list_of_data = [x for x in request.form.values()]

        X =pd.DataFrame([list_of_data], columns=["buyers name","branch name","person in charge","construction site","Product name","specification","quantity","NewOrOldsite"])
        # Takes in a document, returns the list of words:
        trans_buyers_name = pickle.load(open("en_buyers_name.pkl",'rb'))
        trans_val_buyers_name = trans_buyers_name.transform(X['buyers name'].values)

        trans_branch_name = pickle.load(open("en_branch_name.pkl",'rb'))
        trans_val_branch_name = trans_branch_name.transform(X['branch name'].values)

        trans_person_in_charge = pickle.load(open("en_person_in_charge.pkl",'rb'))
        trans_val_person_in_charge = trans_person_in_charge.transform(X['person in charge'].values)
        print("3",trans_val_person_in_charge.shape)

        test_trans = csr_matrix(pd.get_dummies(X[['quantity','NewOrOldsite']], sparse=True).values)

        trans_construction_site = pickle.load(open("en_construction_site.pkl",'rb'))
        trans_val_construction_site = trans_construction_site.transform(X['construction site'].values)

        trans_Product_name = pickle.load(open("en_Product_name.pkl",'rb'))
        trans_val_Product_name = trans_Product_name.transform(X['Product name'].values)

        trans_specification = pickle.load(open("en_specification.pkl",'rb'))
        trans_val_specification = trans_specification.transform(X['specification'].values)

        X = hstack((trans_val_buyers_name,trans_val_branch_name, trans_val_person_in_charge,test_trans, trans_val_construction_site,
                            trans_val_Product_name, trans_val_specification)).tocsr().astype('float32')

    X = {"feature": X}
    return jsonify(X)

if __name__ == '__main__':
    # app.run(host='0.0.0.0',port=8080)
    app.run(debug=True)

Next, we use xgb to make a prediction. You don't need import nagisa here. Here, the requests library is used to call the API (the feature_extraction API) created above.

import requests

from flask import Flask,render_template,request
import pandas as pd
from scipy.sparse import csr_matrix, hstack
import pickle
import os

@app.route('/predict',methods=['POST'])
def predict():
    if request.method == 'POST':

        # Please change your Heroku's API URL
        URL = "https://APPNAME.herokuapp.com/feature_extraction"
        X = requests.post(URL)
    df = X["feature"]

        model = pickle.load(open("xgb_model.pkl",'rb'))
        prediction = model.predict(df)
        print(prediction[0])

        return render_template('index.html', user_image = [nohara1,photo],price='Estimated Price : ¥ {:.2f}'.format(prediction[0]))

if __name__ == '__main__':
    # app.run(host='0.0.0.0',port=8080)
    app.run(debug=True)

Separating API in this way reduces memory consumption.

[taishi-i]

Hi @Pranjal-bisht. It also presents a simpler solution. It does not require separating the API. If you can train the models again, try using the following library (Janome) instead of nagisa for tokenizing Japanese sentences.

Install the library.

pip install janome

The following function is used as tokenize_jp.

from janome.tokenizer import Tokenizer                                         

t = Tokenizer()                                                                

def tokenize_jp(doc):                                                          
    words = list(t.tokenize(doc, wakati=True))                                 
    return words

[Pranjal-bisht]

Hi @taishi-i Thankyou for such a detailed explaination. I tried the second approach. It has resolved the memory timeout error but still the application is crashing after clicking predict on Heroku ( Runs fine locally ). Here is the logs for Heroku :

2022-07-05T04:55:51.000000+00:00 app[api]: Build succeeded
2022-07-05T04:55:57.323255+00:00 app[web.1]: 3 (1, 77)
2022-07-05T04:55:57.328028+00:00 app[web.1]: [2022-07-05 04:55:57,327] ERROR in app: Exception on /predict [POST]
2022-07-05T04:55:57.328030+00:00 app[web.1]: Traceback (most recent call last):
2022-07-05T04:55:57.328031+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.10/site-packages/flask/app.py", line 2077, in wsgi_app
2022-07-05T04:55:57.328031+00:00 app[web.1]: response = self.full_dispatch_request()
2022-07-05T04:55:57.328032+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.10/site-packages/flask/app.py", line 1525, in full_dispatch_request
2022-07-05T04:55:57.328032+00:00 app[web.1]: rv = self.handle_user_exception(e)
2022-07-05T04:55:57.328032+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.10/site-packages/flask/app.py", line 1523, in full_dispatch_request
2022-07-05T04:55:57.328033+00:00 app[web.1]: rv = self.dispatch_request()
2022-07-05T04:55:57.328033+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.10/site-packages/flask/app.py", line 1509, in dispatch_request
2022-07-05T04:55:57.328033+00:00 app[web.1]: return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args)
2022-07-05T04:55:57.328033+00:00 app[web.1]: File "/app/app.py", line 51, in predict
2022-07-05T04:55:57.328034+00:00 app[web.1]: trans_construction_site = pickle.load(open("en_construction_site.pkl",'rb'))
2022-07-05T04:55:57.328034+00:00 app[web.1]: AttributeError: Can't get attribute 'tokenize_jp' on <module '__main__' from '/app/.heroku/python/bin/gunicorn'>
2022-07-05T04:55:57.328590+00:00 app[web.1]: 10.1.92.72 - - [05/Jul/2022:04:55:57 +0000] "POST /predict HTTP/1.1" 500 265 "https://nohara-pricing-app.herokuapp.com/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.5060.66 Safari/537.36"
2022-07-05T04:55:57.329121+00:00 heroku[router]: at=info method=POST path="/predict" host=nohara-pricing-app.herokuapp.com request_id=6acdec18-bba5-4977-bde9-735649be40b4 fwd="49.36.217.231" dyno=web.1 connect=0ms service=359ms status=500 bytes=438 protocol=https

I am unable to figure this out. If you have any knowledge of NLP model deployment on Heroku, Please help me with this. Thanks once again.

[taishi-i]

Hi @Pranjal-bisht. Let me check before I answer. Have you tried this methods https://github.com/taishi-i/nagisa/issues/28#issuecomment-1174043806? It is to replace tokenizers.

[Pranjal-bisht]

Hi @taishi-i Yes I have tried this. This method actually resolved memory exceed error, but the AttributeError: Can't get attribute 'tokenize_jp' on <module '__main__' from '/app/.heroku/python/bin/gunicorn'> still persist. I tried to resolved this by building the function inside the same API , but the application crashed in the beginning only. Flask code :

from flask import Flask,render_template,request
import pandas as pd
from scipy.sparse import csr_matrix, hstack
import pickle
import os
from janome.tokenizer import Tokenizer 

app = Flask(__name__)

picFolder = os.path.join('static','icons')
app.config['UPLOAD_FOLDER'] = picFolder

@app.route('/')
def index():
    nohara1 = os.path.join(app.config['UPLOAD_FOLDER'],'nohara1.png')
    photo = os.path.join(app.config['UPLOAD_FOLDER'],'photo.jpg')
    return render_template('index.html', user_image = [nohara1,photo])

@app.route('/predict',methods=['POST'])
def predict():
    if request.method == 'POST':
        nohara1 = os.path.join(app.config['UPLOAD_FOLDER'],'nohara1.png')
        photo = os.path.join(app.config['UPLOAD_FOLDER'],'photo.jpg')
        list_of_data = [x for x in request.form.values()]

        X =pd.DataFrame([list_of_data], columns=["buyers name","branch name","person in charge","construction site","Product name","specification","quantity","NewOrOldsite"])

        # Takes in a document, returns the list of words:
        trans_buyers_name = pickle.load(open("en_buyers_name.pkl",'rb'))
        trans_val_buyers_name = trans_buyers_name.transform(X['buyers name'].values)

        trans_branch_name = pickle.load(open("en_branch_name.pkl",'rb'))
        trans_val_branch_name = trans_branch_name.transform(X['branch name'].values)

        trans_person_in_charge = pickle.load(open("en_person_in_charge.pkl",'rb'))
        trans_val_person_in_charge = trans_person_in_charge.transform(X['person in charge'].values)
        print("3",trans_val_person_in_charge.shape)

        test_trans = csr_matrix(pd.get_dummies(X[['quantity','NewOrOldsite']], sparse=True).values)

        trans_construction_site = pickle.load(open("en_construction_site.pkl",'rb'))
        trans_val_construction_site = trans_construction_site.transform(X['construction site'].values)

        trans_Product_name = pickle.load(open("en_Product_name.pkl",'rb'))
        trans_val_Product_name = trans_Product_name.transform(X['Product name'].values)

        trans_specification = pickle.load(open("en_specification.pkl",'rb'))
        trans_val_specification = trans_specification.transform(X['specification'].values)

        X = hstack((trans_val_buyers_name,trans_val_branch_name, trans_val_person_in_charge,test_trans, trans_val_construction_site,
                            trans_val_Product_name, trans_val_specification)).tocsr().astype('float32')
        df = X
        model = pickle.load(open("xgb_model.pkl",'rb'))
        prediction = model.predict(df)
        print(prediction[0])

        return render_template('index.html', user_image = [nohara1,photo],price='Estimated Price : ¥ {:.2f}'.format(prediction[0]))

def tokenize_jp(doc):                                                          
    words = list(Tokenizer().tokenize(doc, wakati=True))                                 
    return words

if __name__ == '__main__':
    # app.run(host='0.0.0.0',port=8080)
    app.run(debug=True)

In this code, I have shifted the tokenize_jp inside the /predict route. But this solution didn't work as the Application is not working due to memory issues.

[taishi-i]

Hi @Pranjal-bisht. It seems that the memory error has been resolved. However, it seems that the function (tokenize_jp) of the pickled file you read is not loaded on gunicorn. This is similar to this problem. I haven't seen the code to save the pickle file, so I can't give an accurate answer, but please refer to Answers from this page. I think it will be a hint.

[Pranjal-bisht]

Hi @taishi-i . This is how I had pickled the vectorizer ( which uses tokenized_jp )

from janome.tokenizer import Tokenizer                                         

def tokenize_jp(doc):                                                          
    words = list(Tokenizer().tokenize(doc, wakati=True))                                 
    return words

and called this in this:

# column = 'construction site'
cnt_vect_construction_site = CountVectorizer(tokenizer=tokenize_jp, ngram_range = (1, 2), max_features = 1000000)
cnt_vect_construction_site.fit(X['construction site'].values)
cnt_category_construction_site = cnt_vect_construction_site.transform(X['construction site'].values)
pickle.dump(cnt_vect_construction_site,open('en_construction_site.pkl','wb'))

[taishi-i]

Hi @Pranjal-bisht. OK. I will try to come up with a solution using this site as a reference.

Please write tokenize_jp to the file(e.g., utils_tokenizer.py).

from janome.tokenizer import Tokenizer                                         

def tokenize_jp(doc):                                                          
    words = list(Tokenizer().tokenize(doc, wakati=True))                                 
    return words

Load it in the python script where pickle is saved and in the API scirpt, respectively.

from utils_tokenizer import tokenize_jp

...

[Pranjal-bisht]

Hi @taishi-i is utils file same as saving something like a pickle?I havent worked with them till now. like the StackOverflow link mentions , To fix this, I moved the tokenizer function to a separate utilities python file

What function to use to move tokenizer function to a separate utilities python file ?

[Pranjal-bisht]

Hi @taishi-i , I tried another approach from internet. I used (dill) as pickle as it can store the parameters ( tokenizer ) along with the vectorizers we are pickling. This approach was works fine in Local PC but in Heroku, again there is error in the memory limit exceeded. ( This time app is also not opening ). Log :

2022-07-06T08:33:31.524480+00:00 heroku[web.1]: Process running mem=1280M(250.0%)
2022-07-06T08:33:31.593416+00:00 app[web.1]: [2022-07-06 08:33:31 +0000] [4] [WARNING] Worker with pid 9 was terminated due to signal 9
2022-07-06T08:33:31.630902+00:00 app[web.1]: [2022-07-06 08:33:31 +0000] [25] [INFO] Booting worker with pid: 25
2022-07-06T08:33:31.645496+00:00 heroku[web.1]: Error R15 (Memory quota vastly exceeded)
2022-07-06T08:33:31.668053+00:00 heroku[web.1]: Stopping process with SIGKILL
2022-07-06T08:33:31.689593+00:00 heroku[web.1]: Process running mem=686M(134.0%)
2022-07-06T08:33:31.719819+00:00 heroku[router]: at=error code=H13 desc="Connection closed without response" method=GET path="/" host=nohara-pricing-app.herokuapp.com request_id=90c1ba20-f48f-4d9d-b8f3-37e242729661 fwd="49.36.219.41" dyno=web.1 connect=0ms service=5503ms status=503 bytes=0 protocol=https
2022-07-06T08:33:31.725113+00:00 heroku[web.1]: Error R15 (Memory quota vastly exceeded)
2022-07-06T08:33:31.732826+00:00 heroku[web.1]: Stopping process with SIGKILL
2022-07-06T08:33:31.834124+00:00 heroku[web.1]: Process exited with status 137
2022-07-06T08:33:32.061857+00:00 heroku[web.1]: State changed from up to crashed
2022-07-06T08:33:32.066810+00:00 heroku[web.1]: State changed from crashed to starting
2022-07-06T08:33:51.226764+00:00 heroku[web.1]: Starting process with command `gunicorn app:app`
2022-07-06T08:33:52.530826+00:00 app[web.1]: [2022-07-06 08:33:52 +0000] [4] [INFO] Starting gunicorn 20.1.0
2022-07-06T08:33:52.531218+00:00 app[web.1]: [2022-07-06 08:33:52 +0000] [4] [INFO] Listening at: http://0.0.0.0:12927 (4)
2022-07-06T08:33:52.531254+00:00 app[web.1]: [2022-07-06 08:33:52 +0000] [4] [INFO] Using worker: sync
2022-07-06T08:33:52.534736+00:00 app[web.1]: [2022-07-06 08:33:52 +0000] [9] [INFO] Booting worker with pid: 9
2022-07-06T08:33:52.538455+00:00 app[web.1]: [2022-07-06 08:33:52 +0000] [10] [INFO] Booting worker with pid: 10
2022-07-06T08:33:52.920048+00:00 heroku[web.1]: State changed from starting to up
2022-07-06T08:33:02.645590+00:00 app[api]: Release v19 created by user bishtpranjal27@gmail.com
2022-07-06T08:33:02.645590+00:00 app[api]: Deploy bb2a6130 by user bishtpranjal27@gmail.com
2022-07-06T08:34:00.722397+00:00 heroku[web.1]: Process running mem=1279M(250.0%)
2022-07-06T08:34:00.961301+00:00 heroku[web.1]: Error R15 (Memory quota vastly exceeded)
2022-07-06T08:34:00.996628+00:00 heroku[web.1]: Stopping process with SIGKILL
2022-07-06T08:34:01.116065+00:00 heroku[router]: at=error code=H13 desc="Connection closed without response" method=GET path="/favicon.ico" host=nohara-pricing-app.herokuapp.com request_id=44f74f83-a9b6-4cf7-9490-9afcc02d62f6 fwd="49.36.219.41" dyno=web.1 connect=0ms service=7554ms status=503 bytes=0 protocol=https
2022-07-06T08:34:01.460610+00:00 heroku[web.1]: Process exited with status 137
2022-07-06T08:34:01.621150+00:00 heroku[web.1]: State changed from up to crashed
2022-07-06T08:34:11.823003+00:00 heroku[router]: at=error code=H10 desc="App crashed" method=GET path="/" host=nohara-pricing-app.herokuapp.com request_id=4ade0a50-b2b0-4495-a5f6-983bff322a0d fwd="49.36.219.41" dyno= connect= service= status=503 bytes= protocol=https
2022-07-06T08:34:12.325891+00:00 heroku[router]: at=error code=H10 desc="App crashed" method=GET path="/favicon.ico" host=nohara-pricing-app.herokuapp.com request_id=8067c0fb-7ece-4b20-82b0-82af7c440f2d fwd="49.36.219.41" dyno= connect= service= status=503 bytes= protocol=https

[taishi-i]

Hi @Pranjal-bisht. Once you check the memory usage of the ./predict API using the following library. I want to see how much memory your API needs. Then we work together to come up with a solution.

from memory_profiler import profile

This is a sample of usage.

Hi @Pranjal-bisht. Thank you for the Python libraries' information. I used the following code to check memory usage. As a result, the Python libraries in your configuration use at least 345.8 MiB of memory. In addition, loading xgb models and pickle files will use additional memory.

from memory_profiler import profile

@profile
def main():
    import os
    import pickle

    import nagisa
    import pandas as pd
    import xgboost as xgb

    from random import shuffle
    from flask import Flask, render_template, request
    from scipy.sparse import csr_matrix, hstack

    text = "これはサンプルの文です。"
    words = nagisa.tagging(text)
    print(words)

if __name__ == '__main__':
    main()

Line #    Mem usage    Increment  Occurrences   Line Contents
=============================================================
     4     39.1 MiB     39.1 MiB           1   @profile
     5                                         def main():
     6     39.1 MiB      0.0 MiB           1       import os
     7     39.1 MiB      0.0 MiB           1       import pickle
     8                                         
     9    270.2 MiB    231.1 MiB           1       import nagisa
    10    293.5 MiB     23.2 MiB           1       import pandas as pd
    11    339.4 MiB     45.9 MiB           1       import xgboost as xgb
    12                                         
    13    339.4 MiB      0.0 MiB           1       from random import shuffle
    14    345.3 MiB      5.9 MiB           1       from flask import Flask, render_template, request
    15    345.3 MiB      0.0 MiB           1       from scipy.sparse import csr_matrix, hstack
    16                                         
    17    345.3 MiB      0.0 MiB           1       text = "これはサンプルの文です。"
    18    345.3 MiB      0.0 MiB           1       words = nagisa.tagging(text)
    19    345.8 MiB      0.5 MiB           1       print(words)

I think the 512MB of memory in the Free plan of heroku will not be enough. Your local environment has enough memory, so it works fine.

This is not a problem with the nagisa library itself. It is a problem of how memory is used in Heroku. As a solution, the free plan Heroku allows you to use two processes. So how about separating the API for tokenizing and the API for xgb models to separate the memory usage?

[Pranjal-bisht]

Hi @taishi-i I think I can give u access to my private repo. This will fasten things up. Again I am very much obliged and thankful to you for your support. I have requested to add you.

[taishi-i]

Hi @Pranjal-bisht. No problem! I have accepted the invitation to your repository. Since this issue is no longer in nagisa's repository, may I close this issue? Then let's discuss it in your repository.

[Pranjal-bisht]

Sure @taishi-i San.