model_to_json raises Object of type 'int64' is not JSON serializable on Model object

[Kieran-Bacon]

After training a prophet model with the following code

  model = Prophet(
      holidays=self._specialPeriods,
      growth='logistic',
      changepoint_prior_scale=metric.changepoint_prior_scale,
      seasonality_prior_scale=metric.seasonality_prior_scale
  )

  # Select the training features needed for this metric
  trainingData = clientWeeklyData[['created_on', metric.name, *metric.features]]

  # Set metric params on data
  trainingData.rename({'created_on':'ds', metric.name:'y'}, axis=1, inplace=True) # Prepare data in correct format for the prophet.
  trainingData['cap'] = metric.cap
  trainingData['floor'] = metric.floor

  model.add_country_holidays(country_name='UK')
  for parameter in metric.features:
      model.add_regressor(parameter)

  self._models.setdefault(client, {})[metric.name] = model.fit(trainingData)

where trainingData and self._specialPeriods are pandas data frames

# trainingData.dtypes
ds            datetime64[ns]
y                           float64
****                       float64
****                       float64
****                       float64
****                       float64
****                       float64
****                       float64
****                       float64
****                       float64
****                       float64
****                       float64
****                       float64
****                       float64
****                       float64
****                       float64
****                       float64
****                       float64
cap                          int64
floor                        int64

You receive the following error when you come to save the model using model_to_json

>>>model_to_json(model)
Traceback (most recent call last):
  File "/usr/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/vscode/.vscode-server/extensions/ms-python.python-2021.3.658691958/pythonFiles/lib/python/debugpy/__main__.py", line 45, in <module>
    cli.main()
  File "/home/vscode/.vscode-server/extensions/ms-python.python-2021.3.658691958/pythonFiles/lib/python/debugpy/../debugpy/server/cli.py", line 444, in main
    run()
  File "/home/vscode/.vscode-server/extensions/ms-python.python-2021.3.658691958/pythonFiles/lib/python/debugpy/../debugpy/server/cli.py", line 331, in run_module
    run_module_as_main(target_as_str, alter_argv=True)
  File "/usr/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/workspaces/optimised_package_margin/components/pipeline/pipeline/__main__.py", line 100, in <module>
    model.save(args.model_path)
  File "/workspaces/optimised_package_margin/components/pipeline/pipeline/forecast_model.py", line 331, in save
    json.dump(model_to_json(model), handle)
  File "/home/vscode/.local/lib/python3.6/site-packages/fbprophet/serialize.py", line 91, in model_to_json
    return json.dumps(model_json)
  File "/usr/lib/python3.6/json/__init__.py", line 231, in dumps
    return _default_encoder.encode(obj)
  File "/usr/lib/python3.6/json/encoder.py", line 199, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/usr/lib/python3.6/json/encoder.py", line 257, in iterencode
    return _iterencode(o, 0)
  File "/usr/lib/python3.6/json/encoder.py", line 180, in default
    o.__class__.__name__)
TypeError: Object of type 'int64' is not JSON serializable

Versions: Python 3.6.9 fbprophet 0.7.1 pandas 1.1.5

Please could you take a look into this problem - in the meantime, do I need to convert the DataFrame into something else before I pass it in to avoid having the np.int64?

[bletham]

There's probably a bug in here but I'm not exactly sure where.

Having cap and floor as int64s shouldn't be an issue; those values are serialized via pandas, which should be correctly handling them (at least in pd 1.2.3 that I'm testing it does). You can verify that with this code, which is how fbprophet is serializing it under the hood:

import json
json.dumps(trainingData.to_json(orient='table', index=False))

Does that work?

The holidays input is serialized in the same way.

Other possible culprits are metric.changepoint_prior_scale and metric.seasonality_prior_scale; is one of these by chance an np.int64?

Otherwise I'm not sure where it's coming from. I tried to produce the error with the following code that has an int64 column in the training data and it works (using data from the examples folder on this github):

import pandas as pd
from fbprophet.serialize import model_to_json
from fbprophet import Prophet

df = pd.read_csv('../examples/example_wp_log_R.csv')
df['cap'] = 8
m = Prophet(growth='logistic')
m.add_country_holidays(country_name='UK')
m.fit(df)

print(df.dtypes)
# ds      object
# y      float64
# cap      int64
# dtype: object

model_to_json(m)  # Succeeds

Does that work for you? Otherwise could you construct a code example that I can run that produces the error?

[amw5g]

I experienced something nearly the same. Different data type in the error (longlong), but otherwise the same problem when serializing the model. I updated fbprophet==0.7.1 to prophet==1.0.1 and I am able to write the model successfully.

[thite-amol]

I have experienced the same issue a few days back and after having multiple experiments with the data types I finally found the solution for the exception.

Basically Prophet model tries to convert a few non-core data types into core data types check the link for reference.

Unfortunately, they are not covering all the datatypes and whenever we used non-core data types for the model attributes it throws an error.

So the solution for the above problem is that use core datatypes while building prophet models like below

model = Prophet( holidays=self._specialPeriods, growth='logistic', changepoint_prior_scale=float(metric.changepoint_prior_scale), seasonality_prior_scale=float(metric.seasonality_prior_scale) )

FYI you can find out the core attribute list here