long2ice / meilisync

Realtime sync data from MySQL/PostgreSQL/MongoDB to Meilisearch
https://github.com/long2ice/meilisync
Apache License 2.0
260 stars 40 forks source link

Mongo ObjectId fields issue #102

Open ngdbao opened 4 months ago

ngdbao commented 4 months ago

While _id is not only ObjectId field in most collection, any other objectId may cause below error:

│                                                                              │
│ /usr/local/lib/python3.12/json/encoder.py:180 in default                     │
│                                                                              │
│   177 │   │   │   │   return JSONEncoder.default(self, o)                    │
│   178 │   │                                                                  │
│   179 │   │   """                                                            │
│ ❱ 180 │   │   raise TypeError(f'Object of type {o.__class__.__name__} '      │
│   181 │   │   │   │   │   │   f'is not JSON serializable')                   │
│   182 │                                                                      │
│   183 │   def encode(self, o):                                               │
│                                                                              │
│ ╭────────────────────────── locals ──────────────────────────╮               │
│ │    o = ObjectId('637e1e913e9e101f8f9150b5')                │               │
│ │ self = <json.encoder.JSONEncoder object at 0x7ab18389f770> │               │
│ ╰────────────────────────────────────────────────────────────╯               │
╰──────────────────────────────────────────────────────────────────────────────╯
TypeError: Object of type ObjectId is not JSON serializable

So, It would be better if we can define stringify some fields right in config.yml, example like this:

  - table: picture
    index: beauty-pictures
    full: true
    fields:
      id: str(id)
      uid: str(id)
      description: description
      category: category