search

hwchase17 / notion-qa

MIT License
2.13k stars 376 forks source link

Ingest - Longer than the specified 1500 #14

Open xingfanxia opened 1 year ago

xingfanxia commented 1 year ago

Is this a problem with ingesting?

❯ python3 ingest.py                                                                                                                                                              ─╯
Created a chunk of size 1738, which is longer than the specified 1500
Created a chunk of size 1698, which is longer than the specified 1500
Created a chunk of size 1568, which is longer than the specified 1500
Created a chunk of size 2845, which is longer than the specified 1500
Created a chunk of size 2181, which is longer than the specified 1500
Created a chunk of size 2042, which is longer than the specified 1500
Created a chunk of size 2387, which is longer than the specified 1500
Created a chunk of size 2305, which is longer than the specified 1500
Created a chunk of size 2076, which is longer than the specified 1500
Created a chunk of size 2316, which is longer than the specified 1500
Created a chunk of size 2400, which is longer than the specified 1500
Created a chunk of size 1520, which is longer than the specified 1500
Created a chunk of size 1718, which is longer than the specified 1500
Created a chunk of size 2526, which is longer than the specified 1500
Created a chunk of size 1874, which is longer than the specified 1500
Created a chunk of size 2946, which is longer than the specified 1500
Created a chunk of size 2362, which is longer than the specified 1500
Created a chunk of size 1753, which is longer than the specified 1500
Created a chunk of size 2821, which is longer than the specified 1500
Created a chunk of size 1579, which is longer than the specified 1500
Created a chunk of size 1824, which is longer than the specified 1500
Created a chunk of size 2026, which is longer than the specified 1500
Created a chunk of size 2981, which is longer than the specified 1500
Created a chunk of size 2865, which is longer than the specified 1500
Created a chunk of size 97940, which is longer than the specified 1500
Created a chunk of size 53894, which is longer than the specified 1500
Created a chunk of size 1821, which is longer than the specified 1500
Created a chunk of size 2861, which is longer than the specified 1500
Created a chunk of size 5088, which is longer than the specified 1500
Created a chunk of size 1865, which is longer than the specified 1500
Created a chunk of size 1538, which is longer than the specified 1500
Created a chunk of size 1542, which is longer than the specified 1500
Created a chunk of size 2046, which is longer than the specified 1500
Created a chunk of size 2558, which is longer than the specified 1500
Created a chunk of size 1882, which is longer than the specified 1500
Created a chunk of size 13716, which is longer than the specified 1500
Created a chunk of size 2183, which is longer than the specified 1500
Created a chunk of size 3526, which is longer than the specified 1500
Created a chunk of size 1702, which is longer than the specified 1500
Created a chunk of size 2313, which is longer than the specified 1500
Created a chunk of size 1775, which is longer than the specified 1500
Created a chunk of size 5809, which is longer than the specified 1500
Created a chunk of size 4280, which is longer than the specified 1500
Created a chunk of size 3446, which is longer than the specified 1500
Created a chunk of size 2207, which is longer than the specified 1500
Created a chunk of size 2502, which is longer than the specified 1500
Created a chunk of size 2465, which is longer than the specified 1500
Created a chunk of size 16323, which is longer than the specified 1500
Created a chunk of size 2277, which is longer than the specified 1500
Created a chunk of size 8885, which is longer than the specified 1500
Created a chunk of size 3056, which is longer than the specified 1500
Created a chunk of size 14512, which is longer than the specified 1500
Created a chunk of size 2896, which is longer than the specified 1500
Created a chunk of size 10896, which is longer than the specified 1500
Created a chunk of size 1504, which is longer than the specified 1500
Created a chunk of size 3937, which is longer than the specified 1500
Created a chunk of size 1643, which is longer than the specified 1500
Created a chunk of size 2604, which is longer than the specified 1500
Created a chunk of size 4257, which is longer than the specified 1500
Created a chunk of size 2635, which is longer than the specified 1500
Created a chunk of size 1907, which is longer than the specified 1500
Created a chunk of size 2010, which is longer than the specified 1500
Created a chunk of size 1832, which is longer than the specified 1500
Created a chunk of size 1517, which is longer than the specified 1500
Created a chunk of size 2795, which is longer than the specified 1500
Created a chunk of size 1795, which is longer than the specified 1500
Created a chunk of size 1862, which is longer than the specified 1500
Created a chunk of size 2764, which is longer than the specified 1500
Created a chunk of size 1596, which is longer than the specified 1500
Created a chunk of size 3146, which is longer than the specified 1500
Created a chunk of size 1655, which is longer than the specified 1500
Created a chunk of size 1563, which is longer than the specified 1500
Created a chunk of size 1547, which is longer than the specified 1500
Created a chunk of size 1570, which is longer than the specified 1500
Created a chunk of size 1688, which is longer than the specified 1500
Created a chunk of size 1626, which is longer than the specified 1500
Created a chunk of size 2224, which is longer than the specified 1500
Created a chunk of size 3120, which is longer than the specified 1500
Created a chunk of size 2197, which is longer than the specified 1500
lmsanch commented 1 year ago

I am having the same issue. Not sure if I need to specify a different chunk size in the code and/or if this affects performance, etc.