flucoma / flucoma-core

Core algorithms and objects for the Fluid Corpus Manipulation Library
BSD 3-Clause "New" or "Revised" License
77 stars 14 forks source link

JSON parsing needs to be robust against malformed / zero-size containers #179

Open rconstanzo opened 2 years ago

rconstanzo commented 2 years ago

So I've created a .json file with the wrong info in it (I crossed my wires for a fluid.dataset~ and fluid.knnclassifier~ state save) and when I read this into fluid.knnclassifier~ I get a 100% reproducible (on beta7) insta crash.

Steps to reproduce. Read the .json file below into the patch below. Instacrash.

This is the patch:


----------begin_max5_patcher----------
574.3ocuUssbaBCD8Y3qfQOSYPBYPjekNc7HLJoJAKXDBWmIS52dkVANts3q
klGLd1Uq1iNqN6p2BCPUs6E8nnGh9ZTPvagAAfKmifQ6.zV99MM7dHLjR7i1
pmQw9kLh8Fv8iMCx5jWTJHR4iRg9mSA0NXZDFyqcBOPHTr8Wz2FWVVCYvl0u
Pl1hZXqTY2DfIdzYG2r46R0Sq0hMFepnzjz3HRQl6ObZAXQRROjbah7vCYh3
b9dXn6S7UR3pAioUcRtTwUOMKWvonCmaMeqvHzqEJdUCryzajnXJySQbRICW
VvX37hbZJNyRXeQfdRZi+Doc1RxZ+06prI9s3r7DpYSTUjVvqOO8spXHnYKC
36SJO6ELs7+ft9DTuVZOPmh1tEksJt9UnEN9LsxX1rE.xEK.Vg9JJgTfoXZJ
KGSoVEPZxpyV.nid646D0qs3ay5ZtwnkVcre.Wvg5P.RrsRTejf7rp0iWbKu
q6vxvpPUegp8cZQmPUG0zdFo27U6x6TuQxv+c4t3R5M7xp2RFTc7MuD8waGO
bizmcmubfYEIEqxwkkd5SxYVmjraf+PHnFo5OeGE.04+2KJ8sC5MSrYZTQzG
HUK5MRE20mcTPt1IWPyV3uIjveJHcETZh2+S.AB+KBEYIPhQtl6oxEgUrqRR
rHzJ89TedYucb3NgteLZ.DaW+ysZmIKFLkJuYFXpE6jSwCsmHt11aZrMlCZ3
fg1mSQ9s1VKzpA4XKmkdVHggDJ673d6LC+FbCdBeO7W.+JEgXB
-----------end_max5_patcher-----------

Screenshot 2022-06-15 at 9 05 05 pm

And this is the file you read into it: clustersnare.json.zip

Here is a crash report: flucoma crash.txt

The relevant bit, I believe, is:

Thread 0 Crashed:: CrBrowserMain  Dispatch queue: com.apple.main-thread
0   libc++.1.dylib                  0x00007fff58e7e5b7 std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >::basic_string(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) + 27
1   org.flucoma.fluid.libmanipulation   0x00000001301ada9a fluid::algorithm::KDTree::unflatten(fluid::algorithm::KDTree::FlatData const&, long) const + 74
2   org.flucoma.fluid.libmanipulation   0x0000000130198925 fluid::algorithm::from_json(nlohmann::basic_json<std::__1::map, std::__1::vector, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, bool, long long, unsigned long long, double, std::__1::allocator, nlohmann::adl_serializer, std::__1::vector<unsigned char, std::__1::allocator<unsigned char> > > const&, fluid::algorithm::KDTree&) + 725
3   org.flucoma.fluid.libmanipulation   0x00000001301a5a59 fluid::client::knnclassifier::from_json(nlohmann::basic_json<std::__1::map, std::__1::vector, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, bool, long long, unsigned long long, double, std::__1::allocator, nlohmann::adl_serializer, std::__1::vector<unsigned char, std::__1::allocator<unsigned char> > > const&, fluid::client::knnclassifier::KNNClassifierData&) + 89
4   org.flucoma.fluid.libmanipulation   0x00000001302a6954 _ZNK8nlohmann10basic_jsonINSt3__13mapENS1_6vectorENS1_12basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEEbxydS7_NS_14adl_serializerENS3_IhNS7_IhEEEEE3getIN5fluid6client13knnclassifier17KNNClassifierDataESI_EEDTcldtclL_ZNS1_7declvalIRKSD_EEDTclsr3std3__1E9__declvalIT_ELi0EEEvEE8get_implIT0_EtlNS_6detail12priority_tagILj4EEEEEEv + 308
5   org.flucoma.fluid.libmanipulation   0x00000001302a3b4d fluid::client::DataClient<fluid::client::knnclassifier::KNNClassifierData>::load(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >) + 285
6   org.flucoma.fluid.libmanipulation   0x00000001302b7890 _ZNK5fluid6client10MessageSetINSt3__15tupleIJNS0_7MessageIZNS0_11makeMessageINS0_13MessageResultIvEENS0_13knnclassifier19KNNClassifierClientEJNS0_15SharedClientRefIKNS0_7dataset13DataSetClientEEENSA_IKNS0_8labelset14LabelSetClientEEEEEEDaPKcMT0_FT_DpT1_EEUlRS9_SE_SI_E_S7_S9_JSE_SI_EEENS4_IZNS0_11makeMessageIS7_S9_JSE_NSA_ISG_EEEEESJ_SL_MSM_KFSN_SP_EEUlSS_SE_SW_E_S7_S9_JSE_SW_EEENS4_IZNSV_INS6_INS2_12basic_stringIcNS2_11char_traitsIcEENS2_9allocatorIcEEEEEES9_JNS2_10shared_ptrIKNS0_13BufferAdaptorEEEEEESJ_SL_SY_EUlSS_S1B_E_S17_S9_JS1B_EEENS4_IZNSV_INS6_IlEENS0_10DataClientINS8_17KNNClassifierDataEEEJEEESJ_SL_SY_EUlRS1H_E_S1E_S1H_JEEENS4_IZNS5_IS7_S1H_JEEESJ_SL_SR_EUlS1I_E_S7_S1H_JEEES1K_NS4_IZNS5_IS7_S1H_JS16_EEESJ_SL_SR_EUlS1I_S16_E_S7_S1H_JS16_EEENS4_IZNS5_IS17_S1H_JEEESJ_SL_SR_EUlS1I_E_S17_S1H_JEEES1O_S1O_EEEE6invokeILm6EJRNS0_24NRTSharedInstanceAdaptorIS9_E12SharedClientERSL_EEEDcDpOT0_ + 288
7   org.flucoma.fluid.libmanipulation   0x00000001302b7759 decltype(auto) fluid::client::NRTThreadingAdaptor<fluid::client::NRTSharedInstanceAdaptor<fluid::client::knnclassifier::KNNClassifierClient> >::invoke<6ul, fluid::client::NRTThreadingAdaptor<fluid::client::NRTSharedInstanceAdaptor<fluid::client::knnclassifier::KNNClassifierClient> >, char const*&>(fluid::client::NRTThreadingAdaptor<fluid::client::NRTSharedInstanceAdaptor<fluid::client::knnclassifier::KNNClassifierClient> >&, char const*&&&) + 569
8   org.flucoma.fluid.libmanipulation   0x00000001302b729f void fluid::client::FluidMaxWrapper<fluid::client::NRTThreadingAdaptor<fluid::client::NRTSharedInstanceAdaptor<fluid::client::knnclassifier::KNNClassifierClient> > >::doLoad<6ul>(fluid::client::FluidMaxWrapper<fluid::client::NRTThreadingAdaptor<fluid::client::NRTSharedInstanceAdaptor<fluid::client::knnclassifier::KNNClassifierClient> > >*, symbol*, long, atom*) + 319
weefuzzy commented 2 years ago

Following the crash through, it looks like it doesn't like the idea of 0-size, empty containers

"classifier" :  {
    "labels" :      {
        "cols" : 1
    },
    "tree" :        {
        "cols" : 0,
        "data" : null,
        "ids" : [  ],
        "rows" : 0,
        "tree" : null
    }
}

So, I'll toughen up against this pathological case, but in the meanwhile, this data wouldn't have done anything useful anyway.