search

solid-adventure / trivial-api

Other
1 stars 1 forks source link

Sending an object to datastore and then adding a value generates an error. #155

Closed vgkids closed 1 year ago

vgkids commented 1 year ago

Sending an object and then adding a value generates an error.

Started with this, which created the table and columns as expected.

POST http://127.0.0.1:3001/datastore/insert_values

{
    "records":
    [
        {
            "test": "b",
            "b": false,
            "a": 34242,
            "activated": false,
            "time": "2023-04-20T17:21:15.218Z",
            "id": 34241254545534,
            "value": "3422"
        }
    ],
    "table_name": "testing_JM",
    "unique_keys": "id"
}

Followed by this, identically but with new key c:

{
    "records":
    [
        {
            "test": "b",
            "b": false,
            "a": 34242,
            "c": 123,
            "activated": false,
            "time": "2023-04-20T17:21:15.218Z",
            "id": 34241254545534,
            "value": "3422"
        }
    ],
    "table_name": "testing_JM",
    "unique_keys": "id"
}

Generates an error:

executing sql: SELECT column_name, data_type FROM information_schema.columns WHERE table_name = 'testing_JM'
Completed 500 Internal Server Error in 248ms (ActiveRecord: 74.3ms | Allocations: 890)

ArgumentError (bad value for range):

app/helpers/datastore_manager.rb:190:in `verify_model_and_insert_records'
app/controllers/datastore_controller.rb:14:in `insert_values'
vgkids commented 1 year ago

First send

{
    "records":
    [
{
    "id": "2PgE9m9L4MURH978Q8x6FZseV",
    "state": "COMPLETED",
    "taxes":
    [
        {
            "uid": "b5f287d3-c07b-4aa5-9766-8862d81b122c",
            "name": "Michigan Sales Tax",
            "type": "ADDITIVE",
            "scope": "LINE_ITEM",
            "percentage": "6",
            "applied_money":
            {
                "amount": 1862,
                "currency": "USD"
            },
            "catalog_version": 1676048493922,
            "catalog_object_id": "E4PWSW6C3IMFEFV2PRFAAFT7"
        }
    ],
    "tenders":
    [
        {
            "id": "RAM8aIa7KNu8g0lY8o6NlhkFvaB",
            "type": "CARD",
            "tip_money":
            {
                "amount": 0,
                "currency": "USD"
            },
            "created_at": "2023-03-01T14:33:50Z",
            "customer_id": "0E6GGR7MKX0ZS67DTHF85P1R5W",
            "location_id": "L8159DX4SRP29",
            "amount_money":
            {
                "amount": 42948,
                "currency": "USD"
            },
            "card_details":
            {
                "card":
                {
                    "last_4": "9164",
                    "card_brand": "VISA"
                },
                "status": "CAPTURED",
                "entry_method": "ON_FILE"
            },
            "transaction_id": "2PgE9m9L4MURH978Q8x6FZseV",
            "processing_fee_money":
            {
                "amount": 1518,
                "currency": "USD"
            }
        }
    ],
    "closed_at": "2023-03-01T14:33:50Z",
    "discounts":
    [
        {
            "uid": "d4b7f602-bd7f-46e4-a2e8-1a554109db33",
            "name": "PM plan Part Discount",
            "type": "FIXED_PERCENTAGE",
            "scope": "LINE_ITEM",
            "percentage": "25",
            "applied_money":
            {
                "amount": 10346,
                "currency": "USD"
            },
            "catalog_version": 1673900707623,
            "catalog_object_id": "GPXDKBVEMG4FXFBQ3Q3N3XWI"
        },
        {
            "uid": "1856c39a-ae8d-41ed-a3a9-3cf7371b42fc",
            "name": "PM plan Labor Discount",
            "type": "FIXED_PERCENTAGE",
            "scope": "LINE_ITEM",
            "percentage": "33",
            "applied_money":
            {
                "amount": 4950,
                "currency": "USD"
            },
            "catalog_version": 1673900670604,
            "catalog_object_id": "PWM6X72FDBW43CY7YTOPIMGF"
        }
    ],
    "created_at": "2023-03-01T14:33:50Z",
    "line_items":
    [
        {
            "uid": "ebaa20e6-89a0-4461-9a85-4b98517c3954",
            "name": "Optipure Filter - CTO-Q",
            "quantity": "2",
            "item_type": "ITEM",
            "total_money":
            {
                "amount": 19748,
                "currency": "USD"
            },
            "applied_taxes":
            [
                {
                    "uid": "fcb1785a-35de-4a31-8394-72bdfa99df39",
                    "tax_uid": "b5f287d3-c07b-4aa5-9766-8862d81b122c",
                    "applied_money":
                    {
                        "amount": 1118,
                        "currency": "USD"
                    }
                }
            ],
            "variation_name": "Regular",
            "catalog_version": 1676048493922,
            "total_tax_money":
            {
                "amount": 1118,
                "currency": "USD"
            },
            "base_price_money":
            {
                "amount": 12420,
                "currency": "USD"
            },
            "applied_discounts":
            [
                {
                    "uid": "e0e42111-fe53-4308-bced-b41adbdda9b8",
                    "discount_uid": "d4b7f602-bd7f-46e4-a2e8-1a554109db33",
                    "applied_money":
                    {
                        "amount": 6210,
                        "currency": "USD"
                    }
                }
            ],
            "catalog_object_id": "6TSZCCTTKAU25NNO2FYF7C4X",
            "gross_sales_money":
            {
                "amount": 24840,
                "currency": "USD"
            },
            "total_discount_money":
            {
                "amount": 6210,
                "currency": "USD"
            },
            "variation_total_price_money":
            {
                "amount": 24840,
                "currency": "USD"
            }
        },
        {
            "uid": "21098acf-00d5-43df-b7a2-6ead36f69553",
            "name": "Optipure Filter - MA-Q10",
            "quantity": "1",
            "item_type": "ITEM",
            "total_money":
            {
                "amount": 7918,
                "currency": "USD"
            },
            "applied_taxes":
            [
                {
                    "uid": "ddd12012-e693-4752-96b6-33b49d5707cd",
                    "tax_uid": "b5f287d3-c07b-4aa5-9766-8862d81b122c",
                    "applied_money":
                    {
                        "amount": 448,
                        "currency": "USD"
                    }
                }
            ],
            "variation_name": "Regular",
            "catalog_version": 1676048493922,
            "total_tax_money":
            {
                "amount": 448,
                "currency": "USD"
            },
            "base_price_money":
            {
                "amount": 9960,
                "currency": "USD"
            },
            "applied_discounts":
            [
                {
                    "uid": "7c6c6f3f-98b6-4edb-beb6-b4fa9577478d",
                    "discount_uid": "d4b7f602-bd7f-46e4-a2e8-1a554109db33",
                    "applied_money":
                    {
                        "amount": 2490,
                        "currency": "USD"
                    }
                }
            ],
            "catalog_object_id": "AXAS5XBYUJUAOAEVSRIHTLIU",
            "gross_sales_money":
            {
                "amount": 9960,
                "currency": "USD"
            },
            "total_discount_money":
            {
                "amount": 2490,
                "currency": "USD"
            },
            "variation_total_price_money":
            {
                "amount": 9960,
                "currency": "USD"
            }
        },
        {
            "uid": "ffcd8aff-c6b0-43e3-8091-7d42105145e8",
            "name": "La Marzocco Group head Micro switch",
            "quantity": "1",
            "item_type": "ITEM",
            "total_money":
            {
                "amount": 5232,
                "currency": "USD"
            },
            "applied_taxes":
            [
                {
                    "uid": "5dbe5fc4-370a-4cd4-a105-5e42f79a1eae",
                    "tax_uid": "b5f287d3-c07b-4aa5-9766-8862d81b122c",
                    "applied_money":
                    {
                        "amount": 296,
                        "currency": "USD"
                    }
                }
            ],
            "variation_name": "Regular",
            "catalog_version": 1677183730401,
            "total_tax_money":
            {
                "amount": 296,
                "currency": "USD"
            },
            "base_price_money":
            {
                "amount": 6582,
                "currency": "USD"
            },
            "applied_discounts":
            [
                {
                    "uid": "dd491933-c222-45ab-a9d7-e2a9af0949e6",
                    "discount_uid": "d4b7f602-bd7f-46e4-a2e8-1a554109db33",
                    "applied_money":
                    {
                        "amount": 1646,
                        "currency": "USD"
                    }
                }
            ],
            "catalog_object_id": "LPNICJGJ2S5XA2S4X4K3NWH5",
            "gross_sales_money":
            {
                "amount": 6582,
                "currency": "USD"
            },
            "total_discount_money":
            {
                "amount": 1646,
                "currency": "USD"
            },
            "variation_total_price_money":
            {
                "amount": 6582,
                "currency": "USD"
            }
        },
        {
            "uid": "c74b9fe6-7697-4337-b208-895e3128b383",
            "name": "Tech Labor",
            "quantity": "2",
            "item_type": "ITEM",
            "total_money":
            {
                "amount": 10050,
                "currency": "USD"
            },
            "quantity_unit":
            {
                "precision": 2,
                "measurement_unit":
                {
                    "type": "TYPE_TIME",
                    "time_unit": "GENERIC_HOUR"
                },
                "catalog_object_id": "W5MGJHWPZF4FKYFM2ZPFODOZ"
            },
            "variation_name": "Regular",
            "catalog_version": 1673902002379,
            "total_tax_money":
            {
                "amount": 0,
                "currency": "USD"
            },
            "base_price_money":
            {
                "amount": 7500,
                "currency": "USD"
            },
            "applied_discounts":
            [
                {
                    "uid": "36c422d9-9c93-49cd-9046-010cff51dffd",
                    "discount_uid": "1856c39a-ae8d-41ed-a3a9-3cf7371b42fc",
                    "applied_money":
                    {
                        "amount": 4950,
                        "currency": "USD"
                    }
                }
            ],
            "catalog_object_id": "44OAGVN5ZXFTOXWRDQWAGZUS",
            "gross_sales_money":
            {
                "amount": 15000,
                "currency": "USD"
            },
            "total_discount_money":
            {
                "amount": 4950,
                "currency": "USD"
            },
            "variation_total_price_money":
            {
                "amount": 15000,
                "currency": "USD"
            }
        }
    ],
    "updated_at": "2023-03-01T14:33:50Z",
    "customer_id": "0E6GGR7MKX0ZS67DTHF85P1R5W",
    "location_id": "L8159DX4SRP29",
    "net_amounts":
    {
        "tax_money":
        {
            "amount": 1862,
            "currency": "USD"
        },
        "tip_money":
        {
            "amount": 0,
            "currency": "USD"
        },
        "total_money":
        {
            "amount": 42948,
            "currency": "USD"
        },
        "discount_money":
        {
            "amount": 15296,
            "currency": "USD"
        },
        "service_charge_money":
        {
            "amount": 0,
            "currency": "USD"
        }
    },
    "total_money":
    {
        "amount": 42948,
        "currency": "USD"
    },
    "return_amounts":
    {
        "tax_money":
        {
            "amount": 0,
            "currency": "USD"
        },
        "tip_money":
        {
            "amount": 0,
            "currency": "USD"
        },
        "total_money":
        {
            "amount": 0,
            "currency": "USD"
        },
        "discount_money":
        {
            "amount": 0,
            "currency": "USD"
        },
        "service_charge_money":
        {
            "amount": 0,
            "currency": "USD"
        }
    },
    "total_tax_money":
    {
        "amount": 1862,
        "currency": "USD"
    },
    "total_tip_money":
    {
        "amount": 0,
        "currency": "USD"
    },
    "net_amount_due_money":
    {
        "amount": 0,
        "currency": "USD"
    },
    "total_discount_money":
    {
        "amount": 15296,
        "currency": "USD"
    },
    "total_service_charge_money":
    {
        "amount": 0,
        "currency": "USD"
    }
}
    ],
    "table_name": "square_orders",
    "unique_keys": "id"
}

Then

{
    "records":
    [
{
    "id": "GXGRkpfcqJrBYFCnLO2vHB7eV",
    "state": "COMPLETED",
    "refunds":
    [
        {
            "id": "EhUAhphLgYsLbJfs4G95S",
            "reason": "Canceled order",
            "status": "APPROVED",
            "tender_id": "vljyWAiioW0pqkw3j0uhmVYnvaB",
            "created_at": "2023-03-21T11:34:59Z",
            "location_id": "LV14NM8EM4H67",
            "amount_money":
            {
                "amount": 5000,
                "currency": "USD"
            },
            "transaction_id": "6Z2VjhodCVBlauI0jMstja4eV"
        }
    ],
    "returns":
    [
        {
            "source_order_id": "6Z2VjhodCVBlauI0jMstja4eV",
            "return_line_items":
            [
                {
                    "uid": "1be1d02b-6689-4eb5-89d3-947fb6b00217",
                    "name": "Helios House Blend",
                    "quantity": "1",
                    "item_type": "ITEM",
                    "total_money":
                    {
                        "amount": 5000,
                        "currency": "USD"
                    },
                    "variation_name": "5 lbs - Helios - Whole Bean",
                    "catalog_version": 1674577161296,
                    "total_tax_money":
                    {
                        "amount": 0,
                        "currency": "USD"
                    },
                    "base_price_money":
                    {
                        "amount": 5000,
                        "currency": "USD"
                    },
                    "catalog_object_id": "5KYLLQJTACLQE6EETA7AA7IV",
                    "gross_return_money":
                    {
                        "amount": 5000,
                        "currency": "USD"
                    },
                    "source_line_item_uid": "d63b483f-d626-4d62-bd4e-13b0acdfb184",
                    "total_discount_money":
                    {
                        "amount": 0,
                        "currency": "USD"
                    },
                    "variation_total_price_money":
                    {
                        "amount": 5000,
                        "currency": "USD"
                    }
                }
            ]
        }
    ],
    "closed_at": "2023-03-21T12:08:26.000Z",
    "created_at": "2023-03-21T11:34:59.000Z",
    "updated_at": "2023-03-21T12:08:26.000Z",
    "location_id": "LV14NM8EM4H67",
    "net_amounts":
    {
        "tax_money":
        {
            "amount": 0,
            "currency": "USD"
        },
        "tip_money":
        {
            "amount": 0,
            "currency": "USD"
        },
        "total_money":
        {
            "amount": -5000,
            "currency": "USD"
        },
        "discount_money":
        {
            "amount": 0,
            "currency": "USD"
        },
        "service_charge_money":
        {
            "amount": 0,
            "currency": "USD"
        }
    },
    "total_money":
    {
        "amount": 0,
        "currency": "USD"
    },
    "return_amounts":
    {
        "tax_money":
        {
            "amount": 0,
            "currency": "USD"
        },
        "tip_money":
        {
            "amount": 0,
            "currency": "USD"
        },
        "total_money":
        {
            "amount": 5000,
            "currency": "USD"
        },
        "discount_money":
        {
            "amount": 0,
            "currency": "USD"
        },
        "service_charge_money":
        {
            "amount": 0,
            "currency": "USD"
        }
    },
    "total_tax_money":
    {
        "amount": 0,
        "currency": "USD"
    },
    "total_tip_money":
    {
        "amount": 0,
        "currency": "USD"
    },
    "net_amount_due_money":
    {
        "amount": 0,
        "currency": "USD"
    },
    "total_discount_money":
    {
        "amount": 0,
        "currency": "USD"
    },
    "total_service_charge_money":
    {
        "amount": 0,
        "currency": "USD"
    }
}
    ],
    "table_name": "square_orders",
    "unique_keys": "id"
}
david-richards-tech commented 1 year ago

Okay, this should be resolved.

david-richards-tech commented 1 year ago

Ill wait until you have a chance to confirm before closing.

vgkids commented 1 year ago

Pulled this down and restarted the rails server. Now on the 2nd version I get a different error:

executing sql: SELECT column_name, data_type FROM information_schema.columns WHERE table_name = 'testing_JM3'
Completed 500 Internal Server Error in 916ms (ActiveRecord: 92.1ms | Allocations: 995)

RuntimeError (Existing table hash no longer matches table hash in the database):

app/helpers/datastore_manager.rb:201:in `verify_model_and_insert_records'
app/controllers/datastore_controller.rb:14:in `insert_values'

Rails loves caching schemas, if that's not what you're expecting I'd be making sure you don't need something like https://apidock.com/rails/ActiveRecord/Base/reload, since Rails is not expecting the schema to change between server starts by default.

david-richards-tech commented 1 year ago

Does this happen every time? Could you send the two different objects being used?

david-richards-tech commented 1 year ago

I made a change to the way it generates a schema so if you created this prior to pulling down the latest change it would make sense.

vgkids commented 1 year ago

Ok, first example is working now.

Second example, the larger Square order payloads, does not. The difference between them is the 2nd one has more keys to describe a returned item, FYI. I've got a little collection of uncommon edge cases I'm building up, and that's one of them.

PG::InvalidTextRepresentation (ERROR:  invalid input syntax for type json
LINE 1: ...VALUES ('GXGRkpfcqJrBYFCnLO2vHB7eV', 'COMPLETED', '[{"id"=>"...
                                                             ^
DETAIL:  Token "=" is invalid.
CONTEXT:  JSON data, line 1: [{"id"=...
):

app/helpers/datastore_manager.rb:16:in `exec'
app/helpers/datastore_manager.rb:16:in `execute_datastore_statement'
app/helpers/datastore_manager.rb:98:in `insert_values'
app/helpers/datastore_manager.rb:219:in `block in verify_model_and_insert_records'
app/helpers/datastore_manager.rb:218:in `each'
app/helpers/datastore_manager.rb:218:in `each_slice'
app/helpers/datastore_manager.rb:218:in `verify_model_and_insert_records'
app/controllers/datastore_controller.rb:14:in `insert_values'
david-richards-tech commented 1 year ago

Could you send me the request body for this one?

vgkids commented 1 year ago

Here you go

square_return.json.txt

david-richards-tech commented 1 year ago

Got it. I think I accidentally removed this during iteration, but its handled now. The lists need to be converted to json using .to_json

vgkids commented 1 year ago

Cool, that's working as expected now. Closing.