Synthoid / ExportSheetData

Add-on for Google Sheets that allows sheets to be exported as JSON or XML.
MIT License
236 stars 46 forks source link

Exporting Array of JSON objects fails #97

Closed MikeSzym closed 4 years ago

MikeSzym commented 4 years ago

Hello,

First of all - super awesome plugin, paired with Playfab it's an absolute blessing when it comes to live game balancing!

Unfortunately, I hit an issue that I can't solve and it looks like a simple but painful bug. Namely Arrays don't work with JSON objects. Super simple repo:

JA_test
{"Price":5}
{"Price":5},{"Price":5}

And this will export:

{
  "Balancing": [
    {
      "test": [
        {
          "Price": 5
        }
      ]
    },
    {
      "test": [
        {}
      ]
    }
  ]
}

As you can see, the same JSON object will export properly if there's just one element in array, but multiple will simply not work

I have tried all possible export settings, messed with array separator characters, putting the array in brackets and removing JA_ prefix (which produces an ugly result: "test": "[{\"Price\":5},{\"Price\":5}]") etc. Nothing will work

Please help, you're our only hope ;)

Synthoid commented 4 years ago

What settings are you using? Can you link to a repro sheet please? It sounds like you want to export a sheet as an array of JSON objects which should be doable, but I'd like to see how you're formatting your data as that can sometimes throw a wrench in things.

MikeSzym commented 4 years ago

I don't think there's really any need for repro sheet, just open new sheet and paste this in A1, this is full repro and nothing else is needed:

JA_test {"Price":5} {"Price":5},{"Price":5}

But here's a repro sheet anyways ;) https://docs.google.com/spreadsheets/d/1wiQiTjPMCM5rDCRGoLUvnFrKmVAE63-_g1XqJtoUyxQ/edit?usp=sharing

And yes - that's exactly what I want to do. Settings

https://gyazo.com/dee8cefbb38437e35be57a11c712481c https://gyazo.com/19247c3f47ac11298789b66d6ae87078

Thanks for being so quick!

Synthoid commented 4 years ago

Ah, I see what's going on. The Export cell objects option is largely supposed to be used for entering whole JSON objects in a single cell. If you want to export an array of JSON objects, you should format your sheet like so (extra fields added for GitHub formatting purposes):

Price Available
5 false
6 true

Then you should just need to enable Export sheet arrays and get your expected result:

{
  "Test": [
    {
      "Price": 5,
      "Available": false
    },
    {
      "Price": 6,
      "Available": true
    }
  ]
}
MikeSzym commented 4 years ago

Well, but what you pasted is just the default exporting option, isn't it? We use it extensively as it is, but we need to go beyond that

The whole object structure is a lot more complicated, but basically it's an item that can be upgraded. So there's a lot of stuff that can be formatted the "normal" way, but we also need to export an array/list of prices required for upgrading the item. I updated the example to make it more obvious: https://docs.google.com/spreadsheets/d/1wiQiTjPMCM5rDCRGoLUvnFrKmVAE63-_g1XqJtoUyxQ/edit#gid=0

I know that you're using Unity, so basically we need to deserialize the JSON into something similar to this: https://gyazo.com/772576899a69b54202e4d014fd780737

So... Array of objects (price) within the object (item)


I also tried to do it by avoiding the Array Prefix and putting the formatted JSON array directly, but this didn't work either: This: [{"Price":5},{"Price":5}] Gets auto converted to this (need image to show correctly): https://gyazo.com/3d36f2140649faed8156f560c7926ea3

Synthoid commented 4 years ago

Hmm. It sounds like you are going to need Nested Elements. Do you have an example of what JSON you are expecting? From your screenshot I would guess something like:

{
  "F" : 1,
  "I" : 0,
  "S" : "",
  "List" : []
}
MikeSzym commented 4 years ago

This example was just something found on Google, can't share the actual data here as it's deep in NDA territory...

But yes - that's pretty much what we need. An array of Objects within the object. But definitely more than a regular array of ints or strings (which work without issues)

Here's an example of what we need to fill in-game:

Currency/Price class (need to be have both type and amount as we have multiple currencies)

    public class CurrencyData
    {
        [SerializeField] private int m_type;
        [SerializeField] private int m_amount;
    }

And Item class with array/list of prices and upgrades, which both need to be populated with array of JSON objects:

    public class Item
    {
        string m_itemName;
        CurrencyData[] m_upgradePrices;
        UpgradeData[] m_upgrades;
    }
MikeSzym commented 4 years ago

Hey, apologies for bugging you about it, but it's somewhat critical for us and is blocking moving vast parts of game data for online balancing / LiveOps

Do you think this can be fixed somewhat soon? I truly believe that this is a bug, as exporting object lists should "just work"

Thanks

Synthoid commented 4 years ago

Sorry for the delayed response! It's been a hectic worklife for me this past week. Did you have a chance to look at the Nested Elements link I posted above? Unfortunately, exporting object lists by directly writing multiple objects into a cell does not work but I can see that being useful so I may try allowing that. This would probably require some special formatting, specifically wrapping square brackets like this:

[ {"field01":1}, {"field01":2} ].

I'll take a look to see if I can get something usable based on your structure comments. If you can post an example of what exactly you expect the JSON to look like that will be immensely helpful. You don't need to post identifying info, just the structure you are expecting with some dummy content. This is mostly so I don't misunderstand what you are expecting, but I think I can get something usable based on your requests.

MikeSzym commented 4 years ago

Hey, thanks for the answer!

Yes, I did look at the Nested Elements but I don't think they can be used conveniently when requiring a list of objects multiple items long...

Yes, the example you posted with brackets is exactly what we would like to have in the cell itself. This is from the example sheet I'm using: https://gyazo.com/e43373fb5703ab65b0a601766cce3d25

And the results as it is today: https://gyazo.com/3ac567a8ced33636ec5574482cf9acb3

As you can see currently test1 and test3 fail on formatting (they add extra characters) and test2 "just" fails and doesn't serialize anything

Here's the test scenario copied to the new sheet + expected output. This is with the exports settings as in my previous post, basically this: https://gyazo.com/e2a1995f6c1fc0151014df1f0b75b670

https://docs.google.com/spreadsheets/d/13EOItmPrBMxM4FXeacLvXIsWqoDUZpFracM6r_Ey6y8/edit#gid=0

Oh, hate to ask but got to :( Do you know how much time this might take?

Synthoid commented 4 years ago

Before I make an estimate, does this JSON work for what you are wanting?

{
  "Items": [
    {
      "id": "Sword",
      "upgradePrices": [
        {
          "Price": 5
        },
        {
          "Price": 10
        }
      ]
    },
    {
      "id": "Spear",
      "upgradePrices": [
        {
          "Price": 10
        }
      ]
    }
  ]
}
MikeSzym commented 4 years ago

YES!

The actual object can be much larger than price, but this is exactly what I asked about :) list of JSON objects within a JSON object

MikeSzym commented 4 years ago

Oj, just noticed that there was some update to the source and quickly tested:

https://gyazo.com/6fc23138dce1b0d4fc2f05d6af99b240

List of objects (JA_test2) does not work and is empty, but placing a complete JSON object like in test3 works. While test2 option would be the "cleaner" one, the test3 option is definitely workable for what we need :)

Thank you!

Synthoid commented 4 years ago

Happy to help! If you don't mind, would you be able to post a link to the game you are working on? (if not now for NDA reasons, after release if fine) As a game developer myself, I always take an interest in other developers using ESD and like to see games that use it!

As for the formatting, the square brackets were the easiest way to handle exporting arrays without me having to write my own JSON parser (which I started to do before thinking of a faster, more universal solution). I do still recommend using Nested Elements instead of manually writing JSON in cells, but this should allow developers a little extra control over their data without being as complicated as Nested Elements.

In any case, I hope you are able to move forward without ESD blocking you now. Good luck on the game!

MikeSzym commented 4 years ago

Damn, it was working ~14 hours ago and now it's not working again. Using same PC, same browser, same sheet, same export settings and suddenly get such output again...

https://gyazo.com/051873bcf5b808777c476d32b249e943 

Any idea what might have happened? Here are the objects that I'm trying to export, they are proper valid JSONs....

https://gyazo.com/1ac9eed9ad9aa2d8c1ecacfccfa6f523

Synthoid commented 4 years ago

It looks like the published version got into a weird state were it should be v57 but is instead using code from v56 for some reason? I'll try re-publishing it to see if that fixes the issue.

Synthoid commented 4 years ago

New version should be pushed again. Not sure what happened there, but it seems like Google automatically rolled the code back to v56 but still listed the version as being 57.