loadJSON always returns Object, never an Array

[bradocchs]

Nature of issue?

• [x] Found a bug
• [ ] Existing feature enhancement
• [ ] New feature request

Most appropriate sub-area of p5.js?

• [ ] Color
• [ ] Core
• [ ] Data
• [ ] Events
• [ ] Image
• [x] IO
• [ ] Math
• [ ] Typography
• [ ] Utilities
• [ ] WebGL
• [ ] Other (specify if possible)

Which platform were you using when you encountered this?

• [ ] Mobile/Tablet (touch devices)
• [x] Desktop/Laptop
• [ ] Others (specify if possible)

Details about the bug:

• p5.js version: any?
• Steps to reproduce this:

data = loadJSON('file.json'); // data is always an Object even when file.json is an Array

loadJson('file.json', callback); function callback(data) { // data is Object or Array }

Line 139 of src/io/files.js - ret is always an Object.

[bradocchs]

The code comments say ret is needed for preload. Can a check be done before it returns to determine if an Array needs returned instead?

[limzykenneth]

I had a look at this but cannot come up with a solution. Checks cannot be done when first defining ret to know if an array or an object will be returned because technically, the data has not been requested yet so there's no way of knowing what comes back.

The part I don't understand is why delaying the initalization of ret to within the callback will end up with ret not being populated...

I've tried delayed initialization, converting from object to array, initialize as array (that case array works but objects cannot be defined) but all of them didn't work. Anyone have an idea?

[lmccart]

JS allows you to modify an object but reassigning or replacing it completely breaks the pointer. So yes, I also do not have a good idea of a solution here. :\

[limzykenneth]

I guess that's the price we pay for trying to make asynchronous javascript synchronous... ☹️

[bradocchs]

I love JS, but sometimes it can be a pain. ;)

If there's not a current fix, can the p5js.org reference page be updated with a note about using the callback for array data?

Thanks for looking into this!

[meiamsome]

Setting ret.__proto__ to the returned array seems to make it behave as if it were the array itself - length, indexing, array prototype functions all work - even for..in and for..of seem to work on latest chrome and firefox. Probably not the best to mess around with the prototype hierarchy unless this is how it should actually respond. Anyone well versed in this stuff?

[erikbuunk]

I used Object.values(data) to turn the Object to an Array. Is this a solution that could be used, or added to documentation? I still got the Object errors in the current P5 version.

[limzykenneth]

Object.values(data) cannot be used as a solution internally as it still requires redefining the returned object from the function which changes the pointer.

[ffd8]

Just ran into this issue with a student, trying to load a giant JSON array of records from Zotero, which really tripped me up (same as mentioned in #2290). Using the method above, Object.values(data) worked great, but required searching for p5js loadJSON as array to stumble across these issue tips. The reference for loadJSON() clearly states:

Note that even if the JSON file contains an Array, an Object will be returned with index numbers as keys.

Is it possible (maybe already) to set a param to force the returned object to be an array of objects if that's the case? Abstract example of what was being loaded in: [ {}, {}, {}, ... ]. Or an intuitive function to smooth this process?

[limzykenneth]

@ffd8 I've tried everything I can think of but there's just no way to return an array from the function due to the async nature of AJAX and how Javascript handles (or doesn't handle) references.

If you don't mind an extra step to convert the object into a proper array, you can use Array.from() with a mapping function as the second argument shown here.

[ffd8]

@limzykenneth Had a quick look at the loadJSON() function, specifically where items are indexed into ret obj and if it's not too hacky.. wonder if this ret should be manipulated juust before returning it if:

the first and last index key values match their position in key count?

// simulating loadJSON() ret obj
let ret = {};
ret[0] = 'somestring';
ret[1] = 42;
ret[2] = false;
ret[3] = {foo:'bar'};

let keys = Object.keys(ret);
let keysCount = Object.keys(ret).length-1;
console.log(ret); // object with index numbered keyes

// check first and last keys / position
if(keys[0] == 0 && keys[keysCount] == keysCount){
    console.log('match! is indexed array');
    ret = Object.values(ret)
}

console.log(ret); // array as expected
// ... return ret;

or to be safe, cycle entire obj and set a flag if one key doesn't match its position...

// simulating loadJSON() ret obj
let ret = {};
ret[0] = 'somestring';
ret[1] = 42;
ret[2] = false;
ret[3] = {foo:'bar'};

let keys = Object.keys(ret);
let keysCount = Object.keys(ret).length-1;
console.log(ret); // object with index numbered keyes

// check all key / positions
let retArray = true;
let rCounter = 0;
for(r in ret){
    if(r != rCounter){
        retArray = false;
    }
    rCounter++;
}
if(retArray){
    console.log('match! is indexed array');
    ret = Object.values(ret)
}

console.log(ret); // array as expected
// ... return ret;

[limzykenneth]

The problem is that the object will be returned before any info about the requested JSON is available, so we won't have anyway to make any checks that relies on any values returned from the AJAX request before returning the ret object, and once the data has returned, ret cannot be reassigned, ie. ret = Object.values(ret) will not work as this reassignment will not be reflected in the return value of loadJSON().

In other words, return ret will always return {} because when it returns, we don't know anything about the data that is being loaded in. The only way that I can think of that this can potentially be solved is to somehow transform the object into an array without reassignment which I tried by modifying the object prototype after the data has been loaded but only got a quasi array (with some of the qualities of actual arrays) that I think will cause more confusion that if it isn't an array at all.

[neiraRail]

Just read this anwer in stack overflow: https://stackoverflow.com/a/40837276, that propose to change the origin JSON from an array to an object containing an array and then taking the array out of the object returned by loadJson(), is there a way to do something like this under the hood?

[limzykenneth]

@neiraRail Since that relies on the origin sending back an object wrapped array, there is no way to do this if the origin just send back an array. The fundamental problem is that a regular object cannot be made into an array without losing the original reference to it.

[davepagurek]

Would it be more useful if, when the data we fetch is a JSON type other than an object (array, number, string, etc) put it in the object under some default key like data? So an array would become { data: [ ... ] } rather than { 1: ..., 2: ..., 3: ... }. It's probably still unexpected, but it might be a bit easier to deal with.

Another option which is probably a bad idea but I'll just throw it out there is to alter the original object's prototype by doing Object.setPrototypeOf(ret, Array.prototype). It's probably a bad idea because although it actually mostly works, I can't get Array.isArray to work.

const ret = {}

// Add some array properties
ret[0] = 'a'
ret[1] = 'b'
ret[2] = 'c'
ret.length = 3
Object.setPrototypeOf(ret, Array.prototype)

// Array methods work
console.log(ret.reduce((a, b) => a + b)) // 'abc'
console.log(ret.map((a) => `Letter ${a}`)) // ['Letter a', 'Letter b', 'Letter c']

// instanceof works
console.log(ret instanceof Array) // true

// Array.isArray does not work :'(
console.log(Array.isArray(ret)) // false

[limzykenneth]

Would it be more useful if, when the data we fetch is a JSON type other than an object (array, number, string, etc) put it in the object under some default key like data? So an array would become { data: [ ... ] } rather than { 1: ..., 2: ..., 3: ... }. It's probably still unexpected, but it might be a bit easier to deal with.

This approach will be a breaking change and is somewhat inconsistent with the object return.

Another option which is probably a bad idea but I'll just throw it out there is to alter the original object's prototype by doing Object.setPrototypeOf(ret, Array.prototype). It's probably a bad idea because although it actually mostly works, I can't get Array.isArray to work.

I've explored all kinds of methods relating to this but none work fully. Instead of returning a pseudo array that sometimes behaves like an array and sometimes doesn't (super confusing), I'd prefer using Object.values() nowadays after receiving the value.