search

ecosystem2 / FastSvelte3Pback

Apache License 2.0
1 stars 0 forks source link

TP testing materials #17

Open DsposalTom opened 10 months ago

DsposalTom commented 10 months ago

Ok starting on materials.

I copied and pasted the json out of the materials.

Again spotted issues with the json.

Fixed those for the file but again it wouldn't load. Using edge. materials.json

This is the console image

DsposalTom commented 10 months ago

Created this file which contains 5 lines and a extra column called notes all filled in, all correct only required, all correct all filled in, all wrong only required, all wrong mismatch materials.csv

Tried to upload it but nothing happened.

Check the console and got these errors.

image

ecosystem2 commented 10 months ago

I think the first error is to do with how Pandas is reading the JSON file. Will look into this, I might need to change how it ingests JSON.

https://bobbyhadz.com/blog/value-error-mixing-dicts-with-non-series-may-lead-to-ambiguous-ordering

ecosystem2 commented 10 months ago

@DsposalTom Wrapping the JSON object in an array as opposed to an object seems to work, this seems to be an issue with Pandas inherited from Python.

By wrapping the JSON object in an array [...], you create a structure that Pandas recognizes as a list of dictionaries (JSON objects). This aligns with the expectation of Pandas when reading JSON data using pd.read_json().

When working with JSON data:

JSON objects {left brace and ends with }right brace correspond to Python dictionaries. JSON arrays [left bracket and ends with ]right bracket correspond to Python lists.

We should decide if we stipulate this format, sending as an array or not?

There is a function that seems to work where we can detect if it is already a list "[ ]" in python or if it needs to be converted into a list. This should allow both formats but needs to be more thoroughly tested for any bugs when validating fields with nested dictionaries or lists.

For example your materials.json is passed successfully for validation when it is structured like this: [ { "identifier": "DCEE1F88-A83B-5BBC-D2D9-6A862B344977", "materialName": "PET", "externalIdentifiers": { "GTIN": "123456789101" }, "materialConstituents": [ "DCEE1F88-A83B-5BBC-D2D9-6A862B344977" ], "combinationPurpose": null, "certification": true, "certificationClaims": [ "1" ], "manufacturedCountry": 826, "updateDate": "01/08/2022" } ]

One potential error however is the validation results for an array seem to omit the field - this needs further investigation but leaving this here for now fyi. "externalIdentifiers": { "GTIN": "123456789101" },