search

gbowne1 / json-maestro

JSONMaestro is a powerful tool designed for cleaning and processing JSON-like files. It simplifies tasks such as removing comments, eliminating duplicate keys, adding schema keys, and sorting keys. Ideal for developers working with configuration files and API responses, JSONMaestro enhances data integrity and prepares JSON data for further analysis
MIT License
3 stars 2 forks source link

json.load automatically removes duplicate keys in valid json #22

Closed shoshta73 closed 3 weeks ago

shoshta73 commented 3 weeks ago

I have notice that when loading json from string

example:

{
    data: "foo",
    key: "bar",
    data: "baz",
}

json.load will automatically put data=baz into the dictionary

shoshta73 commented 3 weeks ago

So are we dropping remove_duplicate_keys function out becuase json.load already takes in last instace of the key as final value?

gbowne1 commented 3 weeks ago

Well, it should in any particular case, be able to remove any duplicate key unless something could be set in cases where json and jsonc arrays and objects and other permutations of this, to keep it from removing any necessary duplicate item.

I would avoid duplicating functionality.

shoshta73 commented 3 weeks ago

Reopening this issue after cheking v8 parser implementation.

So currently as per implentation of AST of javascript, when duplicate keys are encountered in javascript object, no error is reported. Given that JSON is basically subset of javascripts AST used for representing objects without their respective methods atached it handles duplicate keys in same as used javascript parser does.

So my conclusion is that "duplicate key error" message is just a LSP giving a person warning about having duplicate keys.

In my opinion removing remove_duplicate_keys would resolve this issue.

gbowne1 commented 3 weeks ago

we should look into some sort of pythonesque CI/CD.

gbowne1 commented 3 weeks ago

I agree with your comments here 100%

shoshta73 commented 3 weeks ago

we should look into some sort of pythonesque CI/CD.

So more of like buildbot actions rather than GH styled workflows?

gbowne1 commented 3 weeks ago

I haven't extensively looked at actions/workflows for Python stuff, but yes.

This was recommended:

https://www.youtube.com/watch?v=qznIeQBMTyM https://www.youtube.com/watch?app=desktop&v=WTofttoD2xg

https://docs.github.com/en/actions/use-cases-and-examples/building-and-testing/building-and-testing-python

https://github.com/resources/articles/devops/ci-cd

https://packaging.python.org/en/latest/guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows/