Closed Glazelf closed 2 years ago
@Glazelf I have a JS file I use locally to do the .ndjson conversions, do you think this would be valuable to add to the code base? While it runs without the need of any JS packages, I dont want it to feel like you need any specific programming knowledge to use anything in this repo. It might also be worth wondering if we actually need both formats. Originally I only had the .ndjson file, as this was addapted from a mass export of a personal DB I was using, just then later on added the .json format as it is more widely used.
The $oid are for if inserting into a noSQL db (specifically postgress uses this format, another legacy from my originally mass export). I think that technically these would be auto added in most implementations, but for people who want to use this outside an actual DB, these just provide a unique ID value. I also have a local script to check the uniqueness as well which I can commit to the code base. It might also be worth just updating the key to id
instead, while still ensuring that each id in a unique value.
Come to think of this, I might have a play around with git pipelines with some of my local script to do things like an id check, or auto generate the ndjson file
Massive thank you for all your work. Have been super busy with my real job lately so havn't been able to devote the time I would like to to this
@Glazelf I have a JS file I use locally to do the .ndjson conversions, do you think this would be valuable to add to the code base? While it runs without the need of any JS packages, I dont want it to feel like you need any specific programming knowledge to use anything in this repo. It might also be worth wondering if we actually need both formats. Originally I only had the .ndjson file, as this was addapted from a mass export of a personal DB I was using, just then later on added the .json format as it is more widely used.
The $oid are for if inserting into a noSQL db (specifically postgress uses this format, another legacy from my originally mass export). I think that technically these would be auto added in most implementations, but for people who want to use this outside an actual DB, these just provide a unique ID value. I also have a local script to check the uniqueness as well which I can commit to the code base. It might also be worth just updating the key to
id
instead, while still ensuring that each id in a unique value. Come to think of this, I might have a play around with git pipelines with some of my local script to do things like an id check, or auto generate the ndjson fileMassive thank you for all your work. Have been super busy with my real job lately so havn't been able to devote the time I would like to to this
I don't personally use the ndjson files, I had never heard of them before this repo. You could share the code or add the ndjson file to the PR if you think it's valueable. A git pipeline sounds useful for this for sure haha, avoids double work.
I think regular IDs would provide a more global value over oIDs, especially if oids are automatically generated I don't see the point of having them hardcoded as well.
Closing manually since #35 got merged before linking it to this issue.
Data additions
Technical:
.ndjson
, this file is otherwise unupdated. I can't figure out how to do this, or what it's even for, so it would be nice if someone else could explain/do it for me.$oid
fields. I don't know what these are or what they're for, so someone else might have to do so.