axiomhq / axiom-js

Official language bindings and library extensions for Axiom
https://axiom.co
MIT License
97 stars 14 forks source link

Axiom- js: Option to flatten data when ingesting #46

Open prokopec-simon opened 1 year ago

prokopec-simon commented 1 year ago

Hi, I have a minor issue with Axiom-js. The fact that data ingesting uses basic JSON.stringify to convert data to JSON means it fails in cases where it runs into a circular JSON situation. This situation is not something super rare, exceptions or request logging often contains circular structures.

One way to deal with it is to flatten the JSON before ingesting. Would it be possible to add an option to flatten the data if needed? I don't think it would cause any issue if it flattened it by default, but I would have to check the possible performance change. The best option, to me, seems to be having a flattening option in the IngestOptions next to the timestamp options and the CSV delimiter here.

To replicate the issue:

axiom = new Axiom({
    token: process.env.AXIOM_TOKEN,
    orgId: process.env.AXIOM_ORG_ID,
  });

  async circularJsonLog() {
    const obj = { prop1: 'Test', prop2: null };
    obj.prop2 = obj;

    this.axiom.ingest('logs', obj);
  }

I would be willing to create a PR but I reckon the JSON serialization is on multiple places and discussing it with someone would probably take a lot more time than someone changing it themselves.

Thank you.

dasfmi commented 1 year ago

hi @prokopec-simon. sorry for my late reply. The json serialization is in two places, webVitals and logger, I believe we don't need to do that for webVitals. would be happy to review your PR on the logger json serialization.

prokopec-simon commented 1 year ago

Any chance anyone could have a look at the mentioned PR?

dasfmi commented 1 year ago

hi @prokopec-simon , sorry it took so long, I reviewed your change, looks great, I left a comment, we need to make sure the build step works.