search

Giveth / feathers-giveth

Featherjs server for caching giveth data.
MIT License
87 stars 99 forks source link

Add analytics tracker to backend #548

Closed aminlatifi closed 3 years ago

aminlatifi commented 3 years ago

Segment analytics has been added to frontend (https://github.com/Giveth/giveth-dapp/pull/2333) in order to track events like page visits, propose, reject, cancel, etc. of trace, campaign creation, community creation, donate and delegates.

But Brave browser by default blocks connection to segment or other analytic services. So we may need to implement event tracking in backend (feathers). analytics-node package is ready for this purpose and methods can be called like in frontend (giveth-dapp). The essential crux is finding the points in the backend to call the tracking and other methods. https://www.npmjs.com/package/analytics-node

mohammadranjbarz commented 3 years ago

@MoeNick For testing this issue go to https://app.segment.com/giveth-io/sources/trace_development/debugger

then you go to develop.giveth.io with Brave browser you should see the pages and tracking in segment panel ( if you use chrome the events go from your end direct to segment, but if your browser be Brave the analytics events go to our backend and then will go to segment)

MoeNick commented 3 years ago

Thanks @mohammadranjbarz for the awesome job. Hey @divine-comedian Would u please see the result in segment and confirm?

divine-comedian commented 3 years ago

@mohammadranjbarz Do you have a full list of all backend actions that have analytics attached to them?

mohammadranjbarz commented 3 years ago

@mohammadranjbarz Do you have a full list of all backend actions that have analytics attached to them?

All the tracking and pages that used in front, we used them in backend, actually we somehow proxy segment events with our backend, so the events are the same as front send ( if browser is not brave, browser send events directly otherwise browser send the events by feathers-backend)

mohammadranjbarz commented 3 years ago

@divine-comedian but if you wanna list of events that we are sending in front, I can prepare a list for you

divine-comedian commented 3 years ago

Okay I tested all of them, or most of them hopefully and I have a few improvements we'll need to make in order to make this an effective analytics tool.

first things is we'll need to create new objects for the calls that we can use to avoid using nested properties. this caused some issues with identifying information on giveth.io, the solution that we used for flattening properties as an example is

Save Donation


const projectOwner = await User.findOne({ id: Number(project.admin) })

        if (projectOwner) {

          await tokenValues

          analytics.identifyUser(projectOwner)

          const segmentDonationReceived = {
            email: projectOwner.email,
            title: project.title,
            firstName: projectOwner.firstName,
            projectOwnerId: project.admin,
            slug: project.slug,
            amount: Number(amount),
            transactionId: transactionId.toString().toLowerCase(),
            transactionNetworkId: Number(transactionNetworkId),
            currency: token,
            createdAt: new Date(),
            toWalletAddress: toAddress.toString().toLowerCase(),
            fromWalletAddress: fromAddress.toString().toLowerCase(),
            donationValueUsd: donation.valueUsd,
            donationValueEth: donation.valueEth
          }

          analytics.track(
            'Donation received',
            projectOwner.segmentUserId(),
            segmentDonationReceived,
            projectOwner.segmentUserId()
            )
        }

so we specify all the information we want to capture and assign an object with flattened property names that get the information we want. It should come out clean on the segment side like this: 

analytics.track({
  anonymousId: 'givethId-161',
  userId: 'givethId-161',
  event: 'Donation received',
  properties: {
    amount: 0.001,
    createdAt: new Date('2021-07-08T12:41:36.526Z'),
    currency: 'ETH',
    donationValueEth: 0.001,
    donationValueUsd: 2.17127,
    email: 'mrnikkhah@gmail.com',
    firstName: 'Moe',
    fromWalletAddress: '0x62f49a875f09c82e104197afbb8a30e66bb96ba3',
    projectOwnerId: '161',
    slug: 'myfirstprojectonio',
    title: 'MyFirstProjectOnIO',
    toWalletAddress: '0xf3bab2f1add2af035f489d7f3fb45a2411ed89ba',
    transactionId: '0x0ddd1a41af785ea73a54073a669456303adafba99f53183324fa2911043b656c',
    transactionNetworkId: 3
  }
});
divine-comedian commented 3 years ago

Second I noticed is that we'll need to gather the USD and ETH values for transactions in standardized property names, where applicable,so we're able to analyze the movement of funds in whole terms. For example in the comment above we send back the property donationValueEth and donationValueUsd.. something similar will be extremely important for analytics.

divine-comedian commented 3 years ago

I have made a list for each event type for the information I think we should track in order to make more efficient analytics gathering.

A couple of events during testing however I noticed did not produce analytics events

Reach out to me if you want assistance, I would love to help out and get my hands involved in finishing this implementation!

mohammadranjbarz commented 3 years ago

Okay I tested all of them, or most of them hopefully and I have a few improvements we'll need to make in order to make this an effective analytics tool.

first things is we'll need to create new objects for the calls that we can use to avoid using nested properties. this caused some issues with identifying information on giveth.io, the solution that we used for flattening properties as an example is

Save Donation


const projectOwner = await User.findOne({ id: Number(project.admin) })

        if (projectOwner) {

          await tokenValues

          analytics.identifyUser(projectOwner)

          const segmentDonationReceived = {
            email: projectOwner.email,
            title: project.title,
            firstName: projectOwner.firstName,
            projectOwnerId: project.admin,
            slug: project.slug,
            amount: Number(amount),
            transactionId: transactionId.toString().toLowerCase(),
            transactionNetworkId: Number(transactionNetworkId),
            currency: token,
            createdAt: new Date(),
            toWalletAddress: toAddress.toString().toLowerCase(),
            fromWalletAddress: fromAddress.toString().toLowerCase(),
            donationValueUsd: donation.valueUsd,
            donationValueEth: donation.valueEth
          }

          analytics.track(
            'Donation received',
            projectOwner.segmentUserId(),
            segmentDonationReceived,
            projectOwner.segmentUserId()
            )
        }

so we specify all the information we want to capture and assign an object with flattened property names that get the information we want. It should come out clean on the segment side like this:

analytics.track({
  anonymousId: 'givethId-161',
  userId: 'givethId-161',
  event: 'Donation received',
  properties: {
    amount: 0.001,
    createdAt: new Date('2021-07-08T12:41:36.526Z'),
    currency: 'ETH',
    donationValueEth: 0.001,
    donationValueUsd: 2.17127,
    email: 'mrnikkhah@gmail.com',
    firstName: 'Moe',
    fromWalletAddress: '0x62f49a875f09c82e104197afbb8a30e66bb96ba3',
    projectOwnerId: '161',
    slug: 'myfirstprojectonio',
    title: 'MyFirstProjectOnIO',
    toWalletAddress: '0xf3bab2f1add2af035f489d7f3fb45a2411ed89ba',
    transactionId: '0x0ddd1a41af785ea73a54073a669456303adafba99f53183324fa2911043b656c',
    transactionNetworkId: 3
  }
});

Sorry but I didn't get your point, It would be great if we talk in DM about it

mohammadranjbarz commented 3 years ago

I have made a list for each event type for the information I think we should track in order to make more efficient analytics gathering.

A couple of events during testing however I noticed did not produce analytics events

  • Deleted TRACE
  • propose delegattion from Community to Campaign
  • accept delegation from Community to Campaign

Reach out to me if you want assistance, I would love to help out and get my hands involved in finishing this implementation!

It's ok, I'll add these events

divine-comedian commented 3 years ago

Following a quick call with mohammad this morning I'll move the issue for flattening and improving the analytics to a new issue. Tag me @mohammadranjbarz when you have implemented the missing track calls and I will test these again!

mohammadranjbarz commented 3 years ago

Following a quick call with mohammad this morning I'll move the issue for flattening and improving the analytics to a new issue. Tag me @mohammadranjbarz when you have implemented the missing track calls and I will test these again!

it's implemented in https://github.com/Giveth/giveth-dapp/pull/2374 you can test it locally, or wait after the PR has been merged to develop I'll let you know

divine-comedian commented 3 years ago

Okay I finished testing for the 3 missing events

good to go @mohammadranjbarz !!!

mohammadranjbarz commented 3 years ago

Okay I finished testing for the 3 missing events

  • propose delegation from community to campaign
  • commit delegation from community to campaign (as donor)
  • delete trace

and they all work now! brave browser on develop.giveth.io

good to go @mohammadranjbarz !!!

Thanks @divine-comedian you helped a lot on this story