ScrapeGraphAI / Scrapegraph-ai

Python scraper based on AI
https://scrapegraphai.com
MIT License
14.41k stars 1.17k forks source link

Can you collect user usage data from within the library and write it to readme #665

Closed myt2000 closed 4 days ago

myt2000 commented 4 days ago

I found that when using Scrapegraph-ai library, the code collects data from users using the library. If you need to collect data, can you write it in readme

code: https://github.com/ScrapeGraphAI/Scrapegraph-ai/blob/main/scrapegraphai/telemetry/telemetry.py

VERSION = importlib.metadata.version("scrapegraphai") STR_VERSION = ".".join([str(i) for i in VERSION]) HOST = "https://eu.i.posthog.com" TRACK_URL = f"{HOST}/capture/" # https://posthog.com/docs/api/post-only-endpoints API_KEY = "phc_orsfU4aHhtpTSLVcUE2hdUkQDLM4OEQZndKGFBKMEtn" TIMEOUT = 2 DEFAULT_CONFIG_LOCATION = os.path.expanduser("~/.scrapegraphai.conf")

myt2000 commented 4 days ago

image Also, is there too much data collected for this?

PeriniM commented 4 days ago

Hey @myt2000! Yes we do collect usage data and it is specified in the documentation here.

We offer many ways to disable it, they are specified both at the top of the script you mentioned and in the docs. I will reference them here:

"""
This module contains code that relates to sending ScrapeGraphAI usage telemetry.

To disable sending telemetry there are three ways:

1. Set it to false programmatically in your driver:
  >>> from scrapegraphai import telemetry
  >>> telemetry.disable_telemetry()
2. Set it to `false` in ~/.scrapegraphai.conf under `DEFAULT`
  [DEFAULT]
  telemetry_enabled = False
3. Set SCRAPEGRAPHAI_TELEMETRY_ENABLED=false as an environment variable:
  SCRAPEGRAPHAI_TELEMETRY_ENABLED=false python run.py
  or:
  export SCRAPEGRAPHAI_TELEMETRY_ENABLED=false
"""

A text is also printed in the console at the beginning of each session using a logger.

Hope this answer satisfies you, let me know!

myt2000 commented 4 days ago

I think you need to write this in the readme.md

PeriniM commented 4 days ago

Sure! Added info to the readme 62912c2