search

internetarchive / Zeno

State-of-the-art web crawler 🔱
GNU Affero General Public License v3.0
83 stars 11 forks source link

Invalid logging level #147

Open CorentinB opened 2 months ago

CorentinB commented 2 months ago

I'm seeing a lot of DEBUG logs printed to stdout:

time=2024-09-21T09:25:34.468+02:00 level=DEBUG msg="unable to extract URLs from JSON in script tag" error="invalid character 'd' looking for beginning of value" url=https://old.reddit.com/r/PublicFreakout/comments/1fla2ks/another_video_of_israeli_soldiers_throwing/
time=2024-09-21T09:25:34.485+02:00 level=DEBUG msg="unable to extract URLs from JSON in script tag" error="invalid character '$' looking for beginning of value" url=https://old.reddit.com/r/PublicFreakout/comments/1fla2ks/another_video_of_israeli_soldiers_throwing/
time=2024-09-21T09:25:34.486+02:00 level=DEBUG msg="unable to extract URLs from JSON in script tag" error="unexpected end of JSON input" url=https://old.reddit.com/r/PublicFreakout/comments/1fla2ks/another_video_of_israeli_soldiers_throwing/
time=2024-09-21T09:25:34.486+02:00 level=DEBUG msg="unable to extract URLs from JSON in script tag" error="unexpected end of JSON input" url=https://old.reddit.com/r/PublicFreakout/comments/1fla2ks/another_video_of_israeli_soldiers_throwing/
time=2024-09-21T09:25:34.487+02:00 level=DEBUG msg="unable to extract URLs from JSON in script tag" error="unexpected end of JSON input" url=https://old.reddit.com/r/PublicFreakout/comments/1fla2ks/another_video_of_israeli_soldiers_throwing/
time=2024-09-21T09:25:34.487+02:00 level=DEBUG msg="unable to extract URLs from JSON in script tag" error="unexpected end of JSON input" url=https://old.reddit.com/r/PublicFreakout/comments/1fla2ks/another_video_of_israeli_soldiers_throwing/
time=2024-09-21T09:25:34.487+02:00 level=DEBUG msg="unable to extract URLs from JSON in script tag" error="invalid character '\\'' looking for beginning of value" url=https://old.reddit.com/r/PublicFreakout/comments/1fla2ks/another_video_of_israeli_soldiers_throwing/
time=2024-09-21T09:25:34.497+02:00 level=DEBUG msg="unable to extract URLs from JSON in script tag" error="unexpected end of JSON input" url=https://old.reddit.com/r/PublicFreakout/comments/1fla2ks/another_video_of_israeli_soldiers_throwing/
time=2024-09-21T09:25:34.501+02:00 level=DEBUG msg="unable to extract URLs from JSON in script tag" error="unexpected end of JSON input" url=https://old.reddit.com/r/PublicFreakout/comments/1fla2ks/another_video_of_israeli_soldiers_throwing/

Even though the code seems to specify the INFO level for stdout.

    customLoggerConfig := log.Config{
        FileConfig: &log.LogfileConfig{
            Dir:    logfileOutputDir,
            Prefix: "zeno",
        },
        FileLevel:                slog.LevelDebug,
        StdoutEnabled:            !config.NoStdoutLogging,
        StdoutLevel:              slog.LevelInfo,
        RotateLogFile:            true,
        RotateElasticSearchIndex: true,
        ElasticsearchConfig: &log.ElasticsearchConfig{
            Addresses:   config.ElasticSearchURLs,
            Username:    config.ElasticSearchUsername,
            Password:    config.ElasticSearchPassword,
            IndexPrefix: config.ElasticSearchIndexPrefix,
            Level:       slog.LevelDebug,
        },
    }
    if len(config.ElasticSearchURLs) == 0 || (config.ElasticSearchUsername == "" && config.ElasticSearchPassword == "") {
        customLoggerConfig.ElasticsearchConfig = nil
    }

    customLogger, err := log.New(customLoggerConfig)
    if err != nil {
        return nil, err
    }
    c.Log = customLogger