Hayden User Blog

[sky-cake]

Hi @bbepis, I have just started using the Hayden Scraper again for archiving select 4chan threads with the Hayden database schema. It is working very well 😄 Thank you for this great tool!

I wanted to share some information which could help other users (and myself) in the future.

Hayden Version: (not sure where to find this) OS: Ubuntu Server 22 LTS MySQL: 8.0.34 .NETCore: 6.0.18

/etc/systemd/system/hayden.service

[Unit]
Description=Hayden Scraper
After=network-online.target mysql.service

[Service]
Type=simple
ExecStart=/mnt/hayden_asagi/Hayden scrape /mnt/hayden_asagi/config.json
WorkingDirectory=/mnt/hayden_asagi
User=m
Group=www-data
Restart=always
RestartSec=600
StandardOutput=null #append:/home/user/hayden_info.log
StandardError=append:/home/user/hayden_error.log
SyslogIdentifier=hayden

[Install]
WantedBy=multi-user.target

/mnt/hayden_asagi/config.json

{
    "source": {
        "type": "4chan",
        "boards": {
            "g": {
                "AnyFilter": "battlestation",
                "AnyBlacklist": "stable diff|dall.*e.*3"
            },
            "ck": {}, # download everything
        },
        "apiDelay": 5.5,
        "boardScrapeDelay": 45
    },
    "readArchive": false,
    "proxies": [],
    "consumer": {
        "type": "Asagi",
        "databaseType": "MySQL",
        "connectionString": "Server=127.0.0.1;Port=3306;Database=hayden;Uid=USER;Pwd=PASSWORD;",
        "downloadLocation": "/mnt/ayase_quart/src/static/hayden_asagi",
        "fullImagesEnabled": true,
        "thumbnailsEnabled": true
    }
}

My Hayden Scraper instance has run for 3 days now. I can confirm that it will continue archiving existing threads after several hours downtime -- restarting the Hayden Scraper service is no issue.

I gtg now, but I plan to add to this blogpost. I've also added some Hayden Scraper instructions at https://github.com/sky-cake/ayase-quart#hayden. Let me know what other information I should include here.