Setup for the whole seosnap stack including dashboard, cache server and cache warmer used for prerendering and full page caching PWA's.
./install.sh
or create a .env file manuallydocker-compose up
# Clone
cd ~
git clone --recursive git@github.com:experius/SeoSnap.git seosnap
# Configure
make install
# Start server
make up
Logs directory ./logs
Cache directory ./cache
Make sure you have created a website via dashboard http://127.0.0.1:8080/seosnap/website/add/
make cachewarmer
make warm A="cache <website id>"
To sync the sitemap pages to the SeoSnap database
make warm A="sync <website id>"
To run the cache warmer multiple time
make thread A="cache <website id>"
Check the nginx.conf in the example folder
check the crontab.txt the example folder and use the cachewarmer.sh & healthchecks.sh
In the dashboard you add the website url along with the website sitemap that you want to make 'SeoSnaps' off.
When the crawler is started it connects with the dashboard api. It uses scrapy to crawl the sitemap. The scrapy results are send to the administration/dashboard. Scrapy requests are send to the cache server. In a similar way that you would do a request to rendertron.
The cache server is a simple file caching server. If a file exist with the content of the page it serves the html from the file. If not, it renders the requested url with rendertron and saves the html output in a file. To refresh the cache the cache-warmer uses PUT requests instead of GET. This will force update from the cache file.
Rendertron + Cache is added and will replace the SeoSnap Cache Server
Recommended Rendertron config has been set in rendertron-config.json