cedricium
commented
6 years ago While browsing Indie Hackers (https://www.indiehackers.com), I came across a product called microlink. It's actually open-source 1, and what they provide is a way to find out all relevant information for a given url, such as title, description, author, icons, etc. Not only that, they also have two additional and amazing features: screenshots and palettes. The screenshots are stored in Imgur (which always exist unless asked to be deleted). 2
With this one service (the free tier allows for 500 requests a day!), the submission process on Firefly for adding a new site can be as simple as asking for a url, and all else would be fetched thanks to microlink.
Solution: Simply the submission process by using microlink's service.
- Client: ask user for url of new site, then submit
- Server: using url provided by the user, make an API call to microlink, requesting the following:
- title
- description
- screenshot (
full_page=true) - logo (maybe)
- the response from the microlink call can then be
.pipe()to theclient/3
Refs: 1: @microlinkhq 2: https://docs.microlink.io/api/#api-parameters/screenshot 3: External API call with Node.js + Express.js
Currently, the user provides the majority of the info regarding the site when adding a new one (they must add
title,url,screenshot, theircountry(?), andtags). Other of the webde(v | sign) collection sites also have the user include some info, but most of it gets automatically generated in the backend.I'll leave this open for now since its kinda late and I don't want to make a decision at the moment, but I am leaning towards using meta-scrapers and a site screenshot API to get most of the info for a site just based off of a url.