-
An example is the url
http://www.tyres-pneus-online.co.uk/car-tyres/PIRELLI/
that contains lines like
class="listeProduitGris">P2500 FOUR SEASONS (4S)
Regards
Matteo
http://github.com/matteore…
-
For better scalability, ebot should be a real scalable NOSQL database
Couchdb is more a replicated than a distribuited NOSQL (document) database...
Switching to riak? apache cassandra? Disco?
-
now ebot works only with a couchdb server running in localhost
hostname, user and password should be put in priv/ebot_db.conf
-
now ebot works only with a amqp server running in localhost
hostname, user and password should be put in priv/ebot_amqp.conf
-
it would be useful to reload crawlers options without restarting ebot...
-
Hello,
I would like to retreive with couchbeam library the info you can get with an http get request to http://localhost:5984/ebot
I
{"db_name":"ebot","doc_count":78849,"doc_del_count":0,"update_seq"…
-
it would be usefull adding at configuration level any custom function to be executed when a body page is visited. the output could be a key/value list that wll be added in the database using ebot_db:u…
-
it could be useful to permit the user to run a custom function for normalizing urls, couldn't it?
-
ebot_web:fetch_url_links( ).
{ok,[,
,
-
removing options embedded in urls. maybe adding url rewrites regexps?
http://www.gettyre.it/motoweb/login_input.action;jsessionid=0D6C52EE922