issues
search
USCDataScience
/
sparkler
Spark-Crawler: Apache Nutch-like crawler that runs on Apache Spark.
http://irds.usc.edu/sparkler/
Apache License 2.0
411
stars
143
forks
source link
issues
Newest
Newest
Most commented
Recently updated
Oldest
Least commented
Least recently updated
Actually do a release
#119
buggtb
opened
7 years ago
1
[MEMEX] Create a FileDumper tool
#118
sujen1412
closed
6 years ago
1
Merging branch memex of fork with branch memex-dd of main Sparkler repo
#117
sujen1412
closed
7 years ago
0
[MEMEX] Store segment path of url content in SOLR
#116
sujen1412
closed
6 years ago
0
Define config file as cmd line flag
#115
buggtb
closed
3 years ago
0
Define plugin dir as a cmd line flag
#114
buggtb
closed
6 years ago
1
Catch missing plugins
#113
buggtb
closed
6 years ago
1
SPKLR-111: Bash script implementing an interactive tutorial
#112
giuseppetotaro
closed
7 years ago
3
Script Tutorial for Kick Start
#111
giuseppetotaro
closed
7 years ago
0
Sparkler-93: YAML backed config with Schema validation
#110
thammegowda
closed
3 years ago
4
SparklerConfig upgrade
#109
sk-s-hub
closed
7 years ago
7
Update CONTRIBUTING.md to include commit comments instructions
#108
sujen1412
closed
7 years ago
1
Add tutorial for adding a new plugin
#107
sujen1412
closed
6 years ago
1
[NUTCH][MEMEX] Add cosine similarity plugin
#106
sujen1412
closed
3 years ago
0
[MEMEX] Define a scoring plugin interface
#105
sujen1412
closed
6 years ago
1
Extractor of fields using xpath or css selectors and map them to Solr fields?
#104
mzeidhassan
opened
7 years ago
3
[MEMEX] Introduce REST API for Sparkler
#103
sujen1412
closed
3 years ago
3
behavior driven development (BDD) for sparkler
#102
thammegowda
closed
6 years ago
1
Sparkler-94 Github templates for Pull requests and Issues, and contribution guidelines
#101
thammegowda
closed
7 years ago
0
fetcher.htmlunit - Fetcher plugin based on HTML Unit
#100
thammegowda
closed
7 years ago
8
Add buggtb to dockerhub collaborators
#99
buggtb
closed
7 years ago
2
URL validator used in injector is too strict
#98
thammegowda
opened
7 years ago
0
Attach Sparkler to Travis
#97
buggtb
closed
7 years ago
2
Add a Gitter chat badge to README.md
#96
gitter-badger
closed
7 years ago
0
Sparkler Configuration Update
#95
sk-s-hub
closed
7 years ago
13
Create guidelines and templates for issues and PRs using github templates
#94
thammegowda
closed
7 years ago
2
Revamp Sparkler Config - Type checking + Validation + *-site.yaml overriding
#93
thammegowda
opened
7 years ago
6
Running iterations more than one in crawl gives null pointer exceptions sometimes.
#92
sk-s-hub
closed
7 years ago
4
ScanDirectory FIXME in Injector.scala
#91
sk-s-hub
closed
7 years ago
2
Parse and Index Response headers
#90
thammegowda
closed
7 years ago
2
Store Response Headers
#89
thammegowda
closed
7 years ago
0
Implement URL Filter plugin to restrict crawl to the same domain and same host
#88
thammegowda
closed
6 years ago
1
crawling all links of same domain in Sparkler - best practice
#87
rainergeissendoerfer
closed
7 years ago
2
No Terms displayed in Banana
#86
rainergeissendoerfer
closed
7 years ago
2
sparkDriver Exception while crawling
#85
adityardesai
closed
7 years ago
3
Exception while configuring with Solr 6.4
#84
adityardesai
closed
7 years ago
5
Configure stopwords in solr/crawldb
#83
thammegowda
closed
7 years ago
0
Default Headers + rotating user agents
#82
thammegowda
closed
7 years ago
3
Dockler : Sparkler-Docker Cli tool
#81
thammegowda
closed
7 years ago
1
Fix for #78 and #79
#80
karanjeets
closed
6 years ago
10
Catch exception(s) when filtering outlinks
#79
karanjeets
closed
6 years ago
1
Add content limit in the default fetcher
#78
karanjeets
closed
6 years ago
0
Felix version conflicts fixed AND option to disable plugins
#77
thammegowda
closed
7 years ago
2
Felix Version issue
#76
thammegowda
closed
7 years ago
2
Cannot talk to ZooKeeper - Updates are disabled
#75
karanjeets
closed
6 years ago
0
Solving issues #63 and #65
#74
karanjeets
closed
7 years ago
3
Deduplication of Solr Documents
#73
karanjeets
closed
7 years ago
2
De-Duplicate documents in CrawlDB (Solr)
#72
karanjeets
closed
7 years ago
16
*_md fields generated by parser are not present in the Solr schema
#71
karanjeets
closed
7 years ago
0
Felix Plugins : Option to enable and disable easily
#70
thammegowda
closed
7 years ago
3
Previous
Next