Big Data Harvest analysis / POC

nasa-pds-engineering-node / pds-registry-app

(deprecated) See https://github.com/NASA-PDS/registry for new and improved capability.

https://nasa-pds.github.io/registry/

Other

1 stars 1 forks source link

Big Data Harvest analysis / POC #195

Closed tloubrieu-jpl closed 2 years ago

tloubrieu-jpl commented 2 years ago

See slides Big Data Harvest.pdf

tloubrieu-jpl commented 2 years ago

@tdddblog will present a development plan for the POC next Thursday, this should include:

crawler
message queue manager
harvest/registry-mgr
elasticSearch

So far @tdddblog has deployed rabbitmq but noticed that activeMQ supports JMS protocol which is compliant with AWS SQS. So he will try to deploy activeMQ.

tdddblog commented 2 years ago

Big Data Harvest.pdf