Cosmos
This project is part of FIWARE.
Cosmos is the code name for the Reference Implementation of the BigData Generic Enabler of FIWARE, a set of tools and developments helping in the task of enabling a Hadoop as a Service (HasS) deployment:
- A set of administration tools such as HDFS data copiers and much more, under cosmos-admin folder.
- An OAuth2 tokens generator, under cosmos-auth folder.
- A web portal for users and accounts management, running MapReduce jobs and doing I/O of big data, under cosmos-gui folder.
- A custom authentication provider for Hive, under cosmos-hive-auth-provider.
- A REST API for running MapReduce jobs in a shared Hadoop cluster, under cosmos-tidoop-api.
- A specific OAuth2-base proxy for Http/REST operations, under cosmos-proxy.
If you want to use Cosmos Global Instance in FIWARE Lab
If you are looking for information regarding the specific deployment of Cosmos Global Instance in FIWARE Lab, a HaaS ready to use, please check this documentation:
- Quick Start Guide for Cosmos users.
- Details on using OAuth2 tokens as authentication and authorization mechanism.
- Details on using WebHDFS REST API for data I/O (you can also check this link).
- Details on using Tidoop REST API for MapReduce job submission.
- Details on developing MapReduce jobs and Hive clients (Already developed Hive clients can also be found here).
- In general, you may be insterested in the User and Programming Guide, also available in readthedocs.
If you want to deploy and use your own private Hadoop instance
This is the case you don't rely on the Global Instance of Cosmos in FIWARE Lab. In this case, you'll have to install, configure and manage your own Hadoop private instance. The Internet is plenty of documentation that will help you.
If you want to deploy your own public Cosmos instance
In the (extremly rare) case you are not interested in using the Global Instance of Cosmos or a private instance of Hadoop, but you want to become a Big Data service provider, and you want to base on Cosmos software, you may be interested in the following links:
- Deployment details for administrators trying to replicate Cosmos Global Instance in FIWARE Lab.
- In general, you may be insterested in the Installation and Administration Guide, also available in readthedocs.
Reporting issues and contact information
There are several channels suited for reporting issues and asking for doubts in general. Each one depends on the nature of the question:
- Use stackoverflow.com for specific questions about this software. Typically, these will be related to installation problems, errors and bugs. Development questions when forking the code are welcome as well. Use the
fiware-cosmostag. - Use ask.fiware.org for general questions about FIWARE, e.g. how many cities are using FIWARE, how can I join the accelerator program, etc. Even for general questions about this software, for instance, use cases or architectures you want to discuss.
- Personal email:
- francisco.romerobueno@telefonica.com [Main contributor]
- fermin.galanmarquez@telefonica.com [Contributor]
- german.torodelvalle@telefonica.com [Contributor]
- pablo.coellovillalba@telefonica.com [Contributor]
NOTE: Please try to avoid personally emailing the contributors unless they ask for it. In fact, if you send a private email you will probably receive an automatic response enforcing you to use stackoverflow.com or ask.fiware.org. This is because using the mentioned methods will create a public database of knowledge that can be useful for future users; private email is just private and cannot be shared.