Thanks for everyone who replied! Really helped me.
So all last week i have been trying to setup up this project using AWS.
I have multiple questions about development!
1) Does cosr-ops script provide everything needed for creation of Es clusters and all that worker stuff ?
2) You didnt mention you use rocksdb and gombo, i installed it manually in spark-master.
3) When i try to index test data, by running command make aws_spark_deploy_cosrback, it gives me error:
This happens after ALEXATOP1M is downloaded and it tried to write it to rocksdb.
Traceback (most recent call last):
File "urlserver/import.py", line 21, in <module>
ds.import_dump()
File "./urlserver/datasources/__init__.py", line 62, in import_dump
for i, row in self.iter_dump():
File "./urlserver/datasources/__init__.py", line 102, in iter_dump
f = self.open_dump()
File "./urlserver/datasources/__init__.py", line 144, in open_dump
return GzipStreamFile(f)
File "/cosr/back/venv/src/gzipstream/gzipstream/gzipstreamfile.py", line 62, in __init__
super(GzipStreamFile, self).__init__(self._gzipstream)
File "/usr/lib64/python2.6/io.py", line 921, in __init__
raw._checkReadable()
AttributeError: '_GzipStreamFile' object has no attribute '_checkReadable'
Any ideas ? I have been beating my head on this for a long time.
4) After running make aws_elasticsearch_create and successfully creating ES instances(3 of them). How do i access them ? Whats the port of ES to point frontend to ?
I really hope to finally set everything up, so i can work on the ISSUES( frontend and backend ). I have hight hopes for this project.
Hello,
Thanks for everyone who replied! Really helped me.
So all last week i have been trying to setup up this project using AWS.
I have multiple questions about development!
1) Does cosr-ops script provide everything needed for creation of Es clusters and all that worker stuff ?
2) You didnt mention you use rocksdb and gombo, i installed it manually in spark-master.
3) When i try to index test data, by running command
make aws_spark_deploy_cosrback
, it gives me error:This happens after ALEXATOP1M is downloaded and it tried to write it to rocksdb.
Any ideas ? I have been beating my head on this for a long time.
4) After running
make aws_elasticsearch_create
and successfully creating ES instances(3 of them). How do i access them ? Whats the port of ES to point frontend to ?I really hope to finally set everything up, so i can work on the ISSUES( frontend and backend ). I have hight hopes for this project.