taojing2002
commented
2 years ago Hi Peter:
Thank you for your thoughts about this. I am thinking the first step is to set up solr on K8s. Running indexer on k8s needs the code modification and it should be the next step. I think we need three solr instances on k8s:
- cn-sandbox-solr
- cn-stage-solr
- cn-production-solr
The first two would be in the dev cluster and the last one would be in the production cluster.
I checked the sizes of the directory of /var/solr on cn-stage-ucsb-1 and cn-ucsb-1. They are 47G and 179G respectively. So I think 100G will be good for the test clusters and 500G for the production cluster. What do you think?
Thanks,
Jing
On Wed, Jan 19, 2022 at 10:12 AM Peter Slaughter @.***> wrote:
@mbjones https://github.com/mbjones @taojing2002 https://github.com/taojing2002
What are the steps needed to setup k8s for CN indexing? I'll start with a few basic items:
- determine k8s namespace and setup authentication
- setup persistent volume
- how much space is needed?
- should it use a static CephFS subvolume or dynamic RBD volume?
- install Solr Cloud on k8s
- this Lucid doc https://lucidworks.com/post/running-solr-on-kubernetes-part-1/ shows a setup for Google GKE, but this could be adapted for our bare-metal setup.
- how will indexing be split among k8s pods?
— Reply to this email directly, view it on GitHub https://github.com/DataONEorg/k8s-cluster/issues/22, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB5QQDBMW54JO6LWG44SHX3UW35JLANCNFSM5MKUO3BA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you were mentioned.Message ID: @.***>
@mbjones @taojing2002
What are the steps needed to setup k8s for CN indexing? I'll start with a few basic items: