search

clear-datacenter / plan

MIT License
46 stars 17 forks source link

存储 #6

Open wanghaisheng opened 8 years ago

wanghaisheng commented 8 years ago

中文文档 https://github.com/shoudi/alluxio/tree/832b925a1359e45c348c40938c490ebcd28df3f6/docs/cn

https://github.com/Alluxio/alluxio

wanghaisheng commented 8 years ago

http://www.infoq.com/cn/news/2016/06/LinkedIn-Ambry LinkedIn 开源其分布式对象存储系统 Ambry

wanghaisheng commented 8 years ago

这是个存储测试

wanghaisheng commented 8 years ago

世界上功能最牛X的开源数据库PostgreSQL，有个功能叫FDW，你一定得要了解。FDW，也就是PostgreSQL中的外部表功能，当前支持MongoDB、Redis、MySQL、SQL Server、Oracle、Hadoop、Hive、Elastic Search等近30种不同的外部数据源：O https://yq.aliyun.com/articles/53760

wanghaisheng commented 8 years ago

云端存储的基本技巧和上云实践 https://yq.aliyun.com/articles/53760 @ndwx325 把这份ppt前面的一些对比收录一下

wanghaisheng commented 8 years ago

微信自研生产级paxos类库PhxPaxos实现原理介绍 http://mp.weixin.qq.com/s?__biz=MzI4NDMyNTU2Mw==&mid=2247483695&idx=1&sn=91ea422913fc62579e020e941d1d059e#rd

wanghaisheng commented 8 years ago

@ndwx325 如何使用国内源部署Ceph？ https://mp.weixin.qq.com/s?__biz=MzA3NjkwNjM4Nw==&mid=2651867716&idx=1&sn=141917797cfb69fb5c00aab016f445b2&scene=0&key=77421cf58af4a6531a6222efbc182ff718bbe58ce5480de2ff6f5725015cdaaf01e233f9412b26831bf7ea7549bfb90c&ascene=7&uin=Njg5NTc0NDYx&devicetype=iPhone+OS9.3.2&version=16031610&nettype=WIFI&fontScale=100&pass_ticket=SNnZQJgU1JiQkut6YgudGsHrmzk0J6oaovmvJ3RnwIT5Es1ng7P9YvKiPgwcQgak

wanghaisheng commented 8 years ago

[good]//@唐僧_huangliang:[good] //@杨奇龙: 回复@唐僧_huangliang: 恩，大致的意思是rsync 同步文件的bug,新的解决方式使用 no-cache 命令 或者 给rsync 打一个补丁(对于构建自有yum源的运维团队要考虑自己维护rysnc rpm )，加上 --drop-cache 命令解决了，效果非常明显 。不然内存free 显示为0 的 @杨奇龙 使用 rsync 备份MySQL 数据的时候，遇到 内存被完全占用，可以看看这篇 https://bugzilla.samba.org/show_bug.cgi?id=9560#c3

@ndwx325

wanghaisheng commented 8 years ago

图像存储

【Dropbox open sources its new lossless Middle-Out image compression algorithm】O网页链接 Dropbox开源其新的无损压缩算法Lepton，它可以15M/s速度平均22%的压缩比压缩图像。

https://blogs.dropbox.com/tech/2016/07/lepton-image-compression-saving-22-losslessly-from-images-at-15mbs/

wanghaisheng commented 8 years ago

林子雨老师团队经过两个月奋战，终于把100多页的经典英文论文《Architecture of a Database System》翻译成了中文版，与广大数据库爱好者共享，无论是翻译还是校对，都花了大量时间。请到厦门大学数据库实验室网站下载全文，分别提供了网页版和PDF版。论文中文版的网址是O http://dblab.xmu.edu.cn/post/architecture-of-a-database-system/

wanghaisheng commented 8 years ago

https://github.com/EnterpriseDB/mysql_fdw PostgreSQL foreign data wrapper for MySQL

wanghaisheng commented 8 years ago

数据加密 https://github.com/openstack/Barbican

wanghaisheng commented 8 years ago

http://blog.csdn.net/skdkjzz/article/details/41046837 ceph存储 ceph整体学习记录(未整理较乱)

ceph存储 分布式系统设计系列 -- 基本原理及高可用策略 http://blog.csdn.net/skdkjzz/article/details/50737241

wanghaisheng commented 8 years ago

ceph 卷与rancher的整合 https://github.com/rancher/rancher/issues/1395 https://github.com/rancher/rancher/issues/4873 https://github.com/rancher/rancher/issues/117 https://github.com/rancher/rancher/issues/2574 是当成备份还是直接用作数据卷

http://tonybai.com/2016/11/07/integrate-kubernetes-with-ceph-rbd/ 使用Ceph RBD为Kubernetes集群提供存储卷 @ndwx325 @zhuguangqiang 后续可以试一下这个 ceph和单机的k8s rancher里跑的k8s

wanghaisheng commented 7 years ago

http://debezium.io/docs/tutorial/ oracle 连接器 https://issues.jboss.org/browse/DBZ-137 https://streamsets.com/documentation/datacollector/latest/help/#Origins/OracleCDC.html https://github.com/streamsets/datacollector https://github.com/streamsets/tutorials/tree/master/sample_data

wanghaisheng commented 7 years ago

https://github.com/alibaba/mdrill https://github.com/prestodb/presto