search

vmware-archive / quickstep

Quickstep Project
Apache License 2.0
27 stars 13 forks source link

Avoided update CatalogDatabaseCache on the fly. #141

Closed zuyu closed 8 years ago

zuyu commented 8 years ago

In distributed version, it is not safe to update cached CatalogRelationSchema on the fly, while some storage blocks still have reference to the existing schema.

So, disable updates in the cache for now.

pateljm commented 8 years ago

@zuyu: So what happens when the catalog changes?

hbdeshmukh commented 8 years ago

I have one refactoring suggestion for better performance and echoing @pateljm's question on catalog changes.

zuyu commented 8 years ago

@pateljm @hbdeshmukh As noted in #110, we does not support schema changes.

When we support schema changes, we may want to have a version number in CatalogRelationSchema to indicate whether we need to update CatalogDatabaseCache in another separate PR.

If the changed schema has been cached, a naive way is to flush all the blocks belonged to the schema to the disk, before we reset the schema. There are certain scheduling optimization could be done as well, i.e., postpone the schema changes unless no queries are using it.

zuyu commented 8 years ago

@hbdeshmukh I've refactored using fine-grained locks to update CatalogRelationSchemas. Please take a look. Thanks!

hbdeshmukh commented 8 years ago

Looks good, merging.