search

data-dot-all / dataall

A modern data marketplace that makes collaboration among diverse users (like business, analysts and engineers) easier, increasing efficiency and agility in data projects on AWS.
https://data-dot-all.github.io/dataall/
Apache License 2.0
235 stars 82 forks source link

Inter region sharing is not allowed. #318

Closed manjulaK closed 1 year ago

manjulaK commented 1 year ago

Describe the bug

It is not possible to share a data set from Account A (region A) to Account B (region B) . This is a critical requirement.

How to Reproduce

1) create dataset in account A region 1 2) request dataset from account b region 2 3) the approval of table fails for cross region.

Expected behavior

No response

Your project

No response

Screenshots

No response

OS

unix

Python version

3.1

AWS data.all version

v1.3

Additional context

No response

dlpzx commented 1 year ago

Hi @manjulaK, yes, this is a known limitation of Lake Formation. It is supposed to throw an error when a request is open from a different region, but I was able to reproduce that in fact is allows you to create the request.

In particular for sharing issues, I highly recommend you to upgrade to V1.4.0. In this last release we reviewed the whole sharing workflow to make it more robust and error-less.

dlpzx commented 1 year ago

In the PR #322 we added an error message that prevents users to open a cross-region share request, this will ensure you do not run into this issue. In the future we can explore options to allow cross-region sharing, for example with Bucket policies #284, but in general working with data in different accounts is not recommended. So if possible we advice you to avoid this architecture, if it is not possible because of compliance or because of AWS services not available in a region we can discuss alternatives.