Open yihaoWang opened 1 month ago
fbukevin
commented
1 month ago It looks some wired logic in your code. You declare a class name TestModel.
class TestModel(ndb.Model):
owner = ndb.UserProperty()Then you import a custom module with the same name.
from junyi.activity.test_model import TestModel
...
test_model = TestModel(owner=user)Won't it be conflicts when you invoke it?
yihaoWang
commented
1 month ago It looks some wired logic in your code. You declare a class name TestModel.
class TestModel(ndb.Model): owner = ndb.UserProperty()Then you import a custom module with the same name.
from junyi.activity.test_model import TestModel ... test_model = TestModel(owner=user)Won't it be conflicts when you invoke it?
Apologies for the confusion. I've simplified the sample code to make it more straightforward and reproducible. Please check if the following code works for you
import os
import sys
from google.appengine.api import users
from google.cloud import ndb
import dev_appserver
class TestModel(ndb.Model):
owner = ndb.UserProperty()
def init_ndb():
os.environ["AUTH_DOMAIN"] = "example.com"
ndb_client = ndb.Client(project="test")
return ndb_client
def test_get_user_data():
ndb_client = init_ndb()
with ndb_client.context():
user = users.User(email="test@test.com")
test_model = TestModel(owner=user)
test_model.put()
query = TestModel.query().filter(TestModel.owner == user)
result = query.get()
print("result", result)
if __name__ == "__main__":
test_get_user_data()@yihaoWang
With removing import dev_appserver, which is only supported in Python2 Client Library, I can reproduce the same error as you.
Actually, this error "An entity value is not allowed" mostly occurs when you attemp to store types like google.appengine.api.users.User into Google Cloud NDB module. However, the value of this data type is not supported by Cloud NDB. Cloud NDB library implemented updating for Python 3. It no longer support some data types or modules of App Ebgine.
The line ndb.UserProperty() is the old API of App Engine. It's not compatible to Python 3 with Cloud NDB library
A solution is that using custom defined property to store data such as email. You can create a user instances with google.appengine.api.users and store email by using user.email(), instead of storing entire users.User object (i.e. owner = ndb.UserProperty()).
Here is an example based on amended your code:
import os
import sys
from google.appengine.api import users
from google.cloud import ndb
class TestModel(ndb.Model):
owner_email = ndb.StringProperty()
def init_ndb():
os.environ["AUTH_DOMAIN"] = "example.com"
ndb_client = ndb.Client(project="example-project")
return ndb_client
def test_get_user_data():
ndb_client = init_ndb()
with ndb_client.context():
user = users.User(email="test@test.com")
test_model = TestModel(owner_email=user.email())
test_model.put()
query = TestModel.query().filter(TestModel.owner_email == user.email())
result = query.get()
print("result", result)
if __name__ == "__main__":
test_get_user_data()And the result of query is:
result TestModel(key=Key('TestModel', 5644004762845184), owner_email='test@test.com')
youchenlee
commented
2 weeks ago Actually, this error "An entity value is not allowed" mostly occurs when you attemp to store types like
google.appengine.api.users.Userinto Google Cloud NDB module.
@fbukevin
test_model.put() succeeds, but the error appears at:
result = query.get()Sorry about the confused wording. The meaning of the statement is that if you attempt to store type google.appengine.api.users.User into Google Cloud NDB module, when you try to get it with user property, it could lead to the error.
youchenlee
commented
2 weeks ago Thank you, @fbukevin .
We are migrating from google.appengine.ext.db to Cloud NDB. Many existing models and queries rely on UserProperty. Is there any workaround to make these queries compatible without needing to migrate billions of rows of data? 😢
youchenlee
commented
2 weeks ago According to the comment, this issue can be resolved by adding meaning = 20 to the final request
project_id: "test"
partition_id {
project_id: "test"
}
read_options {
}
query {
kind {
name: "TestModel"
}
filter {
property_filter {
property {
name: "owner"
}
op: EQUAL
value {
entity_value {
properties {
key: "email"
value {
string_value: "test@test.com"
exclude_from_indexes: true
}
}
properties {
key: "auth_domain"
value {
string_value: "example.com"
exclude_from_indexes: true
}
}
}
meaning: 20 ### Added this ###
}
}
}
limit {
value: 1
}
}This was done using a temporary hack in the code:
+++ /site-packages/google/cloud/datastore/helpers.py 2024-11-01 02:21:15.749998162 +0800
@@ -498,6 +498,8 @@
elif attr == "entity_value":
entity_pb = entity_to_protobuf(val)
value_pb.entity_value.CopyFrom(entity_pb._pb)
+ if 'auth_domain' in val.keys() and 'email' in val.keys():
+ value_pb.meaning = 20
elif attr == "array_value":
if len(val) == 0:
array_value = entity_pb2.ArrayValue(values=[])._pbLooking forward to a better fix, where the correct meaning is applied when encountering UserProperty or appengine User object.
fbukevin
commented
2 weeks ago Hi @youchenlee ,
You can achieve your requirement like this as a workaround. I suggest you can create a pull request with your fixing. Either Google engineering team have their consideration to design like this, or it is a bug and can be accepted 🙂.
fbukevin
commented
2 weeks ago @googleapis
3.12.3pip install appengine-python-standard google-cloud-ndb
import os
import sys
from google.appengine.api import users
from google.cloud import ndbclass TestModel(ndb.Model): owner = ndb.UserProperty()
def init_ndb(): os.environ["AUTH_DOMAIN"] = "ikala.ai"
ndb_client = ndb.Client(project="cloud-sa-sandbox-1")
return ndb_clientdef test_get_user_data(): ndb_client = init_ndb() with ndb_client.context(): user = users.User(email="test@test.com") test_model = TestModel(owner=user) test_model.put() query = TestModel.query().filter(TestModel.owner == user) result = query.get() print("result", result)
if name == "main": test_get_user_data()
**Demo**
https://github.com/user-attachments/assets/aabdf2a1-1e29-4395-ab01-7cff9da630d2
Thanks for stopping by to let us know something could be better!
PLEASE READ: If you have a support contract with Google, please create an issue in the support console instead of filing on GitHub. This will ensure a timely response.
Please run down the following list and make sure you've tried the usual "quick fixes":
If you are still having issues, please be sure to include as much information as possible:
Environment details
Steps to reproduce
Code example
Stack trace
Thanks!