search

GoogleCloudPlatform / datastore-ndb-python

Client library for use with Google Cloud Datastore from within the Google App Engine Python runtime.
https://cloud.google.com/appengine/docs/standard/python/ndb/
Apache License 2.0
114 stars 48 forks source link

Allow ndb to run from outside of App Engine #272

Open pcostell opened 7 years ago

kevinjdolan commented 7 years ago

IS THIS HAPPENING FOR REAL?

erugeri commented 7 years ago

Would love to see it for objectify / ndb to be able to migrate to flex. Thanks!

gcbirzan commented 7 years ago

The ndb interface to the datastore is vastly superior to the datastore one, at least for Python, hopefully this gets fixed.

jredl-va commented 7 years ago

With the deprecation of the compat-runtime for appengine-flex this functionality would be a game changer ultimately enabling us to move some of our workloads off of appengine standard into kubernetes without rewriting large chunks of our applications.

Erfa commented 7 years ago

This seems like one of the most important things to fix in order to make it easier to migrate from standard App Engine to flexible environments.

jonathan-s commented 7 years ago

Would love to see this!

aatreya commented 7 years ago

@pcostell Is this still under development? Do you have a rough timeframe on an alpha/beta release?

ekampf commented 7 years ago

Any news on this?
Google is really dropping the ball on GAE users here...

artooro commented 7 years ago

Just want to comment, in the hopes there will be an EAP for this. I'd love be on it.

vanpelt commented 7 years ago

+1000

Bogdanp commented 7 years ago

Excuse the shameless self-promotion but those of you on Python 3.6+ might find this project useful: https://github.com/Bogdanp/anom-py

bsmithyman commented 7 years ago

Would it be possible for someone from Google (e.g., @pcostell) to update this issue?

With the new flexible environment expansion and GA status, we are left trying to decide whether to pursue a Python 3 refactor of our backend or continue using NDB. At present, we can't do both.

This is pushing us to consider switching databases, purely from a business risk point of view. Likely, that would be to Cloud Datastore directly, but it is not the only one in the running. A bit of extra information here would be incredibly helpful. We certainly want to continue to use NDB, but it's starting to look pretty risky. From the outside, it seems like we may need to choose between NDB and Python 3 (which is a pretty tough sell).

Thanks!

artooro commented 7 years ago

@bsmithyman I'm not from Google, but have been told this week that they are working on it. Even so it would be nice to get a technical update from a google engineer here just to regain some confidence on the app engine platform.

bsmithyman commented 7 years ago

Thanks @artooro; appreciated.

mikelambert commented 7 years ago

So from what I understand, the GAE Managed VM python-compat bridge is being turned down end-of-October, in favor of GAE Flexible VMs use of python images.

Is "Make NDB work outside of GAE Standard" a blocker for "turn down the python-compat bridge" for the GAE/FlexVM/NDB teams?

Or will I need to plan for the possibility that my GAE NDB code will not run on python GAE Flexible? And if I need to rewrite...what is the recommended DB solution (gcloud.datastore, DB, and NDB) for common-code that needs to run in both GAE Standard and GAE Flexible? Currently, they all seem to have drawbacks: gcloud.datastore in GAE Standard (super slow versus native bridge) DB/NDB in GAE Flexible (library doesn't exist, and isn't supported)

cpavon commented 7 years ago

@mikelambert Same issue here.... python-compat is gonna be decommissioned in October so we have to face some important decisions.

We also need to keep code running in both environments: Standard and Flexible. So if we don't have a common way to access to datastore we will have a problem.

Someone has face the same problem? alternatives?

kevinjdolan commented 7 years ago

It's very clear that Google is not pushing the standard AppEngine environment anymore -- to the point of deprecation, in my opinion. I decided to just pull the bandaid and move to Container Engine.

MrMaxRoss commented 7 years ago

Hi Kevin,

I'm the eng director for App Engine. I saw your comment about the Standard environment and felt compelled to respond.

I hope Container Engine is working well for you, but there's actually a massive investment in the Standard environment underway, and you're going to start seeing the results of that investment very soon in the form of updated Standard runtimes. I'm not making any announcements here, but if you look at the blog post where we brought the Flex environment to GA you'll see we've been running an Alpha for Java 8 on Standard, and a recent blog post on Python for App Engine also points at big changes in the pipeline.

Thanks, Max

workmanw commented 7 years ago

@MrMaxRoss I really hope you announce something soon, hinting at changes can only take us so far when we're trying to make real business decisions. Right now we use NDB on Flex via legacy (vm:true). However that is being sunset in October. So we're actively trying to figure out what parts of our application we need to rewrite.

bsmithyman commented 7 years ago

I would echo @workmanw, inasmuch as we would love to be using both NDB and Python 3, and the uncertainty makes it hard to bet on this environment. A Python 3 Standard Environment with NDB (as hinted in previous comms) would be very welcome. Thanks for updating as you're able, @MrMaxRoss.

MrMaxRoss commented 7 years ago

We are still planning to sunset vm:true in October, but you can upgrade to env:flex and request an exception to continue using the compat runtime past October. There are some devs testing NDB on env:flex using the compat runtime right now (one bug so far). I will report back when we are confident that it works.

I know this is different from a fully supported NDB on Python on GAE Flex, but I want you to know that we expect there to be an option for NDB beyond October.

brianmhunt commented 7 years ago

Hi @MrMaxRoss — I think a lot of folks (including myself) would very much enjoy an update, insofar as you can, on where this issue stands. Thank you, sincerely!

wsh commented 6 years ago

@brianmhunt hi there! We can't provide specific guidance about dates just yet, but we have engineers working on this right now. (I'm one of them.) I'll be sure to keep you posted on this thread.

lcasartelli commented 6 years ago

Hi @wsh! Do you have any news for us? Thanks!

robhaswell commented 6 years ago

Hey @wsh how are you getting on?

jer-tx commented 6 years ago

Any updates on this? We're looking to move from Standard to Flexible and not having an ORM to talk to datastore is a massive barrier.

cr4zyd3v commented 4 years ago

Any updates on this? I am using python datastore client for my app, but it would be cool to have ndb outside of app engine, since it offers a lot more.. Thanks in advance.

artooro commented 4 years ago

Everyone following this issue should be aware of this project https://github.com/googleapis/python-ndb which is going to replace the existing python 2.7 NDB library. Currently it's only Python 3 compatible but it appears it will support 2.7 as well.