googleapis / google-cloud-datastore

Low-level, Protobuf-based Java and Python client libraries for Cloud Datastore. Check out google-cloud-java and google-cloud-python first!
https://cloud.google.com/datastore
Apache License 2.0
215 stars 133 forks source link

Emulator persistence slow #221

Closed philiplourandos closed 5 years ago

philiplourandos commented 5 years ago

Hi there,

I am seeing extremely long times when persisting data, and there is not a lot I am persisting.

running time (millis) = 55230
-----------------------------------------
ms     %     Task name
-----------------------------------------
04982  009%  load constants
05064  009%  load community
22781  041%  load vouchers
22403  041%  load users

08:23:05.716 [main] INFO  c.c.b.validator.DataSetup - StopWatch '': running time (millis) = 231196
-----------------------------------------
ms     %     Task name
-----------------------------------------
01789  001%  load constants
04150  002%  load community
16533  007%  load vouchers
208724  090%  load users

I also had a look on console.cloud.google.com and I can see that data is being persisted to GCP.

I am using:

Helper setup code:

    @Bean(destroyMethod = "stop")
    public LocalDatastoreHelper dsHelper() throws Exception {
        final LocalDatastoreHelper helper = LocalDatastoreHelper.create();
        helper.start();

        return helper;
    }

and multiple namespaces I setup:

    @Bean
    public DelegatingDatastore testDatastore(final LocalDatastoreHelper helper) {
        final DelegatingGCPDatastore regDs = new DelegatingGCPDatastore();

        nsConfig.getNamespaces().stream().forEach(n -> {
            final Datastore ds = helper.getOptions(n).getService();

            regDs.register(n, ds);
        });

        return regDs;
    }

Is there any thing I am missing?

Many thanks

chadbr commented 5 years ago

@philiplourandos have a look at this one: https://github.com/googleapis/nodejs-datastore/issues/263

Seems it's a grpc issue that needs to make its way downstream...

elharo commented 5 years ago

That's fixed.