Unable to use datalab create with remote shared-vpc

[Naewis]

Following issue #1535, I'm currently trying to connect a datalab instance using an another project shared VPC.

I'm using cli command like this : datalab create [...] --verbosity debug --subnet projects/<project_id_B>/regions/<region>/subnetworks/<subnet_id> --network-name <project_id_B> --project <project_id_A> instance_name

With debug option, it seems everything went fine : DEBUG: Running [gcloud.compute.instances.create] with arguments: [...] -subnet projects/<project_id_B>/regions/<region>/subnetworks/<subnet_id> --network-name <project_id_B> --project <project_id_A>

But then, shortly after :

ERROR: (gcloud.compute.instances.create) Could not fetch resource:
 - Invalid value for field 'resource.networkInterfaces[0]': '{  "network": "https://www.googleapis.com/compute/v1/projects/<project_id_A>/global/net...'. Subnetwork does not belong to the network. 

Why gcloud create instance subcommand is launched with subnet with project_id_B, and then error is reported with project_id_A ?

If I start manually gcloud compute instance create command, all went fine.

Am I missing something ?

Datalab version : 20190116 Gcloud SDK version : 242.0.0-0 core version : 2019.04.12

[Naewis]

Hello @yebrahim,

Sorry to disturb, but I saw that there's a lot of issues not affected/annoted/commented theses late days. No ill intent, but is there still someone who have some time to this project ? Or is it because it's the wrong way to address issues/questions ? If so, where could I ask theses kind of questions ?

[skydalriol]

Hello,

having the same issue here; using shared VPC from host project seems to work but the instance is created on that same host project instead of the service project where I would like it to be.

Command used:

datalab create --network "projects/<host-project-name>/global/networks/<host-project-vpc-name>" --subnet-name <host-project-subnet-name> --for-user <email.address> --project <service-project> NAME --verbosity debug

the instance is created on <host-project> instead of <service-project>

I tried also specifying the full path of service project but it does not work. Any suggestion?

Thanks

[huron25]

I'm running into the same thing - I don't want Datalab to assume it needs to create a network. I would like to use a subnet shared with the datalab project.

Actually we're intending to do this for a HIPAA aligned project and use Datalab for analysis of data in BigQuery - using shared VPC is something that Google recommends:

https://cloud.google.com/solutions/architecture-hipaa-aligned-project

We'd appreciate any help someone would be able to offer.

[jeffmax]

I believe this is fixed in the latest version of datalab, you should be able to specify --network-name and subnet-name flags. I have seen some odd behavior where this doesn't work in projects that were created a while back, but seems to work in a newly created project, but that is just an observation that I have not been able to confirm.

[huron25]

Thanks @jeffmax - I was just able to figure it out - It appears I had to specify the network with /projects/projectId/global/networks/shared-vpc-id to get it to work.

Now it's erroring on me because it appears it's trying to create Cloud Source Repositories in the Shared VPC Host Project (vs the service project I am launching the datalab instance/container in)

[jeffmax]

Try --no-create-repository

[huron25]

@jeffmax - that does work. Now here's something that seems unusual - it creates the datalab instance, but in the Host network project, even when I specify the --project flag with the service project where I want to GCE instance to be created.

[jeffmax]

I am not sure why that would be, but I did not use the --project flag, I used this command:

gcloud config set project $GCP_PROJECT_NAME

prior to the datalab create command.

[huron25]

Thanks @jeffmax - let me give it a try to see if I can get it to create in the project I want. You were able to get it created in the Service Project on a Shared VPC Subnet?

[huron25]

Thanks again @jeffmax - no luck. It seems that the datalab create command assumes that you are creating the datalab instance in the project where the network resides, no matter if you specify the project parameter.

[slagelwa]

Seeing this same issue myself. Despite using --network and --subnetwork with --project or setting the config project it still appears to try to launch the vm in the VPC host project

[65156]

is there any update to this?

[65156]

is this able to be assigned to anyone for review?