CloudFoundry Configuration - Service credentials empty

[szaboegon]

Question

I have a problem when using the Steeltoe.Extensions.Configuration.CloudFoundryCore package in ASP.NET.

There is a service bound to my application. The applications environment section describing the services looks like this:

...
VCAP_SERVICES=
{
  "service1": [
    {
      "label": "service1",
      "provider": null,
      "plan": "Free",
      "name": "service1_dev",
      "tags": [
        "tag1",
        "tag2"
      ],
      "instance_guid": "486515e5-8b46-4e27-8f90-661f5341ae20",
      "instance_name": "service1_dev",
      "binding_guid": "a22e8c91-0b42-4902-bed5-c6c892830c47",
      "binding_name": null,
      "credentials": {
        "serviceInstanceId": "<serviceInstanceId>",
        "password": "<password>",
        "hostname": "<hostname>",
        "uri": "<uri>",
        "username": "<username>"
      },
      "syslog_drain_url": null,
      "volume_mounts": []
    }
  ]
}
...

I initialize the configuration provider and configure the options classes in my Program.cs as shown in the documentation:

 builder.AddCloudFoundryConfiguration();
 builder.Services.AddOptions();
 builder.Services.ConfigureCloudFoundryOptions(builder.Configuration);

After that I resolve the CloudFoundryServicesOptions in my service class from the service provider - so far everything works as expected. However upon inspection I found that the options instance contains the following:

{
   "CONFIGURATION_PREFIX":"vcap",
   "Name":null,
   "Label":null,
   "Tags":null,
   "Plan":null,
   "Services":{
      "service1":[
         {
            "Credentials":{
               "hostname":{

               },
               "password":{

               },
               "serviceInstanceId":{

               },
               "uri":{

               },
               "username":{

               }
            },
            "CONFIGURATION_PREFIX":"services",
            "Name":"service1_dev",
            "Label":"service1",
            "Tags":[
               "tag1",
               "tag2"
            ],
            "Plan":"Free",
            "Services":{

            }
         }
      ]
   }
} 

As you can see all the credential fields are empty when they shouldn't be according to the environment variables. Also the structure of the second JSON looks weird to me, especially all the null fields at the root level.

I tried everything but I don't know what causes this when I'm doing everything right according to the documentation.

Environment (please complete the following information):

• .NET6
• Steeltoe Version 3.2.8

[TimHess]

Hi @szaboegon,

The code you've shared generally looks correct, can you share how you're inspecting the options instance? (eg: remotely debugging app in CF vs app running locally that sets VCAP manually or something else). It may be useful to have a minimal but running application that demonstrates the problem if you have time.

In case this is useful, unit tests for CloudFoundryServiceOptions are here

[szaboegon]

Hi @TimHess,

I am using remote debugging to inspect the options, so the app is deployed to CloudFoundry when inspecting.

As for the application: I cannot share the original, but I created a new sample ASP.NET project and the issue still stands. I attached the code below.

SteeltoeTest.zip

[TimHess]

OK, that makes sense - thanks for sharing the sample, I believe I see what you're seeing. I think that both the null fields and the seemingly empty credentials are side affects of the generic and recursive data structure used in CloudFoundryServicesOptions - it might be worth reviewing how this sample deals with it.

Can we back up a step and discuss what you're trying to accomplish? CloudFoundryServicesOptions is interesting, but probably not typically the right tool for the job

[szaboegon]

I checked the sample and I don't really see any differences between that and my usage.

Originally I wanted to use my own options classes with the ConfigureCloudFoundryService method as stated here in the docs. However, the credentials fields were always null, thats why I moved to the CloudFoundryServicesOptions class, which did not help.

I could get the values via manually parsing the Configuration, but I want to avoid that at all costs, because it is not typed and hard to maintain in my opinion. So I definitely want to use the Options framework somehow.

[TimHess]

The difference is subtle, but important. Credential could be a dictionary or a value, so you need additional logic like in this taghelper

I understand that you are trying to get credentials out of configuration, but can you please be more specific?

We built (and in v4 completely re-built) Connectors to handle the complexity of the generic structures and make it more easily consumable for specific services. Steeltoe tries to create situations where your apps benefit from platform features like VCAP_SERVICES without needing to worry about things like the data structures used to provide values needed to build a connectionstring for MySQL (or whatever other service you want to use).

Additionally, ConfigureCloudFoundryServices<TOption>() is currently being considered for removal from v4 because it was never built out beyond the MySQL example and we think Connectors are easier to work with, so your feedback right now would be useful.

Connector samples for v3 and v4

[szaboegon]

Specifically, my use case is that I want to connect to a MySQL server that is hosted outside of CloudFoundry. For this I am using user-provided services as stated here in the CloudFoundry docs.

Now that I have looked at connectors they really seem a lot easier to use, but It doesn't quite fit my use case. I tried, but it seems like it cannot find the service by name. I'm guessing this is because of the user-provided aspect.

It is interesting, that you are considering the removal of the ConfigureCloudFoundryServices<TOption>() as something of the sort would be very useful to me. I don't know if adding support for user-provided services is plausible, I understand that binding services only by name is probably a lot harder.

Is there any way I can make the package work with user-provided services or should I look into other solutions?

[TimHess]

User-provided services in general are difficult to support since they could contain just about anything, therefore the structure is nearly as general as what IConfiguration and IOptions already provide. The PR dealing with it is expected this week if you want to keep an eye out for it.

In this case all you should need to do is add the tag "mysql" and use the same keys Steeltoe is looking for. Here's the credential mapping in v4. For v3, the mapping is here - please note the check for a Uri setting (I don't know offhand why it's there, but it's been that way for years at this point)

[szaboegon]

I added the tag like you said, but the connector still does not seem to find the service instance. This is what the VCAP_SERVICES env variable looks like:

VCAP_SERVICES=
{
  "user-provided": [
    {
      "label": "user-provided",
      "name": "mysqlService",
      "tags": [
        "mysql"
      ],
      "instance_guid": "210b46fc-c8f0-4aff-83e9-159a5f1f728c",
      "instance_name": "mysqlService",
      "binding_guid": "daa372ff-4269-46ed-901a-1abc2e81ba04",
      "binding_name": null,
      "credentials": {
        "Host": "...",
        "Password": "...",
        "Path": "...",
        "Port": 3306,
        "Uri": "...",
        "Username": "..."
      },
      "syslog_drain_url": null,
      "volume_mounts": []
    }
  ]
}

My code in ASP.NET:

builder.AddCloudFoundryConfiguration();
var mySqlServiceName = "mysqlService"
builder.Services.AddDbContext<MyDbContext>(options =>
{
    options.UseMySql(builder.Configuration, mySqlServiceName);
});

The connector throws an exception saying Steeltoe.Connector.ConnectorException: No service with name: mysqlService found.

On another note: is v4 released already? As I cannot find it in the Nuget Package Manager or the documentation either.

[TimHess]

AH! Sorry for the confusion here, it's been a while since I looked at this code, and it really is too complex... The reason the code checks for the existence of Uri is that is actually what is used to set all the other properties. For Steeltoe v3, set your MySQL Uri like this mysql://uriuser:uripassword@urihost:1234/uripath and you should be good to go, even if you don't set anything else in the user-provided service.

Why would we do that? Because (afaik) all of the MySQL brokers ever created for Cloud Foundry set the Uri consistently, but not the other properties...

Steeltoe v4 has not quite shipped yet, but we are extremely close to public previews on nuget.org (the last API review PR is in progress right now, so we might be done with planned breaking changes as soon as this week). Work on the samples is happening in the latest branch I linked above (take a look at the nuget.config files if you want to try the latest packages from our dev feed), work on documentation is currently in this branch for now, but we'll probably be merging it to main in a matter of weeks