on-prem support

[GottZ]

hello there. today I've started digging a little into the source of this repository and noticed a ton of overlap with on-prem installations of D3, just requiring some header injections and slight changes inside some Adapters and Middleware, to just make it work behind the d.ecs http gateway. due to the fact the required changes are fairly minimal core changes, I wonder if anyone thought about giving support for fully licensed on-prem current environments.

if there ain't any plans for that, and nothing is available for doing so, I'll just port this SDK myself, as the license doesn't forbid it.

[ckuetbach]

hello @GottZ

you are right, there is a huge overlap between the on-prem installation and the cloud version, as most applications are the same or have the same APIs.

If you take a look into the C# pendent you'll see, that there is a fallback for a pre-configured system-base URI and a default tenant-id:

https://github.com/d-velop/dvelop-sdk-cs/blob/main/dvelop-sdk-tenant/TenantMiddleware/TenantMiddleware.cs#L105

This fallback is only used if both header are absent, because of security reasons.

The same behavior could be adopted here:

https://github.com/d-velop/dvelop-sdk-node/blob/main/packages/express-utils/src/middleware/dvelop-context-middleware/dvelop-context-middleware.ts#L26

Or you create a middleware which creates a dvelopContext like so:

export function contextMiddleware(
  req: Request,
  _: Response,
  next: NextFunction
): void {
  const onPremHost = "https://mydomain.local";
  const onPremTenantId = "0";

  req.dvelopContext = {
    systemBaseUri: onPremHost,
    tenantId: onPremTenantId,
    requestId: undefined,
    requestSignature: undefined,
  };
  next();
}

So I think the SDK can already easily be used on Prem. The main issue is, that there may be incompatibilities because of older application versions, which may be found at on-premises systems.

@LenKlose What do you think? Would injecting headers or a default base-uri or tenant-id like in the C# version a better approach?

[LenKlose]

Hello @GottZ,

are we only talking about the express-middleware (the contextMiddleware)? I think it would be reasonable to add a way to add custom values or supply a secondary context middleware function similar to our C# approach.

[GottZ]

Hello there and thanks for the quick reply!

In regards to what I actually want to use this for: I basically want to create some administrative tools for kdvz and a couple other public sector on-prem tenants, that take the AuthSessionId to the identityprovider, to accquire ACL mapping and fire some queries at the dms to do some batch queueing of document changes. (for example, triggering LIN002 tasks or just attribute changes on 100k files without using jpl or groove) It will also fire queries directly at the MSSQL instance for d3. so in essence I'll be using identityprovider, dms, maybe tasks, logging and what ever comes to mind later. I do also use the express utils.

I just want to edit stuff properly outside the db by using the dms api's, to prevent inconsistency in versioning, while also using my preferred IDE with all my developer experience tools.

we also have the big process studio license and could do stuff in there but it just doesn't feel as developer friendly as just writing an addon with boilerplate autocompletion like copilot and similar.

(we also use self-signed certificates with our own CA, proxies, etc. there is a couple things enterprise on-prem setups rely on, this sdk and even most nodejs network modules don't offer natively)

in regards to up2date api's, we are constantly monitoring the availability of updates through the software manager and the service portal and update at least once a month.

to get the contextMiddleware working, I simply do this right now:

app.use((req: Request, res: Response, next: NextFunction) => {
    req.headers["x-dv-baseuri"] = process.env.APP_HOST;
    req.headers["x-dv-tenant-id"] = "0";
    next();
});
app.use(contextMiddleware);

resulting in

so far I'd say it would help a ton, if I could provide a custom DvelopHttpRequestConfig to the sdk for use in the _defaultHttpRequestFunctionFactory. I'm currently tinkering with it, to make it work with our environment, to get a hold of the user object.

[ckuetbach]

Hi,

that looks promising. The important part is, that you don't use a user-provided x-dv-baseuri without verifying x-dv-sig1.

As you always hardcode your base-uri this approach would be fine (from a security perspective) as long as you don't include the middleware to verify the signature.

The only downside I can see right now is, that we may decide to make the check for x-dv-sig1 mandatory if a x-dv-baseuri within the SDK-provided contextMiddleware.

In that case you could use a self written contextMiddleware which only sets the dvelopContext or create the signature yourself .

so far I'd say it would help a ton, if I could provide a custom DvelopHttpRequestConfig to the sdk for use in the _defaultHttpRequestFunctionFactory.

I'm not completely sure about this topic, the user-object is written into the requestcontext through another middleware. But I have to admit, that I'm not deep into the node-sdk to know for sure how they work together.

I would expect that everythig works fine with your code as long as you also include the authenticationMiddleware https://github.com/d-velop/dvelop-sdk-node/blob/main/packages/express-utils/src/middleware/dvelop-authentication-middleware/dvelop-authentication-middleware.ts

[GottZ]

As you always hardcode your base-uri this approach would be fine (from a security perspective) as long as you don't include the middleware to verify the signature.

ip pinning ftw. I don't let any other client but the http gateway make calls to the module. also keep in mind all our stuff runs in a DMZ. nothing is available to untrusted peers.

The only downside I can see right now is, that we may decide to make the check for x-dv-sig1 mandatory if a x-dv-baseuri within the SDK-provided contextMiddleware.

would be reasonable to have a instance initialization construct then.

In that case you could use a self written contextMiddleware which only sets the dvelopContext or create the signature yourself.

true.

so far I'd say it would help a ton, if I could provide a custom DvelopHttpRequestConfig to the sdk for use in the _defaultHttpRequestFunctionFactory.

I'm not completely sure about this topic, the user-object is written into the requestcontext through another middleware. But I have to admit, that I'm not deep into the node-sdk to know for sure how they work together.

I would expect that everythig works fine with your code as long as you also include the authenticationMiddleware

apparently it doesn't take the CA from the system certificate store, so I have to override axum's http configuration to provide the certificate manually. right now I'm not running it on the same machine as d3 itself. in that case, it would likely work. we have a messy DMZ structure thanks to kdvz requirements and demands, to get our server connected to other api's and DMZ.

that's precisely what I'm debugging and tinkering with right now. who knows.. maybe it's also client OS configuration related. I have to find the cause myself.

[GottZ]

ok so apparently that certification issue I noticed is actually common. https://github.com/nodejs/node/issues/4175

nodejs on windows doesn't use the windows certificate store at all, thus tools like https://github.com/ukoloff/win-ca or similar have to be used, in order to use custom certificates in nodejs for windows.

definitely something I'd recommend adding to the sdk readme down the line.

well.. it works:

import winCa from "win-ca";
winCa.inject("+");

just to give some context to what I had to do, in order to debug this somewhat. glad to have copilot.

// ...
import { _getAuthSessionIdFromRequestDefaultFunction, _authenticationMiddlewareFactory, _redirectToLoginPageFactory } from "@dvelop-sdk/express-utils/lib/internal";
import { _validateAuthSessionIdFactory, _validateAuthSessionIdDefaultTransformFunction, _defaultHttpRequestFunctionFactory, _defaultHttpRequestFunction } from "@dvelop-sdk/identityprovider/lib/internal";
import { DvelopHttpClient, DvelopHttpRequestConfig, DvelopHttpResponse, axiosHttpClientFactory, axiosInstanceFactory } from "@dvelop-sdk/core/lib/http/http-client"
import { deepMergeObjects, generateRequestId, buildTraceparentHeader } from "@dvelop-sdk/core";
import axios from "axios";

const onPremDefaultDvelopHttpClientFactory = (): DvelopHttpClient => {
    const ax = axiosInstanceFactory(axios);

    ax.interceptors.request.use(
        (request) => {
            console.log(request);
            return request;
        }, (error) => {
            console.log(error);
            return Promise.reject(error);
        }
    );

    ax.interceptors.response.use(
        (response) => {
            console.log(response);
            return response;
        }, (error) => {
            console.log(error);
            return Promise.reject(error);
        }
    );

    return axiosHttpClientFactory(ax, generateRequestId, buildTraceparentHeader, deepMergeObjects);
}

const onPremDefaultHttpRequestFunction = async (context: DvelopContext, config: DvelopHttpRequestConfig): Promise<DvelopHttpResponse> => {
    return _defaultHttpRequestFunctionFactory(onPremDefaultDvelopHttpClientFactory())(context, config);
}

const onPremValidateAuthSessionId = async (context: DvelopContext): Promise<DvelopUser> => {
    return await _validateAuthSessionIdFactory(onPremDefaultHttpRequestFunction, _validateAuthSessionIdDefaultTransformFunction)(context);
}
const onPremAuthenticatorMiddleware = (req: Request, res: Response, next: NextFunction) => {
    return _authenticationMiddlewareFactory(_getAuthSessionIdFromRequestDefaultFunction, onPremValidateAuthSessionId)(req, res, next);
}

app.get(`/${appName}/me`, onPremAuthenticatorMiddleware, (req: Request, res: Response) => {
    console.log(req.dvelopContext);
    res.status(200).send(`<h1>Hello ${req.dvelopContext.user?.displayName}</h1>`);
});

while in the end, running this actually helped me debug the certificate chain in console:

import https from "node:https";
{
    const { options } = https.globalAgent.options;
    options.enableTrace = true;
    options.requestCert = true;
    // options.rejectUnauthorized = false;
}

https.request(process.env.DV_BASE_URI);

[LenKlose]

Hey @GottZ

could you take a look at #202 and see if this would solve your problem?

This would enable you to initilaze your middleware like this:

import { contextMiddlewareFactoryWithFixedSystemBaseUri } from "@dvelop-sdk/express-utils";

app.use(contextMiddlewareFactoryWithFixedSystemBaseUri("https://my.local.baseuri")); //could optionally supply a tenantId (default: 0)

app.use((req: Request, _: Response, next: NextFunction) => {
  console.log(req.dvelopContext);
  next();
});

[GottZ]

that's looking great and indeed reduces code-smell! nice!

I do recommend adding win-ca / mac-ca or mentioning https.globalAgent.ca to the readme for windows and macos environments behind web application firewalls or when using a self-signed Certificate Authority tho. after all on-prem installations of d3 require windows for various legacy and interop reasons so it is reasonable to write plugins like that in windows environments too.