feature: allow awaiting Promises on properties

NoNameProvided commented 3 years ago

Description

There had been multiple requests about allowing awaiting promises during transformation. This issue is the tracking issue for the feature.

Proposed solution

Our API is currently sync and doesn't support async operations. The main requirements for the feature should be:

adding async support should not decrease the performance of sync only validation
users using sync only validation should not deal with async operations (so no unified Promise only response)
some form of timeout should be supported for async validation

We have multiple options to implement this:

we can introduce an option to the validation methods which signals an async validation, in that case a Promise should be returned
we can duplicate the API surface and create a fnAsync method for all validation method

Questions

How to handle promises in sync validation? Generally, I think Promises in sync mode should be converted to undefined. However, we may want to add a transformation option that forbids promises in sync validation. This can be useful for users who use both methods.

to be continued... this issue will be expanded with more implementation details before work starts

Please do not comment non-related stuff like +1 or waiting for this. If you want to express interest you can like the issue.

NoNameProvided commented 3 years ago

A related fix has been merged in #463, which prevents a JS error from happening when a Promise is found on a property during transformation.

NoNameProvided commented 3 years ago

A related fix has been merged in #262.

cloudratha commented 3 years ago

God bless your sweet soul

XavierJece commented 3 years ago

  class User {
    @Transform(async ({ value }) => generateAvatarUrl(value))
    avatar?: string;
  }

I have this User class that has an avatar attribute that calls an asynchronous function to generate the temporary storage url. The return after using classToPlan is:

{
   avatar: {}
}

From what I analyzed the avatar attribute is still being a promise, I'm not able to solve someone help me? I'm using version "class-transformer": "^0.4.0".

flisboac commented 3 years ago

I think we have three distinct requirements, that are all blocked by the fact that class-transformer does not support promises:

Make class-transformer await on promises that may come from the input object's properties.
Integrate with DI containers that have promise-based APIs.
Allow for custom transformations to return promises, and make class-transformer await on them before assigning to properties.

For (3), the same solution applied to (1) may be applied here. If we are to reuse the sync part of the library (and depending on the order in which we apply the promise-based part), the promises in the input object must be ignored, and be passed as-is. I'm not sure how the library deals with promises today; if it blocks promises, or raises errors when it sees one, that behaviour may perhaps be deactivated by a new flag, like async: true, that we will need to include in each front-facing function of the API. An advantage of this approach is that we can include new signatures for those functions (returning Promises) based on the presence of this flag.

To indicate which DI container to use, we could repeat a mechanism similar to the one implemented by class-validator.

But still, I think that some improvements could be made to the API surface to support for DI at the decoration level. It doesn't need to go into a high level of specificity. All DI containers support retrieving dependencies by token, so that could be the minimum support we could add, as a new decorator. For example:

export interface InjectedTransformOptions {
  token: string | symbol | () => (string | symbol | Constructor<T> | AbstractClass<T>);
}

export function InjectedTransform(options: InjectedTransformOptions): PropertyDecorator {
  // ...
}

Then, before returning the transformed object to the user, we can check for the async flag, and if activated, we resolve all promises before returning. I don't think it'll impact performance of the sync-side of things that much, as long as we deal with promises explicitly via branching (i.e. if), instead of making current functions async.

One disadvantage of this approach, though, is the fact that there'll be no way to order how or when these promises will be resolved.

For those that, like me, cannot wait for promise support to land in class-transformer, I made some proof-of-concepts for NestJS DI support here. I haven't tested them yet, though; I'm sharing just to have some early feedback:

Sync version: https://gist.github.com/flisboac/b778aba38e69e6a3dd9d6a70cda0edf9
Async version: https://gist.github.com/flisboac/6af02b5254088558362757593dc54f9c

Advantage of the sync version is that it does not introduce much, and uses the very same sync API class-transformer presents to us today. Disadvantage is that you'll need to await on properties yourself, in an explicit step, after calling plainToClass, etc.

Advantage of the async version is that it does all the promise resolution for you, with or without a dependency injection (there're two different types of async transformers: async function and injected object). Dependency injection also supports scoped providers. Disadvantage is the introduction of new metadata, and new transform functions (starting with async).

denbite commented 2 years ago

const plainWithoutPromises = Object.assign(
    {}, 
    yourEntity, 
    {
        promiseProp: await yourEntity.propWithPromise,
        promiseProp2: await yourEntity.propWithPromise2,
        ...
    }
);

plainToClass(YourClass, plainWithoutPromises);

This issue still exists on 19 September, so I've solved it in a such way, maybe it will help someone. The main idea is to give objects without promises to the class transformer.

SergioSuarezDev commented 2 years ago

I was having issues with GraphQLUpload and InputType

https://github.com/jaydenseric/graphql-upload/issues/276

mostafa8026 commented 2 years ago

In which version, promises work? I have issue using promises in Transform.

iteratelance commented 2 years ago

Seems there will not be any progress on this. What libraries are others using for async serialization? I've looked at tools like https://www.npmjs.com/package/json-object-mapper and https://www.npmjs.com/package/json-api-serializer but neither seem to support async methods.

fhamm commented 3 months ago

It is completely nuts that a simple feature like this is missing in a library that is used by so many people.

Hagith commented 3 months ago

IMHO, this is not needed in properly crafted software. You should resolve all your data first, then transform.

hungluu commented 3 months ago

IMHO, this is not needed in properly crafted software. You should resolve all your data first, then transform.

It still depends, right? There are cases you've just joined a long-running project and you can't decide or change how it was already designed. There are also personal tastes added up into a mix of decisions we made while designing a software, just saying :)

If it's a requested features by a reasonable number of users / interests since years, why should we avoid extending the library to cover more use cases?

Hagith commented 3 months ago

IMHO, this is not needed in properly crafted software. You should resolve all your data first, then transform.

It still depends, right? There are cases you've just joined a long-running project and you can't decide or change how it was already designed. There are also personal tastes added up into a mix of decisions we made while designing a software, just saying :)

If it's a requested features by a reasonable number of users / interests since years, why should we avoid extending the library to cover more use cases?

Sure, got the point of such request. But it is about changing the whole public interface of this library to async. And, with all due respect, badly crafted software is not a library concern. I personally prefer to dive into refactoring patterns and think/try to fix the app architecture. Instead of asking others to fix my problems, just saying :)

About "reasonable number of users" is always a problem of perspective :) I believe that there is reasonable number of users against this change, which probably is even bigger in numbers, but those are just not interested in taking part in this conversation ;)

hungluu commented 3 months ago

changing the whole public interface of this library

hmmm, my bad, I wasn't fully aware of this changing the whole library interface. I thought it was just @Transform(async () => something) then internally the library can check if it was promise to wait for. Perhaps I didn't think right through. Can you clarify it a bit please?

My use case is transforming then validating a couple of my DTOs with nestjs before they even reach into my controller code, some of the transformation need async here and there. So this is the feature I'm waiting for. IMO we already had async validations with class-validator so I guess async transformers will collaborate better.

daniel100097 commented 2 months ago

If you have an PlainLiteralObject back from instanceToPlain, you can just await the returned promises.

this is the code in TransformOperationExecutor

(isPromise(value) && !isMap) {
            return new Promise((resolve, reject) => {
                value.then(
                    (data: any) =>
                        resolve(this.transform(undefined, data, targetType, undefined, undefined, level + 1)),
                    reject,
                );
            });
        }

So if you await all values downstream it just works :)


                const awaitAllValues = async (obj: PlainLiteralObject) => {
                    if (!obj) {
                        return obj;
                    }
                    obj = await obj;

                    if (Array.isArray(obj)) {
                        for (const vl of obj) {
                            await awaitAllValues(vl);
                        }
                        return obj;
                    } else if (typeof obj === 'object') {
                        for (const [key, value] of Object.entries(obj)) {
                            obj[key] = await awaitAllValues(await value);
                        }
                    }
                    return obj;
                };
                return await awaitAllValues(instanceToPlain(ObjectInstance, options));

0x0bit commented 1 month ago

@NoNameProvided Whether this function has been implemented?

daniel100097 commented 1 month ago

This is the line in TransformOperationExecutor.ts:

https://github.com/typestack/class-transformer/blob/a073b5ea218dd4da9325fe980f15c1538980500e/src/TransformOperationExecutor.ts#L123C34-L123C74

typestack / class-transformer