NRE if using ReactiveModule in Web API Backend server

krombipils commented 2 years ago

Hi, I noticed a NullReferenceException when using any ReactiveModule in an integrated Web API Backend/blazor project. Here is a little screencast demonstrating the problem. In the screencast I: 1) Open the default Blazor Login page 2) Open a second tab for the swagger ui 3) Authenticate 4) Execute a get request for /api/odata/ApplicationUser

A NullReferenceException is thrown in the AttributesExtensions.Attributes. I don't really understand what's going wrong. But somehow I think it might be related to the static ObjectSpaceCreatedSubject:

DevExpress' DataService does not use a XafApplication to query data, but an IObjectSpaceFactory. Therefore when the DataService creates an ObjectSpace, the ObjectSpaceCreatedSubject emits, but there is no matching (correctly initialized) XafApplication instance. But actually I have no idea what I'm talking about. I still haven't got used to this reactive stuff.

Hopefully you can repro the problem in my little sample project. This is basically a solution created by the DX wizard where I manually added the ReactiveModule. DXApplication2.zip

Thank you!

apobekiaris commented 2 years ago

thnks for the detaisl and sample will review it

apobekiaris commented 2 years ago

I still haven't got used to this reactive stuff.

the Reactive is a buzz word all we it is use methods where we have an input the arguments and an output the method result

and we chain the methods together with rx operators such as merge, concat, zip etc.

This kind of design is superior as we ddo not have state thus we do not have to support it.

apobekiaris commented 2 years ago

there are no classes, interfaces etc, simple as it can be instead we simulate new classes with tuples again here we can see the superiority of this simple funcational approach.

All we do as devs is write classes and properties and categorized them in namespace spending enourmous amounts to peak the write names so we can understand our code and navigate to ti.

RX use only methods no new classes no interfaces

hope I clear the sky a bit

krombipils commented 2 years ago

Yeah, but I find it hard to debug. And it is also hard (at least for me) to change 3rd party code (while hunting for bugs). Let's say I think there is a bug in your Attributes extension method. In the 'old' eXpand modules, I would override a method (if virtual) or patch the method using harmony. Is this also possible with the new reactive modules?

apobekiaris commented 2 years ago

you do not override u just subsrbie before or after the execute behaviour.

yes harmony works 100% as u probably know Harmony fails with virtual overriden methods, RX works in a static context.

Debug is done with the RX Logger where u filter the service, module etc and examine the flow

In any case should be also possible to open the source for the version u use add the module in your solution, there are [targets] (https://github.com/eXpandFramework/Reactive.XAF/blob/master/src/Directory.Build.targets) that when build they will copy the assembly directly in your nuget folder replacing your existing assembly. This way is like you own the project and can modify and build and test.

Symbols are also distruted.

apobekiaris commented 2 years ago

In any case should be also possible to open the source for the version u use add the module in your solution, there are [targets] (https://github.com/eXpandFramework/Reactive.XAF/blob/master/src/Directory.Build.targets) that when build they will copy the assembly directly in your nuget folder replacing your existing assembly. This way is like you own the project and can modify and build and test.

this is how I will debug your sample, open it add RX module to the solution modify and build that module in place very handy experience

apobekiaris commented 2 years ago

everything u know from xaf can be used as well to modify the behaviour that breaks e.g. a controller where you remove the attribute that fails and disable the behaviour in the CustomizeTypesInfo

apobekiaris commented 2 years ago

I can repro the issue thnks to your detailed post many thnks.

DevExpress' DataService does not use a XafApplication to query data,

I am not sure how correct is this as there is a WarmUpApplication init but still not have a conclusion to anything

apobekiaris commented 2 years ago

fyi: this is my output when I added the Reactive.Module source code to your sample, the nuget is replaced, I can now modify and rebuild

1>------ Build started: Project: Xpand.XAF.Modules.Reactive, Configuration: Debug Any CPU ------ 1>Xpand.XAF.Modules.Reactive -> C:\Work\Reactive.XAF\bin\Xpand.XAF.Modules.Reactive.dll 1>VERBOSE: Performing the operation "Copy File" on target "Item: C:\Work\Reactive.XAF\bin\Xpand.XAF.Modules.Reactive.dll 1>Destination: 1>C:\Users\Tolis.nuget\packages\xpand.xaf.modules.reactive\4.221.1\lib\netstandard2.0\Xpand.XAF.Modules.Reactive.dll". 1>VERBOSE: Performing the operation "Copy File" on target "Item: C:\Work\Reactive.XAF\bin\Xpand.XAF.Modules.Reactive.pdb 1>Destination: 1>C:\Users\Tolis.nuget\packages\xpand.xaf.modules.reactive\4.221.1\lib\netstandard2.0\Xpand.XAF.Modules.Reactive.pdb". 1>VERBOSE: Performing the operation "Copy File" on target "Item: C:\Work\Reactive.XAF\bin\Xpand.XAF.Modules.Reactive.dll 1>Destination: 1>C:\Users\Tolis.nuget\packages\xpand.xaf.modules.reactive\4.221.1.1\lib\netstandard2.0\Xpand.XAF.Modules.Reactive.dll". 1>VERBOSE: Performing the operation "Copy File" on target "Item: C:\Work\Reactive.XAF\bin\Xpand.XAF.Modules.Reactive.pdb 1>Destination: 1>C:\Users\Tolis.nuget\packages\xpand.xaf.modules.reactive\4.221.1.1\lib\netstandard2.0\Xpand.XAF.Modules.Reactive.pdb". ========== Build: 1 succeeded, 0 failed, 0 up-to-date, 0 skipped ==========

apobekiaris commented 2 years ago

But somehow I think it might be related to the static ObjectSpaceCreatedSubject:

veru correct observation! thanks a lot for sparing my time.

But actually I have no idea what I'm talking about. I still haven't got used to this reactive stuff.

You have to admit that you know more than you think and is just some buzz words that confused you. Hopefully all my comments improve your confidendence and knowledge a bit

apobekiaris commented 2 years ago

also since the problem aappeared cause of two different xaf app Blazor and Warmup and if you have similar problem in the future consider publishing them seperately as a workaround. I mean different solution for Blazor and Web APi service

apobekiaris commented 2 years ago

The pre-release 4.221.1.0 in the Reactive.XAF lab branch includes commits that relate to this task:

fix objectspace emmision #941

To minimize version conflicts we recommend that you use the Xpand.XAF.Core.All, Xpand.XAF.Win.All, Xpand.XAF.Web.All packages. Doing so, all packages will be at your disposal and .NET will add a dependecy only to those packages that you actually use and not to all (see the Modules installation-registrations youtube video).

Released packages:

Please update the related Nuget packages and test if issues is addressed. These are nightly nuget packages available only from our NugetServer.

If you do not use these packages directly but through a module of the main eXpandFramework project, please wait for the bot to notify you again when integration is finished or update the related packages manually.

Thanks a lot for your contribution.

krombipils commented 2 years ago

Thank you for the fast fix! Works like a charm!

also since the problem aappeared cause of two different xaf app Blazor and Warmup and if you have similar problem in the future consider publishing them seperately as a workaround. I mean different solution for Blazor and Web APi service

I wasn't aware of the Warmup application causing this issue. And the integrated Web Api service/Blazor solution is something officially supported by DevExpress, therefore I'm using it.

And also thank you for your greate comments/explanation! I think I still don't understand everything, but probably I'm going to implement my next 'controller' in a reactive style

krombipils commented 2 years ago

Hi, could you please help me with a simple problem (I'm trying to understand your reactive extensions, because I think they really do make sense). I'm trying to set the NewObjectViewController.NewObjectActionItemListMode for PermissionPolicyObjectPermissionsObject and PermissionPolicyMembersPermissionsObject views. With the 'traditional' controller approach this would somehow look like this:

public class CustomizeNewPermissionPolicyObjectPermissionsObjectObjectListController : 
    ObjectViewController<ObjectView, PermissionPolicyObjectPermissionsObject>
{
    protected override void OnActivated()
    {
        base.OnActivated();

        var controller = Frame.GetController<NewObjectViewController>();
        if (controller != null)
        {
            controller.NewObjectActionItemListMode = NewObjectActionItemListMode.LastDescendantsOnly;
        }
    }
}

I would add another controller for the PermissionPolicyMemberPermissionsObject or use a generic controller and check the type in the OnActivated.

How would you code this with your reactive extensions? This code is working

public override void Setup(XafApplication application)
{
    base.Setup(application);

    application.WhenViewOnFrame().
        Where(frame =>
        {
            if (frame.View is not ObjectView objectView)
            {
                return false;
            }

            var objectType = objectView.ObjectTypeInfo.Type;
            var isAssignable = objectType.GetInterface(nameof(IPermissionPolicyMemberPermissionsObject)) != null ||
                               objectType.GetInterface(nameof(IPermissionPolicyObjectPermissionsObject)) != null;

            return isAssignable;
        }).
        Do(frame =>
        {
            var controller = frame.GetController<NewObjectViewController>();
            controller.NewObjectActionItemListMode = NewObjectActionItemListMode.LastDescendantsOnly;
        }).Subscribe();
}

But maybe it can be solved more elegant? E.g. In your code you often subscribe to moduleManager.WhenApplication(...). instead of using application.When(...). Why?

Thank you!

apobekiaris commented 2 years ago

I think they really do make sense

I want to add that coding without RX does not make sense at all, why u want to do that? RX is about compact coding is about coding a monad that then you can chain/use/observer it as you wish. RX has use success in many major languages Java, Js, Kotlin etc expect from MS although it is a .NET baby. Reasoning is MS WTF I will break everything again

But maybe it can be solved more elegant?

yes of course can be written in endless possible ways I provide 2 version and at the end I merge and use them both togetther.

var version1=application.WhenViewOnFrame().WhenFrame(ViewType.DetailView,ViewType.ListView)
                .Where(frame => new[]{typeof(IPermissionPolicyMemberPermissionsObject),typeof(IPermissionPolicyObjectPermissionsObject)}
                    .Any(type => type.IsAssignableFrom(frame.View.ObjectTypeInfo.Type))).
                Do(frame => frame.GetController<NewObjectViewController>().NewObjectActionItemListMode = NewObjectActionItemListMode.LastDescendantsOnly)
                .ToUnit(); 

            var version2=application.WhenFrameCreated().WhenFrame(typeof(IPermissionPolicyMemberPermissionsObject), typeof(IPermissionPolicyObjectPermissionsObject))
                .WhenNotDefault(frame => frame.View.AsObjectView()).ToController<NewObjectViewController>()
                .Do(controller => controller.NewObjectActionItemListMode = NewObjectActionItemListMode.LastDescendantsOnly)
                .ToUnit();

            version1.Merge(version2) //merge is not possible is impossible if observables return different types, however we used ToUnit to make them IObservable<Unit>
                .Subscribe(this); // not this which will unsusbirbe once the module is disposed

of using application.When(...). Why?

if u start using this style u will soon follow the same discoveries I will discuss, as it is a matter of survival, the same is true with the imperative code style u now using.

So the code above should be on a Service class e.g. NewObjectService or whatever is the context that relates to your task.

As I mentioned b4 to u all we do as programmers is name stuff, so how can I know how to name the Service? You do not just put all your methods there and you refactor later to more services as the class grows or have multiple contexts. Or since you use Xaf you can have one Servive per package. There are complex packages though like the DocumentStyleManager that reuire multiple service per module in order not to get lost from the so many methods.

So having the NewObjectService we first need to connect it with our XafApplication so we can write

  public static class IWillDeCideTheNameLaterServiceClass {
    internal static IObservable<Unit> ConnectNewService(this ApplicationModulesManager manager) {
        //here I will connect my methods
    }

}

public override void Setup(ApplicationModulesManager moduleManager) {
        base.Setup(moduleManager);
        moduleManager.ConnectNewService().Subscribe(this); // this line connects the service and disconncts on dispose becaue of `this`
    }

not the internal modifier? we do not want module consumers to access the Connect method it should be called only once. However we want to live in a service class so it must not be internal cause it subscribes from the module setup and we might have more than one services. I think you get my points here.

Why ModuleManager you asked? A service may have a lot of methods so how you can tell what it does? You can tell either from modifiers e.g. Public methods intent to be consumed or by convension e.g. the internal ConnectMyService is where the game starts. Yes you can have one that takes the XafApplication argument and you will need to subscribe to the other module Setup overload or you can even subribe at a later point e.g. any point you like where you have access to a XafApplication. But if you choose so, first you start making making your life complex and you also may need to go lower and support the ApplicationModuleManager cause the manager has extensions to handle the designtime e.g. manager.WhenCustomizeTypesInfo(), manager.WhenExtendingModel(), manager.RegisterSimpleViewAction() etc.


  public static class IWillDeCideTheNameLaterServiceClass {
internal static IObservable<Unit> ConnectNewService(this ApplicationModulesManager manager) 
        => manager.WhenApplication(application =>
            //i used a different varation here the MergeToUnit as the two observables are of different type
            application.NewObjectActionModeV1().MergeToUnit(application.NewObjectActionModeV2())); //this delagate has retry operation cause if an rx subscriptions terminate error and we need to resubscribe

    //do not be scare to refactor a method name many time into smaller methods, it is the method name that tells what is it about and you should never read the code just the name 
    static IObservable<Frame> NewObjectActionModeV1(this XafApplication application) 
        => application.WhenViewOnFrame().WhenFrame(ViewType.DetailView,ViewType.ListView)
            .Where(frame => new[]{typeof(IPermissionPolicyMemberPermissionsObject),typeof(IPermissionPolicyObjectPermissionsObject)}
                .Any(type => type.IsAssignableFrom(frame.View.ObjectTypeInfo.Type))).
            Do(frame => frame.GetController<NewObjectViewController>().NewObjectActionItemListMode = NewObjectActionItemListMode.LastDescendantsOnly);

//they both private as it makes no sense to be consumed and will make it very hard to understand what this service does. static IObservable NewObjectActionModeV2(this XafApplication application) => application.WhenFrameCreated().WhenFrame(typeof(IPermissionPolicyMemberPermissionsObject), typeof(IPermissionPolicyObjectPermissionsObject)) .WhenNotDefault(frame => frame.View.AsObjectView()).ToController() .Do(controller => controller.NewObjectActionItemListMode = NewObjectActionItemListMode.LastDescendantsOnly); }



  I hope I made some sense if you have questions ask

krombipils commented 2 years ago

Impressive answer! Thank you! I need some time to understand it ;-)

krombipils commented 2 years ago

Wow, actually you have a lot of useful extensions methods! I like the WhenFrame(...) methods. But I think I have to get used to them!

Regarding the 'ModuleManager.WhenApplication(...)' pattern: If I get you right, it is a convention of you, because you often have to use extensions/events of the ModuleManager (e.g. WhenCustomizeTypesInfo) additonaly to the Application events. Correct? But if a module/service never has to use these events, it's perfectly fine to subscribe to the 'application' events (apart from not sticking to the 'convention')?

apobekiaris commented 2 years ago

yes it is fine but is not favored at least by me for e.g. lets look at the implementation of this method

        public static IObservable<T> WhenApplication<T>(this ApplicationModulesManager manager,Func<XafApplication,IObservable<T>> retriedExecution,bool emitInternalApplications=true) 
            => manager.WhereApplication().Where(application => emitInternalApplications||!application.IsInternal()).ToObservable(ImmediateScheduler.Instance)
                .SelectMany(application => Observable.Defer(() => retriedExecution(application)).Retry(application));

as you can see does a lot of stuff getting the Application from whatever context apparently will not do all that, maybe it is what u want some times but point is that I follow a certain way always as it is easier to address possible problems later and apply to all the codebase.

apobekiaris commented 2 years ago

in addition see this gist this is a pattern and extensions I very often use

https://gist.github.com/apobekiaris/6dc12607d1ba7be554b1f36320c3e027

i am sure you agree that this implementation with controllers hmm how to say good luck and I did it in 10 lines

apobekiaris commented 2 years ago

in the gist you can see another variation of ToUnit which is the To extension which transforms the observable emission to any type and if you IgnoreElements is like you inject an observable insisde another one

krombipils commented 2 years ago

Very interesting. Do you have a working, minimal sample project (with Account, Product, RedeemService, ...)? That would be very helpful. And I think it does make sense to also add your comments/infos somewhere in the Reactive.XAF readme?!

apobekiaris commented 2 years ago

not really this is a huge internal project that I have been working for years see a post I did back in 2018

Dealing with Back-Pressure - UI - a database - the ReactiveX way

apobekiaris commented 2 years ago

so actually all methods you see there with no implementation are web serive calls that have embedded logic for paging, error handling, rertying, notification.

All these operations are different obserbles e.g. the paging looks like

public static IObservable<T[]> Paginate<T>(this IEnumerable<T> source,Func<int,IObservable<T[]>> selector,XafApplication application) where T : CryptoManagerObject 
            => application.Paginate(selector, () => source);

        private static IObservable<T[]> Paginate<T>(this XafApplication application, Func<int,IObservable<T[]>> selector,Func<IEnumerable<T>> existingSelector){
            var existingObjects = existingSelector().ToArray();
            return Observable.Range(1, 100).SelectManySequential(selector).TakeUntil(array => array.Length < 100)
                .BufferUntilCompleted().WhenNotEmpty().Select(arg => arg.SelectMany().ToArray())
                .SelectMany(objects => application.UseProviderObjectSpace(space => {
                    var keys = objects.Select(arg => space.GetKeyValue(arg)).ToArray();
                    var list = existingObjects.Where(arg => !keys.Contains(space.GetKeyValue(arg))).ToArray();
                    if (list.Any()){
                        space.Delete(list.Select(space.GetObject).ToArray());
                        space.CommitChanges();
                    }
                    return space.Commit().To(objects);
                })).Take(1);
        }

another paging variation is public and used from my Office packages pagination

in my intrnal Manager project the logging is similar to what i use in all my packages

        public static IObservable<TSource> TraceErrorManager<TSource>(this IObservable<TSource> source,
            Func<TSource, string> messageFactory = null, string name = null, Action<string> traceAction = null,
            Func<Exception, string> errorMessageFactory = null, [CallerMemberName] string memberName = "",
            [CallerFilePath] string sourceFilePath = "",
            [CallerLineNumber] int sourceLineNumber = 0)
            => source.Trace(name, ManagerModule.TraceSource,messageFactory,errorMessageFactory, traceAction, ObservableTraceStrategy.OnError, memberName,sourceFilePath,sourceLineNumber);

However I integrate it with RXLogger Notification so I can use both Xaf Message and RXLogger to debug it

<ReactiveLogger>
      <Notifications>
        <ReactiveLoggerNotification Id="Errors" Criteria="[RXAction] = ##Enum#Xpand.XAF.Modules.Reactive.Logger.RXAction,OnError# And Not Contains([Value], 'HttpRequestException') And Not Contains([Value], 'A task was canceled')" ObjectType="Manager.Module.BusinessObjects.ErrorEvent" ShowXafMessage="True" XafMessageType="Error" MessageDisplayInterval="10000000" IsNewNode="True" />
      </Notifications>

In my internal Manager I do not use the manager.WhenApplicaction but an overload of it taking care the integration of the just mentioned tracing and notification and CompleteOnError not to break my subscriptions as I already being notified in the UI and RX.Logger about them

public static IObservable<T> WhenManagerApplication<T>(this ApplicationModulesManager manager, Func<XafApplication, IObservable<T>> selector)
            => manager.WhenApplication(application => selector(application).TraceErrorManager().CompleteOnError());

I want to add that coding without RX does not make sense at all, why u want to do that

now hopefully you agree that my comment is justified.

This level of abstraction and reusability is way too powerfull

apobekiaris commented 2 years ago

all web service calls in all my projects at the end will go through the NetworkExtensions which of course can be overriden in a sense as the Reactive.Rest package does in order to have a rest api auto polling/caching avoid the rate limits

let me know where to stop :)

everything is connected as RX makes this kind of desing natural

apobekiaris commented 2 years ago

The pre-release 4.221.5.0 in the Reactive.XAF lab branch includes commits that relate to this task:

Update Harmony and restore desing time patch #941

To minimize version conflicts we recommend that you use the Xpand.XAF.Core.All, Xpand.XAF.Win.All, Xpand.XAF.Web.All packages. Doing so, all packages will be at your disposal and .NET will add a dependecy only to those packages that you actually use and not to all (see the Modules installation-registrations youtube video).

Released packages:

Please update the related Nuget packages and test if issues is addressed. These are nightly nuget packages available only from our NugetServer.

If you do not use these packages directly but through a module of the main eXpandFramework project, please wait for the bot to notify you again when integration is finished or update the related packages manually.

Thanks a lot for your contribution.

expand commented 9 months ago

Closing issue for age. Feel free to reopen it at any time.

.Thank you for your contribution.

eXpandFramework / eXpand

NRE if using ReactiveModule in Web API Backend server #941