search

geberit / Revit.TestRunner

Unit Test Runner for Autodesk Revit
MIT License
81 stars 28 forks source link

Runner cannot load test dlls. #11

Closed sylvesp closed 2 years ago

sylvesp commented 2 years ago

I am trying to use the Revit.TestRunner for some Revit testing and I am running into a problem that I hope you can help me with.

These are the steps I've done to reproduce my issue.

  1. Clone the Revit.TestRunner repository to my local machine (C:\Work\Projects\Revit.TestRunner)
  2. Open C:\Work\Projects\Revit.TestRunner\src\Revit.TestRunner.sln in VS and compile the whole project for Revit 2021
  3. Start Revit 2021 with No-Active-Document (so that the Revit Runner Server is Active) (This is done from the Windows Revit Icon and NOT from VS/ NOT debugging / VS is NOT attached to Revit)
  4. Start C:\Work\Projects\Revit.TestRunner\src\bin\Client\Revit.TestRunner.App.exe
  5. In the TestRunner.App.exe click the "..." and try open C:\Work\Projects\Revit.TestRunner\src\bin\Revit.TestRunner.SampleTestProject2.dll

As a result I get an error message in the TestRunner.App status bar saying Could not load 'C:\Work\Projects\Revit.TestRunner\src\bin\Revit.TestRunner.SampleTestProject2.dll'

Now if I attach the VS debugger to Revit, after Revit was started at step 3 and try to load the SampleTestProject2.dll then everything works fine - and I am able to run the tests.

Is there anything I am missing? Why is that the runner cannot load the test assembly when is in debug mode? I did try to debug the issue but obviously I did fail because everything works when the debugger is attached.

I'd appreciate your help with this problem. Please let me know if you need any extra info/log files/etc.

Peter.

tobiasfloescher-geberit commented 2 years ago

Hello Peter

You have done everything right.

Please check if the Revit Version is written in the TestRunner.App on the Top of the UI, beside of the Title and Version. If not, the Addin does not run correctly. Try to start the TestRunner.App from Revit Ribbon 'Add-ins' - 'Testing - Open Runner'.

Addin logfile path C:\Work\Projects\Revit.TestRunner\src\bin\Addin\Test.Runner.log. Please show it to me.

Tobias

sylvesp commented 2 years ago

Hi Tobias,

Thanks for your prompt reply. Here are my answers to your questions:

Here is the log file content Unfortunately it doesn't say much :(

2021-10-05 08:11:38,875 [1] INFO Revit.TestRunner.Log - Revit.TestRunner started '10/5/2021 8:11:38 AM' 2021-10-05 08:11:38,883 [1] INFO Revit.TestRunner.Log - Microsoft Windows NT 10.0.19042.0, NetFX 4.0.30319.42000 2021-10-05 08:11:38,885 [1] DEBUG Revit.TestRunner.Log - Log Directory 'C:\Work\Projects\Revit.TestRunner\src\Revit.TestRunner\bin\Debug2021' 2021-10-05 08:11:38,885 [1] DEBUG Revit.TestRunner.Log - CurrentAppDomain.ApplicationBase 'C:\Program Files\Autodesk\Revit 2021\' 2021-10-05 08:11:38,893 [1] INFO Revit.TestRunner.Log - Service started '10/5/2021 8:11:38 AM' 2021-10-05 08:11:57,636 [12] INFO Revit.TestRunner.Log - Process Home request 2021-10-05 08:13:43,451 [12] INFO Revit.TestRunner.Log - Process Home request

As I've said though: if I attach the VS debugger to the Revit process the load is successful and I can run tests. BUT if I detach the debugger from the SAME Revit process, and try to load it again then it fails again.

All this seems to me to be a dependency issue. Revit cannot find one of the dlls needed to load the test dll. When the debugger is attached, then the VS debugger is helping with the finding of the needed dll.

Oh, and one more thing (I am not sure if this is relevant though): Seems that the code on your machine is slightly different than the one on github because you've asked me to show you the content of the log at: C:\Work\Projects\Revit.TestRunner\src\bin\Addin\Test.Runner.log but the log file is at: C:\Work\Projects\Revit.TestRunner\src\Revit.TestRunner\bin\Debug2021

I am still looking into getting this working working without the Debugger attached and will let you know if I find anything new. The problem that makes this hard is that if I start debugging then everything works :)

Have a nice day! Peter

sylvesp commented 2 years ago

Have some updates. This seems to be a timing issue with the handling of concurrent routes. Hopefully will have more on it tomorrow. Peter

tobiasfloescher-geberit commented 2 years ago

Hi Peter. thank you for investigating.

Have you set the addin file by yourself? When compiling, an addin file is automatically created (C:\ProgramData\Autodesk\Revit\Addins\2021\Revit.TestRunner.addin). There you can see the path to the dll. It's in the global bin of the solution (..\Revit.TestRunner\src\bin\Addin\Revit.TestRunner.dll). No Debug2021 folder. Maybe you run the plugin twice ;)

This is why in my opinion the path in RunnerCommand.cs is correct. This could also be the reason for some strange timing problems..

Tobias

sylvesp commented 2 years ago

Hi Tobias,

I am not setting the addin file by myself. I have seen the mechanism for creating the addin file automatically and being copied to C:\ProgramData\Autodesk\Revit\Addins\2021\Revit.TestRunner.addin

In the lasted code checked in to the git repository, for a given configuration (e.g. for the Debug2021) we have :

As a result, the addin is not being run twice. The addin is run from the C:\Work\Projects\Revit.TestRunner\src\Revit.testRunner\bin\Debug2021 folder and so when resolving the location of the TestRunner.App using Path.Combine( file.Directory.FullName, @"..\Client\Revit.TestRunner.App.exe" ); becomes "C:\Work\Projects\Revit.TestRunner\src\Revit.TestRunner\bin\Debug2021..\Client\Revit.TestRunner.App.exe" when simplified it is "C:\Work\Projects\Revit.TestRunner\src\Revit.TestRunner\bin\Client\Revit.TestRunner.App.exe" and unless I am missing something that is a non-existent directory.

Still looking at the timing issue.

tobiasfloescher-geberit commented 2 years ago

Ok, I think I found the problem.

There is a miss configuration in the build settings. It must build under x64. In Any CPU the project are not built correctly (ex. bin\Debug2021\ instead of ..\bin\Addin\ for Revit.testRunner project).

Clean up all, switch to x64 in Configuration Manager, and rebuild the solution. I would expect all is working then, inclusive the RunnerCommand. Can you please try it out, thank you.

Actually there should not be an Any CPU build configuration. That crept in during the migration to net core.

I will fix this issue.

sylvesp commented 2 years ago

Hi Tobias,

I've done the changes you've suggested above and that indeed solved the issues related to the directory from where the TestRunner.App.exe is started - so far so good. Unfortunately this didn't solve my intermittent issues with the runner actually not loading the test dlls properly.

Finally I've got some more time and I did figure out what root of the problem is. The simplest way to reproduce the problem is:

This is caused by the fact that while the Revit window is hidden (or no UI/API events happen in Revit), Revit will NOT send OnIdle Events to the Revit.TestRunner plugin thus the plugin will never run the non-concurrent routes and causing the ExploreRequest to expire.

The OnIdle Event will be sent to the TestRunner plugin if there is any Windows UI Event affecting the Revit window, e.g. you move your mouse across the Revit Window before the ExploreRequest request expires.

To get a better visual on this problem add a log entry to your OnIdle() handler

    private void OnIdle( object sender, Autodesk.Revit.UI.Events.IdlingEventArgs e )
    {
        Log.Info( "**** Idle ****");
        mUiApplication = sender as UIApplication;
        mServer.ProceedNextNotConcurrent();
    }

And then see the log.

Well.. that's for now. Although this 'easiest to reproduce' scenario may not be a 100% realistic one, still the problem with the ExploreRequests expiring because of no triggers of the Revit OnIdle status in the Revit plugin could be quite problematic, e.g. on an automated test machine.

Let me know your thoughts.

tobiasfloescher-geberit commented 2 years ago

Hi Peter

wow, thanks for sharing your experience. I could reproduce the issue with your description in Revit 2021. Since I mostly use Revit 2020, I did not face the problem, even with minimized Revit. The problem does not exist in Revit 2020, although the docu says it.

There would be a workaround with call of SetRaiseWithoutDelay function, but with performance degradation, haven't tested it so far. Maybe it's time to add a possibility to activate/deactivate option for the server.

I have to think little bit more about it, and unfortunately haven't that much time at the moment.

But I will take care of the problem and keep you updated. To run it on a automated test machine, we have some other issues, example how to accept the security popup without user interaction.

If you have some good inputs, let me know ;)

sylvesp commented 2 years ago

I did ask Mrs. Google about: revit trigger ideling event What I've found is that this article could be of tremendous help: Poke Revit

Obviously there could be quite a few different ways to make use of this.

My first thought would be that there should be a new kind of request that also runs parallel to the NoParameterDto. This new request could be called something like: TriggerExploreRequest, or Trigger(Parallel)ProcessingRequest. (or whatever other good name you can come up with :) )

Whenever the Runner.App posts an ExploreRequest (that will be processed sequentially) it should also post a TriggerProcessingRequest that will be processed in the parallel processor. This TriggerProcessingRequest will PokeRevit like described in the article above to trigger the Idle Event in Revit.

Seems that these changes could be easily implemented.

On the other hand - as I've said - this is just a very first idea that I didn't even think it through really well. I will keep thinking and let you know of my progression :)

Maybe you can just keep PokingRevit in the task that runs the concurrent routes? I don't see much harm in this right now, but I'll have to thing a bit more about it.

tobiasfloescher-geberit commented 2 years ago

Now I had little bit time to have a look at PokeRevit.

It looks helpful to trigger the OnIdle event. I think it will work. I have trouble to reproduce the issue. I could reproduce it, but unfortunately only once. Makes it little complicated to test this solution. For a quick win, you can use RevitPoke and trigger it in the endless loop of StartConcurrentRoutes().

/// <summary>
/// Start endless loop of all concurrent routes.
/// </summary>
public void StartConcurrentRoutes()
{
    Task.Run( () => {
        while( true ) {
            foreach( var route in mRoutes.Where( r => r.Concurrent ) ) {
                var request = GetRouteRequest( route );

                if( request != null ) {
                    request.Execute( BasePath );
                    break;
                }
            }

            PokeRevit();
            Thread.Sleep( 200 );
        }
    } );
}

...

/// <summary>
/// Trigger Revit.OnIdle Event.
/// https://forums.autodesk.com/t5/revit-api-forum/how-to-trigger-onidle-event-or-execution-of-an-externalevent/td-p/6645286
/// </summary>
private void PokeRevit()
{
    Process revitProcess = Process.GetCurrentProcess();

    if( !revitProcess.HasExited ) {
        try {
            IntPtr mainWindowHandle = revitProcess.MainWindowHandle;
            if( mainWindowHandle.ToInt64() != 0 ) {
                PostMessage( mainWindowHandle, 0, 0, 0 );
            }
        }

        catch {
            // ignore
        }

    }
}

[DllImport( "USER32.DLL" )]
public static extern bool PostMessage( IntPtr hWnd, uint msg, uint wParam, uint lParam );

But I think it is only symptom control. A solution without OnIdle would probably be better. For example, with ExternalEvent and RevitTask. But this is only thinking. The problem is the same with all non-concurrent routes. Meaning there should not be any non-concurrent routes at all.

To implement this, a change in the communication protocol is necessary. And this must be well considered (response, queueing, …)

For now, please make the change as you like in your fork or downloaded solution. unfortunately i have to deal with it at a later point in time.

Hava a nice weekend, tobias

tobiasfloescher-geberit commented 2 years ago

I have integrated the SetRaiseWithoutDelay version in the newest release. this keeps the OnIdle event alive.