pnp / powershell

PnP PowerShell
https://pnp.github.io/powershell
MIT License
679 stars 347 forks source link

[BUG] PnP and Azure functions assembly conflict. Could not load type 'Microsoft.Extensions.Logging.Abstractions.Internal.NullScope' from assembly 'Microsoft.Extensions.Logging.Abstractions' #2136

Closed jmartncp closed 1 year ago

jmartncp commented 2 years ago

Reporting an Issue or Missing Feature

This is an issue related to PnP using a conflicting assembly dependency than what the Azure functions service loads by default. To be more specific, the issue occurs for us with the module Microsoft.Extensions.Logging.Abstractions on each call of Connect-PnPOnline. PowerShell 7.2 on Azure functions automatically loads version 3.0.3.0 while the one included with PnP is 2.2.0.0. I have tried many potential resolutions but have been unable to fix (Wrapping in ThreadJob does not work). The one solution about downgrading from 7.2 to 7.0 is not an option for us, as we require 7.2 for its functionalities. Due to this issue, we are unable to rely on PnP for our needs at the moment. If a fix for PnP in this scenario is possible, please do. If not, then notify of any possible workarounds. This problem had me stumped for a while, so anything that can be done about it is appreciated.

Expected behavior

Successful connection via Connect-PnPOnline cmdlet.

Actual behavior

An error related to a module in Azure functions: Screenshot 2022-07-05 094943

Steps to reproduce behavior

NOTE: To reproduce, ensure code is run in the cloud within an Azure function. Running Connect-PnPOnline locally will likely NOT cause an error. (The Azure assemblies won't be loaded)

What is the version of the Cmdlet module you are running?

PnP.PowerShell 1.10.0, PowerShell Core 7.2

Which operating system/environment are you running PnP PowerShell on?

Ofer-Gal commented 1 year ago

"functionTimeout": "10:00:00", is 10 hours?

Micael-stack commented 1 year ago

@Ofer-Gal yes, so when you're in basic plan, the maximum is : functionTimeout": "00:10:00" And for premium : functionTimeout": "00:30:00"

We are talking about the host.json file of course! But we are getting out of scope for this issue!

S44dHumayun commented 1 year ago

How do i force PowerShell version to 7.0 in an Azure function?

Micael-stack commented 1 year ago

@SaadKnowIT You can't anymore from the interface, this is exacly what MS removed!

I don't know if by Powershell it still possible or by templating bicep.

Francisco-Gamino commented 1 year ago

@Francisco-Gamino can you comment on the rollout of the updated Azure Functions in January and the situation between now and then with PowerShell 7.0 Azure Functions being deprecated?

The PowerShell 7.2 language worker with the fix will start rolling out January 3rd, and it is expected to complete in one or two weeks. In the meantime, we are committing to provide support for the apps that are on PowerShell 7.0 until this deployment completes.

/cc @anirudhgarg @AnatoliB @MadhuraBharadwaj-MSFT @michaelpeng36 @VpOfEngineering

To downgrade your function app from PowerShell 7.2 to PowerShell 7.0, follow these steps:

Prerequisites: Az.Functions and Az.Resources, which can be downloaded from the PowerShell Gallery.

$SubId = "<your sub id>"
$ResourceGroupName = "<your resource group name>"
$AppName = "your app name"

Set-AzContext -Subscription $Subscription | Out-Null

For a Windows function app

Set-AzResource -ResourceId "/subscriptions/$SubId/resourceGroups/$ResourceGroupName/providers/Microsoft.Web/sites/$AppName/config/web" -UsePatchSemantics -Properties @{ powerShellVersion = '~7' } -Force

# Restart your app for the version changes to take effect
Restart-AzFunctionApp -Name $AppName -ResourceGroupName $ResourceGroupName -Force

# Verify that version was updated
$app = Get-AzFunctionApp -Name $AppName -ResourceGroupName $ResourceGroupName 
$app.SiteConfig.PowerShellVersion

For a Linux function app

Set-AzResource -ResourceId "/subscriptions/$SubId/resourceGroups/$ResourceGroupName/providers/Microsoft.Web/sites/$AppName/config/web" -UsePatchSemantics -Properties @{ linuxFxVersion = 'PowerShell|7' } -Force

# Restart your app for the version changes to take effect
Restart-AzFunctionApp -Name $AppName -ResourceGroupName $ResourceGroupName -Force

# Verify that version was updated
$app = Get-AzFunctionApp -Name $AppName -ResourceGroupName $ResourceGroupName 
$app.SiteConfig.LinuxFxVersion

If you have any questions on why PowerShell 7.0 is "~7", please see https://github.com/Azure/azure-functions-powershell-worker/issues/885

TMVUpShare commented 1 year ago

I'm trying to get PnP Get-PnPSiteTemplate and Invoke-PnPSiteTemplate to work, but as all of you have described i run into this error. I can't downgrade to 7.0 so it looks as though i can't get this to work at all.

If i try to use Get-PnPSiteTemplate and save that file to SharePoint it dosen't create any files, if i try to save locally in my Function app it dosen't create any files. So that seems to not work at all. If i run it on my own machine it works as it should.

If i then try and run Invoke-PnPSiteTemplate it does actually look like it works but again it dosen't. It runs and do add the template to a site, but not all. The left menu dosen't get finished, pages don't get changes and so on. Again if i run it on my own machine it works.

So is it really the correct conclusion that PnP provisioning from a Azure function cannot work, and all we can do is wait on updates?

Is there any alternatives to PnP provisioning because i'm actually not that keen on using PnP seeing as it dosn't seem like a good idea to build business solutions on?

gabbsmo commented 1 year ago

Please try to stay on topic. This error about the error in the title. Create new issues for other problems.

TMVUpShare commented 1 year ago

Please try to stay on topic. This error about the error in the title. Create new issues for other problems.

I'm getting the same error as in this thread

2022-12-11T07:19:52Z [Error] ERROR: Could not load type 'Microsoft.Extensions.Logging.Abstractions.Internal.NullScope

So it looks like a general problem with PnP and 7.2

Ofer-Gal commented 1 year ago

As of today, I can't select PowerShell 7.0 anymore and both 1.12.0 and 1.11.0 show the error Could not load type 'Microsoft.Extensions.Logging.Abstractions.Internal.NullScope Any solution in the future?

gautamdsheth commented 1 year ago

@Ofer-Gal - Right now, you can create a PS 7.2 function app and downgrade to 7.0 using the script in the comments above here. You will be able to use PnP PS 1.11 and 1.12

After mid-Jan, when Microsoft's rollout of the fix is completed, you will be able to use PnP PowerShell in PS 7.2 function app. But then, you will need to use the PnP PowerShell nightly build's which are based on .NET 6.

Ofer-Gal commented 1 year ago

Now it runs only with 1.12.0 but the statement: Set-PnPListItem -List $sitesList -Identity $item -Values @{ "LastMaintenance" = (Get-Date) ; } -Connection $appConn Errors: EXCEPTION: The collection has not been initialized. It has not been requested or the request has not been executed. It may need to be explicitly requested. same code runs fine on the desktop. is there a another way to do the update ?

myatix commented 1 year ago

I am also having an issue with my Azure Function... I have downgraded the Azure Function to .Net Core 7.0 and am using PnP.PowerShell v1.12.0 with Connect-PnPOnline and -CertificateBase64Encoded authentication to connect to a AzureKeyVault

I have numerous errors with thing like Get-PnPList which works fine on my local client connecting via -Interactive but not in the Function APP using -CertificateBase64Encoded? Super frustrating... Any suggestions would be appreciated on how to get around this?

Francisco-Gamino commented 1 year ago

@Francisco-Gamino can you comment on the rollout of the updated Azure Functions in January and the situation between now and then with PowerShell 7.0 Azure Functions being deprecated?

The PowerShell 7.2 language worker with the fix will start rolling out January 3rd, and it is expected to complete in one or two weeks. In the meantime, we are committing to provide support for the apps that are on PowerShell 7.0 until this deployment completes.

/cc @anirudhgarg @AnatoliB @MadhuraBharadwaj-MSFT @michaelpeng36 @VpOfEngineering

The Functions release with the fix for the PowerShell 7.2 language worker has been delayed. Currently, the release is set to start deploying on January 17th, 2023 and it is expected to be completed in one or two weeks if no blocking issues are found. I will continue to update this issue with the deployment status as more information becomes available. Thank you.

gajcowan commented 1 year ago

Do we expect the fix to also be rolled out at the same time to Azure Function Core Tools as we get the same issue with 4.9.495 x64, which means it is not possible to test locally before deployment.

The DLL Hell, if I remove the "FUNCTIONS_WORKER_RUNTIME_VERSION" : "7.2" statement from my settings it works, the MSGraph PowerShell modules no longer longer work due to a different DLL issue!!!!!

Ofer-Gal commented 1 year ago

How were you able to delete the "FUNCTIONS_WORKER_RUNTIME_VERSION"? I only see it as a dropdown with 7.0 and 7.2 options. Is there a file I can edit anywhere?

myatix commented 1 year ago

How were you able to delete the "FUNCTIONS_WORKER_RUNTIME_VERSION"? I only see it as a dropdown with 7.0 and 7.2 options. Is there a file I can edit anywhere?

@Ofer-Gal What are you trying to achieve? To role back to version 7.0 you can use a PowerShell script.

# Configure App to use .Net Core 7.0
$SubId = "<SubScriptionID>"
$ResourceGroupName = "RGXXX"
$AppName = "func-AppName"
$TenantID = "TenantID"

Connect-AzAccount -Tenant $TenantID -SubscriptionId $SubId

Set-AzContext -Subscription $SubId | Out-Null

Set-AzResource -ResourceId "/subscriptions/$SubId/resourceGroups/$ResourceGroupName/providers/Microsoft.Web/sites/$AppName/config/web" -UsePatchSemantics -Properties @{ powerShellVersion = '~7' } -Force

# Restart your app for the version changes to take effect
Restart-AzFunctionApp -Name $AppName -ResourceGroupName $ResourceGroupName -Force

# Verify that version was updated
$app = Get-AzFunctionApp -Name $AppName -ResourceGroupName $ResourceGroupName 
$app.SiteConfig.PowerShellVersion

Or you can see the FUNCTIONS_WORKER_RUNTIME_VERSION locally in VSCode in the local.settings.json file. image

Ofer-Gal commented 1 year ago

I thought you removed it altogether. I got it back to 7.0 with the script too. but it still errors on simple update. [Error] EXCEPTION: The collection has not been initialized. It has not been requested or the request has not been executed. It may need to be explicitly requested.

myatix commented 1 year ago

@Ofer-Gal On your connection string have you remembered to add the parameter -returnconnection?

Ofer-Gal commented 1 year ago

I did.

sli701 commented 1 year ago

As of 2023-01-12. the issue persists. And there is no option to go back to PowerShell 7.0.

myatix commented 1 year ago

@sli701 You can use the powershell script above to rollback to powershell core 7.0 on your function spp.

sli701 commented 1 year ago

@myatix , was able to give a try on the Cloud Shell, it is able to rollback. not sure why it is not working when using Local Power Shell Command. However, it does work on the cloud shell, thanks for the help!

gabbsmo commented 1 year ago

@Francisco-Gamino Has the fix for Azure Functions been released? If yes, is it available in the West Europe region?

Francisco-Gamino commented 1 year ago

The Functions release with the fix for the PowerShell 7.2 language worker has been delayed. Currently, the release is set to start deploying on January 17th, 2023 and it is expected to be completed in one or two weeks if no blocking issues are found. I will continue to update this issue with the deployment status as more information becomes available. Thank you.

The Functions version 14.15.1 deployment that was supposed to start on 1/17/23 has been delayed. However, the deployment started rolling out today 1/23/23, and it is expected to be completed by the beginning of next week if no blocking issues are found.

/cc @anirudhgarg @AnatoliB @MadhuraBharadwaj-MSFT @michaelpeng36 @VpOfEngineering

VpOfEngineering commented 1 year ago

Hi everyone, we're still facing delays in rolling out 4.15.1. We will keep updates here as we fix the issues and continue the rollout. Thank you all for your patience.

karthikramasubramanian commented 1 year ago

Hi everyone, we're still facing delays in rolling out 4.15.1. We will keep updates here as we fix the issues and continue the rollout. Thank you all for your patience.

Do we have any update on when we can expect the rollout of '4.15.1'

VpOfEngineering commented 1 year ago

Barring any issues, we should be fully rolled out by the beginning of next week.

Clam- commented 1 year ago

Will any action need to be taken? Will our currently failing Azure Functions start working again next week without interaction? Thank you.

svermaak commented 1 year ago

Will functions where a workaround was applied stop working?

Get Outlook for Androidhttps://aka.ms/AAb9ysg


From: Clam- @.> Sent: Tuesday, January 31, 2023 2:37:26 PM To: pnp/powershell @.> Cc: svermaak @.>; Comment @.> Subject: Re: [pnp/powershell] [BUG] PnP and Azure functions assembly conflict. Could not load type 'Microsoft.Extensions.Logging.Abstractions.Internal.NullScope' from assembly 'Microsoft.Extensions.Logging.Abstractions' (Issue #2136)

Will any action need to be taken? Will our currently failing Azure Functions start working again next week without interaction? Thank you.

— Reply to this email directly, view it on GitHubhttps://github.com/pnp/powershell/issues/2136#issuecomment-1409704908, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AC5RDEFS5GWCNNYKOSN63UDWVCCHNANCNFSM53M6HTGQ. You are receiving this because you commented.Message ID: @.***>

Francisco-Gamino commented 1 year ago

The PowerShell 7.2 language worker which is being deployed with 4.15.1 updates Microsoft.Extensions.Logging.Abstractions to version 6.0.3. If the workaround was applied to the PowerShell 7.0 language worker, then this should continue to work as no changes has been introduced to this version given that it has reached EOL.

@KoenZomers -- Could you please comment on next steps after 4.15.1 completes rolling out? Thank you.

gautamdsheth commented 1 year ago

Just tested this in an environment which has version 4.15.x.x , works perfectly fine !

To fix this issue in your environment, please wait till the fix is there. I am guessing it will be available very soon worldwide.

I tested this in a resource group which is in North Europe region.

Once the fix is available, you will need to update the PnP PowerShell to the latest nightly builds.

You need to explicitly specify the version number like 2.0.1-nightly or later. The code will keep working as it is.

Once a major release of PnP PowerShell is done, you can also specify it as a wildcard entry 2.x

thunderstorm654 commented 1 year ago

It would be good to know when the next major release of PnP will be out that works with the Azure fix. Some clients don't like to use nightly versions in production :) thanks!

Clam- commented 1 year ago

Just tested this in an environment which has version 4.15.x.x

How does one know if their environment is 4.15.x?

You need to explicitly specify the version number like 2.0.1-nightly or later.

How does one do this?

Thank you.

gautamdsheth commented 1 year ago

@Clam-

It is in the Azure function overview page:

image

You can specify the version of PnP PowerShell here:

App Files > Requirements.psd1 file

image

gyadav-HBS commented 1 year ago

As a workaround, my Azure functions were working fine with PS 7.0. and PnP.PowerShell 1.2. To test the above fix I updated the functions to use PS 7.2 and switched PnP.PowerShell to 2.0.6-nightly. And I am seeing the following exception now when making a call to New-PnPWeb. Any thoughts??

ERROR: System.Net.Http.HttpRequestException: Error while copying content to a stream. ---> System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host.. ---> System.Net.Sockets.SocketException (10054): An existing connection was forcibly closed by the remote host. --- End of inner exception stack trace --- at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken) at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token) at System.Net.Security.SslStream.EnsureFullTlsFrameAsync[TIOAdapter](TIOAdapter adapter) at System.Net.Security.SslStream.ReadAsyncInternal[TIOAdapter](TIOAdapter adapter, Memory`1 buffer) at System.Net.Http.HttpConnection.FillAsync(Boolean async) ................................. .................................

jrudley commented 1 year ago

If you are not running a consumption plan, but under an app service plan, I ran into a weird issue and found a workaround. I was on a B2 sku and kept getting the nullscope error. I was not downgrading my ps version due to other production functions running on the same app plan. Maybe Azure gov does not back patch existing hosts or not fully finished yet. To make it work, upgrade your plan to a PV2/V3 and it will move it to a new host which should have the patched runtime. I verified it worked, then moved it back to a B2 plan which was put on a host that was patched as well.

desmay commented 1 year ago

@Francisco-Gamino Do we know if 4.15.1 Azure function updates were pushed to Azure Gov ?

gautamdsheth commented 1 year ago

Going to close this issue. It has been fixed from Microsoft side when using Az function 4.15.x or later version. To use PnP PowerShell, as we mentioned, please use the latest 2.x.x-nightly builds.

@desmay - no idea if this has been fixed for Azure Gov, but you can reach out to Microsoft support/PG for more information on this.

We have no ETA on the major release of PnP PowerShell as of now.

karthikramasubramanian commented 1 year ago

Hi All,

I recently upgraded PnP.Powershell from the 2.x.x-nightly to 2.1.1 and can confirm that it is working fine. I have deployed it to our prod tenant for almost a month now so far no errors or issues..

Regards

Karthik Ramasubramanian Senior Digital Productivity Developer

Norton Rose Fulbright Canada LLP / S.E.N.C.R.L., s.r.l. 45 O'Connor Street, Suite 1500, Ottawa, ON K1P 1A4, Canada

T: +1 613.780.8674 | M: +1 613.277.2840 | F: +1 613.230.5459

@.**@.> NORTON ROSE FULBRIGHT

From: Fringie @.> Sent: May 18, 2023 9:45 AM To: pnp/powershell @.> Cc: Karthik Ramasubramanian (he/him/his) @.>; Comment @.> Subject: Re: [pnp/powershell] [BUG] PnP and Azure functions assembly conflict. Could not load type 'Microsoft.Extensions.Logging.Abstractions.Internal.NullScope' from assembly 'Microsoft.Extensions.Logging.Abstractions' (Issue #2136)

[External Email – Use Caution]

Any update on this?

Going to close this issue. It has been fixed from Microsoft side when using Az function 4.15.x or later version. To use PnP PowerShell, as we mentioned, please use the latest 2.x.x-nightly builds.

@desmayhttps://github.com/desmay - no idea if this has been fixed for Azure Gov, but you can reach out to Microsoft support/PG for more information on this.

We have no ETA on the major release of PnP PowerShell as of now.

I am getting this issue with: Azure Functions Core Tools Core Tools Version: 4.0.5148 Commit hash: N/A (64-bit) Function Runtime Version: 4.17.3.20392

Can you advise?

— Reply to this email directly, view it on GitHubhttps://github.com/pnp/powershell/issues/2136#issuecomment-1553079938, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AKJU7QZIEUY6ZD42RJIHA6LXGYRVVANCNFSM53M6HTGQ. You are receiving this because you commented.Message ID: @.**@.>>

Law around the world Le droit à l’échelle mondiale nortonrosefulbright.com

CONFIDENTIALITY NOTICE: This email is confidential and may be privileged. If you are not the intended recipient please notify the sender immediately and delete it. AVIS DE CONFIDENTIALITÉ : Ce courriel est confidentiel et peut être protégé par le secret professionnel. Si vous n’en êtes pas le destinataire visé, veuillez en aviser l’expéditeur immédiatement et le supprimer.