search

magic-chunks / magic-chunks-dotnetcore

Easy to use tool to config transformations for JSON, XML and YAML.
MIT License
106 stars 49 forks source link

Could not load file or assembly 'System.IO.FileSystem' #42

Closed petr-stupka closed 3 years ago

petr-stupka commented 6 years ago

Hey Sergey,

looks like the new version doesn't work well.

Please let me know if you need more info.

2017-12-16T17:27:56.0021636Z ============================================================================== 2017-12-16T17:27:56.0021853Z Task : Config transformation 2017-12-16T17:27:56.0022091Z Description : Transform config file with Magic Chunks 2017-12-16T17:27:56.0022254Z Version : 1.3.1 2017-12-16T17:27:56.0022437Z Author : Sergey Zwezdin 2017-12-16T17:27:56.0022609Z Help : More Information 2017-12-16T17:27:56.0022797Z ============================================================================== 2017-12-16T17:27:56.4835847Z Preparing task execution handler. 2017-12-16T17:27:57.1161849Z Executing the powershell script: C:\VSTSAgents\Agent01_work_tasks\MagicChunks_985284e0-a7d2-4e4d-802c-0a516bffaadf\1.3.1\transform.ps1 2017-12-16T17:27:57.1260313Z "version": "1.0.830" 2017-12-16T17:27:57.1260539Z }) 2017-12-16T17:27:57.1313342Z ##[error]System.Reflection.ReflectionTypeLoadException: Unable to load one or more of the requested types. Retrieve the LoaderExceptions property for more information. 2017-12-16T17:27:57.8024964Z at System.Reflection.RuntimeModule.GetTypes(RuntimeModule module) 2017-12-16T17:27:57.8113769Z at System.Reflection.Assembly.GetTypes() 2017-12-16T17:27:57.8114233Z at Microsoft.PowerShell.Commands.AddTypeCommand.LoadAssemblyFromPathOrName(List1 generatedTypes) 2017-12-16T17:27:57.8114760Z at Microsoft.PowerShell.Commands.AddTypeCommand.EndProcessing() 2017-12-16T17:27:57.8115031Z at System.Management.Automation.CommandProcessorBase.Complete() 2017-12-16T17:27:57.8115269Z Transformation found: version: 1.0.830 2017-12-16T17:27:57.8115437Z 2017-12-16T17:27:57.8115577Z 2017-12-16T17:27:57.8115796Z Transforming file C:\VSTSAgents\Agent01\_work\1\s\vss-extension.json 2017-12-16T17:27:57.8115985Z 2017-12-16T17:27:57.8116120Z 2017-12-16T17:27:57.8117825Z ##[error]System.Management.Automation.MethodInvocationException: Exception calling "Transform" with "4" argument(s): "Could not load file or assembly 'System.IO.FileSystem, Version=4.0.1.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified." ---> System.IO.FileNotFoundException: Could not load file or assembly 'System.IO.FileSystem, Version=4.0.1.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified. 2017-12-16T17:27:57.8118521Z at MagicChunks.TransformTask.Transform(String type, String sourcePath, String targetPath, TransformationCollection transformation) 2017-12-16T17:27:57.8118881Z at CallSite.Target(Closure , CallSite , Type , Object , Object , Object , Object ) 2017-12-16T17:27:57.8119172Z --- End of inner exception stack trace --- 2017-12-16T17:27:57.8119478Z at System.Management.Automation.ExceptionHandlingOps.CheckActionPreference(FunctionContext funcContext, Exception exception) 2017-12-16T17:27:57.8121074Z at System.Management.Automation.Interpreter.ActionCallInstruction2.Run(InterpretedFrame frame) 2017-12-16T17:27:57.8121468Z at System.Management.Automation.Interpreter.EnterTryCatchFinallyInstruction.Run(InterpretedFrame frame) 2017-12-16T17:27:57.8121808Z at System.Management.Automation.Interpreter.EnterTryCatchFinallyInstruction.Run(InterpretedFrame frame) 2017-12-16T17:27:57.8126586Z ##[error]PowerShell script completed with 2 errors.

petr-stupka commented 6 years ago

btw, VSTS tasks support versioning. I would recommend to use version 2.x.x in case of any major braking changes. So we can use the old version and then eventualy switch to new version if required.

sergeyzwezdin commented 6 years ago

@schwaizi Could you share your transformations? I tested it myself and it works ok. It seems something specific to your configuration.

sergeyzwezdin commented 6 years ago

Also, do you have .net core installed on that machine?

ghost commented 6 years ago

Dear Sergey,

we also have a similar issue since yesterday but did not change anything in our configuration. We are still using V1 of magic chunks, which worked fine so far. Here is the transform log (I took out some contextual info) 2017-12-17T08:44:12.4404349Z ##[section]Starting: Update environment variables in web.config of service layer 2017-12-17T08:44:12.4404349Z ============================================================================== 2017-12-17T08:44:12.4404349Z Task : Config transformation 2017-12-17T08:44:12.4404349Z Description : Transform config file with Magic Chunks 2017-12-17T08:44:12.4404349Z Version : 1.3.1 2017-12-17T08:44:12.4404349Z Author : Sergey Zwezdin 2017-12-17T08:44:12.4404349Z Help : More Information 2017-12-17T08:44:12.4404349Z ============================================================================== 2017-12-17T08:44:12.4404349Z Preparing task execution handler. 2017-12-17T08:44:12.7841857Z Executing the powershell script: E:\Agent\Release_INT_1_work_tasks\MagicChunks_985284e0-a7d2-4e4d-802c-0a516bffaadf\1.3.1\transform.ps1 2017-12-17T08:44:12.7841857Z "configuration/appSettings/add[@key='sApiAddress']/@value": "http://valid url", 2017-12-17T08:44:12.7841857Z "configuration/appSettings/add[@key='authApiAddress']/@value": "http://valid url", 2017-12-17T08:44:12.7841857Z "configuration/appSettings/add[@key='cLogApiAddress']/@value": "http://valid url/", 2017-12-17T08:44:12.7841857Z "configuration/appSettings/add[@key='LockApiAddress']/@value": "http://valid url", 2017-12-17T08:44:12.7841857Z "configuration/appSettings/add[@key='DataApiAddress']/@value": "http://valid url", 2017-12-17T08:44:12.7841857Z "configuration/ConnectionStrings/add[@name='CDataContext']/@connectionString": "data source=valid data source;initial catalog=blabla;integrated security=True;MultipleActiveResultSets=True;App=EntityFramework", 2017-12-17T08:44:12.7841857Z "configuration/system.web/compilation/@debug": "false" 2017-12-17T08:44:12.7841857Z }) 2017-12-17T08:44:12.9404412Z ##[error]System.Reflection.ReflectionTypeLoadException: Unable to load one or more of the requested types. Retrieve the LoaderExceptions property for more information. 2017-12-17T08:44:12.9404412Z at System.Reflection.RuntimeModule.GetTypes(RuntimeModule module) 2017-12-17T08:44:12.9404412Z at System.Reflection.Assembly.GetTypes() 2017-12-17T08:44:12.9404412Z at Microsoft.PowerShell.Commands.AddTypeCommand.LoadAssemblyFromPathOrName(List1 generatedTypes) 2017-12-17T08:44:12.9404412Z at Microsoft.PowerShell.Commands.AddTypeCommand.EndProcessing() 2017-12-17T08:44:12.9404412Z at System.Management.Automation.CommandProcessorBase.Complete() 2017-12-17T08:44:12.9560615Z Transformation found: configuration/appSettings/add[@key='sApiAddress']/@value: http://valid url 2017-12-17T08:44:12.9560615Z 2017-12-17T08:44:12.9560615Z 2017-12-17T08:44:12.9716817Z Transformation found: configuration/appSettings/add[@key='authApiAddress']/@value: http://valid url 2017-12-17T08:44:12.9716817Z 2017-12-17T08:44:12.9716817Z 2017-12-17T08:44:12.9716817Z Transformation found: configuration/appSettings/add[@key='cLogApiAddress']/@value: http://valid url 2017-12-17T08:44:12.9716817Z 2017-12-17T08:44:12.9716817Z 2017-12-17T08:44:12.9716817Z Transformation found: configuration/appSettings/add[@key='LockApiAddress']/@value: http://valid url 2017-12-17T08:44:12.9716817Z 2017-12-17T08:44:12.9716817Z 2017-12-17T08:44:12.9716817Z Transformation found: configuration/appSettings/add[@key='DataApiAddress']/@value: http://valid url 2017-12-17T08:44:12.9716817Z 2017-12-17T08:44:12.9716817Z 2017-12-17T08:44:12.9716817Z Transformation found: configuration/ConnectionStrings/add[@name='CDataContext']/@connectionString: data source=valid data source;initial catalog=blabla;integrated security=True;MultipleActiveResultSets=True;App=EntityFramework 2017-12-17T08:44:12.9716817Z 2017-12-17T08:44:12.9716817Z 2017-12-17T08:44:12.9716817Z Transformation found: configuration/system.web/compilation/@debug: false 2017-12-17T08:44:12.9716817Z 2017-12-17T08:44:12.9716817Z 2017-12-17T08:44:12.9716817Z Transforming file E:\Agent\Release_INT_1\_work\r5\a\CService-BranchMain\service\_PublishedWebsites\Service.Api\Web.config 2017-12-17T08:44:12.9716817Z 2017-12-17T08:44:12.9716817Z 2017-12-17T08:44:12.9873117Z ##[error]System.Management.Automation.MethodInvocationException: Exception calling "Transform" with "4" argument(s): "Could not load file or assembly 'System.IO.FileSystem, Version=4.0.1.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified." ---> System.IO.FileNotFoundException: Could not load file or assembly 'System.IO.FileSystem, Version=4.0.1.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified. 2017-12-17T08:44:12.9873117Z at MagicChunks.TransformTask.Transform(String type, String sourcePath, String targetPath, TransformationCollection transformation) 2017-12-17T08:44:12.9873117Z at CallSite.Target(Closure , CallSite , Type , Object , Object , Object , Object ) 2017-12-17T08:44:12.9873117Z --- End of inner exception stack trace --- 2017-12-17T08:44:12.9873117Z at System.Management.Automation.ExceptionHandlingOps.CheckActionPreference(FunctionContext funcContext, Exception exception) 2017-12-17T08:44:12.9873117Z at System.Management.Automation.Interpreter.ActionCallInstruction2.Run(InterpretedFrame frame) 2017-12-17T08:44:12.9873117Z at System.Management.Automation.Interpreter.EnterTryCatchFinallyInstruction.Run(InterpretedFrame frame) 2017-12-17T08:44:12.9873117Z at System.Management.Automation.Interpreter.EnterTryCatchFinallyInstruction.Run(InterpretedFrame frame) 2017-12-17T08:44:13.0029354Z ##[error]PowerShell script completed with 2 errors. 2017-12-17T08:44:13.0029354Z ##[section]Finishing: Update environment variables in web.config of service layer

sergeyzwezdin commented 6 years ago

Ok, I see the issue, Will fix that asap.

sergeyzwezdin commented 6 years ago

@schwaizi @miles1302 There is strange problem with assembly built as .NET Standard. While I'm investigating it, I reverted it back at VSTS marketplace to restore your builds, guys. I will let you know about the progress.

petr-stupka commented 6 years ago

Hi Sergey, the new version seems to fix 'System.IO.FileSystem' error. But new error showed up.

2017-12-17T14:37:20.9115921Z ##[error]System.Management.Automation.MethodInvocationException: Exception calling "Transform" with "4" argument(s): "Could not load file or assembly 'Newtonsoft.Json, Version=10.0.0.0, Culture=neutral, PublicKeyToken=30ad4fe6b2a6aeed' or one of its dependencies. The system cannot find the file specified." ---> System.IO.FileNotFoundException: Could not load file or assembly 'Newtonsoft.Json, Version=10.0.0.0, Culture=neutral, PublicKeyToken=30ad4fe6b2a6aeed' or one of its dependencies. The system cannot find the file specified.

I ran the task on Hosted VS2017 Agent pool where the assembly (nuget package) is not installed. https://github.com/adventworks/hosted-pool-images/blob/2017.10.02/vs2017-on-windows-2016/image.md

I can confirm the problem can be fixed on-premise by installing the package https://www.nuget.org/packages/newtonsoft.json/ But how it can be fixed on Hosted Agents?

sergeyzwezdin commented 6 years ago

@schwaizi Which version of extension do you see in console during the build?

==============================================================================
Task         : Config transformation
Description  : Transform config file with Magic Chunks
Version      : 1.3.10
Author       : Sergey Zwezdin
Help         : [More Information](https://github.com/sergeyzwezdin/magic-chunks)
==============================================================================
sergeyzwezdin commented 6 years ago

@schwaizi I can confirm that I'm able to run 1.3.10 both on hosted and on-premise agent. Could you check your agent version?

petr-stupka commented 6 years ago

Hi Segey, i can confirm the 1.3.10 version now. This version works fine now. The last issue is linked to the new version with the bugfix (before you made rollback). As i said, i would prefer the next version will be 2.x.x so ot will not affect our system directly, but after version change of the task in our system.

sergeyzwezdin commented 6 years ago

@schwaizi Ok, I just wanted to ensure it's working for you now. I'll keep investigating. As for version - I'm not very happy with changing version to 2.x as it isn't very big release, but it seems there is no way to manage versions for VSTS extensions, so I'll change it to 2.x for sure.

petr-stupka commented 6 years ago

Many thanks Sergey, you doing great job! I know tasks update in TFS must be manually triggered, but VSTS take the new version imidiately so that’s the issue if the main version of the task don’t change. May be something to address with MS :) Hope it didn’t affected your weekend plans!

ghost commented 6 years ago

Hello Sergey, it's working for me also. Thank you for the quick fix

Regards

sergeyzwezdin commented 6 years ago

Thank you for your kind of word. I'll continue investigation. Don't have solution yet.

sergeyzwezdin commented 6 years ago

Hello everyone!

I ran into the strange issue. For some build agents (like my own and Hosted VS2017) it works fine, but for another is not (for example Hosted agent). I'll continue my invetigation in this area. Asked about that here as well - https://github.com/adventworks/hosted-pool-images/issues/12

Any help/suggestions would be very welcome! :-)

RowlandShaw commented 6 years ago

I'm also seeing the reported issue with Version : 2.0.3 with an on-premise build server - could this just be some pre-requisites missing in the package?

petr-stupka commented 6 years ago

Hi, i just found the issue is related to .NET Framework. I installed the latest one and now is working fine. https://www.microsoft.com/net/download/windows

I guess the Framework is not installed on Hosted Agent but it is on Hosted Agent 2017. You can find the inventory of installed SW here: https://docs.microsoft.com/en-us/vsts/build-release/concepts/agents/hosted

And i see the Hosted is missing version 4.7 as example. you can request it thru MS Support eventually :)

sergeyzwezdin commented 6 years ago

@schwaizi Yeah. I did the same with my local agent. I see this and this - they has 4.7.1 and 4.6.2 versions installed. It should be enough - it least it works fine at local controller with 4.6.2. For now I'm a bit busy with my main work. once I'll have a bit spare time - I'll return to investiation.

sevaa commented 6 years ago

Getting the "Could not load file or assembly 'System.IO.FileSystem, Version=4.0.1.0" error here. Version 2.0.3, that's the latest in the Marketplace. Task version is 2.0.3 too.

On-premises TFS 2017 u2. Config Transform v1 runs normally.

erradimourad commented 6 years ago

I had the same issue for my local build agent, and after updating my .Net Framework Version to 4.7, it start working fine.

sevaa commented 6 years ago

Tried that, didn't work. Still can't found the same assembly.

JulioIzquierdo commented 6 years ago

Version: 2.0.3 Error: Exception calling "Transform" with "4" argument(s): "Could not load file or assembly 'System.IO.FileSystem, Version=4.0.1.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies .NET Framework: v4.6 that comes with the server Comments: Works in one environment but not in another. Pretty certain it has to do something with which framework is installed. All in the same pipeline. Any thoughts anyone?

petr-stupka commented 6 years ago

@JulioIzquierdo can you try the .Net Framework 4.7 ? You can eventually try https://www.nuget.org/packages/System.IO.FileSystem/4.0.1

JulioIzquierdo commented 6 years ago

@schwaizi , I install the .NET Framework 4.7 and it worked! Thank you for your help on this!

petr-stupka commented 6 years ago

@sevaa can you please post your VSTS configuration. You mentioned TFS 2017 is working fine. I'm trying to understand where is your problem as i think the issue has been solved by installing .net framework 4.7

@sergeyzwezdin i would suggest "Install Instruction" on wiki page. To add the .net framework 4.7 as requirements for on-premise and in cloud availability on "Hosted 2017" only.

wallyhall commented 6 years ago

FYI - we fought this issue today. We had wrongly interpreted the instructions as meaning install .NET Framework 4.7 as a .MSI, rather than install .NET Framework 4.7 via the Visual Studio installer on the build agent. The latter fixed the issue for us.

sevaa commented 6 years ago

Would it be too much to expect the task to run on an unpatched Windows server? :) No one will let me run Visual Studio setup on live agent machines.

staff0rd commented 6 years ago

I am getting this problem on one of two EC2 instances, both running Windows 2016. Image is identical as far as I can tell, on the failing one I'm getting;

2018-05-14T06:30:16.5396700Z ##[error]System.Management.Automation.MethodInvocationException: Exception calling "Transform" with "4" argument(s): "Could not load file or assembly 'System.IO.FileSystem, Version=4.0.1.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified." ---> System.IO.FileNotFoundException: Could not load file or assembly 'System.IO.FileSystem, Version=4.0.1.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified.

Both are however using the same version;

2018-05-14T06:30:07.4152238Z ==============================================================================
2018-05-14T06:30:07.4152390Z Task         : Config transformation
2018-05-14T06:30:07.4152506Z Description  : Transform config file with Magic Chunks
2018-05-14T06:30:07.4152630Z Version      : 2.0.3
2018-05-14T06:30:07.4152733Z Author       : Sergey Zwezdin
2018-05-14T06:30:07.4152852Z Help         : [More Information](https://github.com/sergeyzwezdin/magic-chunks)
2018-05-14T06:30:07.4152997Z ==============================================================================

Will investigate further.

mkokabi commented 6 years ago

I am getting the same error: ##[error]System.Management.Automation.MethodInvocationException: Exception calling "Transform" with "4" argument(s): "Could not load file or assembly 'System.IO.FileSystem, Version=4.0.1.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified." ---> System.IO.FileNotFoundException: Could not load file or assembly 'System.IO.FileSystem, Version=4.0.1.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified.

It's a clean win2018. The only thing installed on it was Agent but as some people here found installing .Net 4.7 is fixing their issue I did install that but still get the issue,

LarsKemmann commented 6 years ago

You need the actual .NET 4.7.x SDK, not just the Runtime. No Visual Studio installer needed -- and for automation, you can use choco install netfx-4.7.2-devpack -- just tried this and it worked for me. :-)

janisBerz commented 6 years ago

I'm also getting an error on VSTS. We are running builds on a private build agent. `

[error]System.Management.Automation.MethodInvocationException: Exception calling "Transform" with "4" argument(s): "Could not load file or assembly 'System.IO.FileSystem, Version=4.0.1.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified." ---> System.IO.FileNotFoundException: Could not load file or assembly 'System.IO.FileSystem, Version=4.0.1.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified.

` I'm trying to transform a web.config xml file. The config file looks like this

<configuration>
  <configSections>
  </configSections>
  <appSettings>
    <add key="BaseUrl" value="http://localhost:5000/"/>
    <add key="DatabaseUrl" value="Server=(LocalDb)\\MSSQLLocalDB;Database=db;Trusted_Connection=True;"/>
  </appSettings>
</configuration>

Not sure if the transformations I'm using with magic-chunks extension are correct. This iwhat i have tried and not of them worked.

{
  "configuration/appSettings/add[@key='BaseUrl']/@value": "http://localhost:10000/"
}

Or

{
"configuration/appSettings/@BaseUrl": "http://localhost:10000/"
}

I've installed .net framework 4.7 sdk as well.

janisBerz commented 6 years ago

I can confirm after installing .Net Framework 4.7.2 DevPack the Extension worked as expected.

choco install netfx-4.7.2-devpack