search

dotnet / runtime

.NET is a cross-platform runtime for cloud, mobile, desktop, and IoT apps.
https://docs.microsoft.com/dotnet/core/
MIT License
15.28k stars 4.74k forks source link

Port remainder of Microsoft.VisualBasic #26866

Closed danmoseley closed 4 years ago

danmoseley commented 6 years ago

This tracks porting to .NET Core the rest of the types in Microsoft.VisualBasic, now that we've published the original sources.

Contributions are welcome! The porting process is essentially

  1. Pick some types.
  2. Get the original .NET Framework code from referencesource
  3. Add it to the src project, make it compile. Fix style to match style guidelines, update license headers to match existing code.
  4. Add the API to the ref. Here's an example of a complete ref generated from the .NET Framework binary.
  5. Write tests, add to the tests project, and make them pass.
  6. Put up a PR then we can check the type off.

Here is an example change that ported some more of Strings.

To track progress, here is a list of classes/structs that aren't in .NET Core (or are present, but missing some members). I am not listing the members nor any enums: that detail is here: https://gist.github.com/danmosemsft/d34ec1055c4c23aefcd047089538d3a1

namespace Microsoft.VisualBasic

namespace Microsoft.VisualBasic.ApplicationServices

namespace Microsoft.VisualBasic.CompilerServices

namespace Microsoft.VisualBasic.Devices

namespace Microsoft.VisualBasic.FileIO

namespace Microsoft.VisualBasic.Logging

namespace Microsoft.VisualBasic.MyServices

namespace Microsoft.VisualBasic.MyServices.Internal

A few types may not be portable yet because of dependency issues - either they depend on something that isn't quite yet in .NET Core (principally Windows Forms) or that is, but is in a higher dependency level (Windows Forms again). I have marked some of these with (*). This problem can be solved but I suggest that we skip these for the moment.

ccing folks who have expressed interest in the past: this is now unblocked. @KathleenDollard @OmarTawfik @karelz @jaredpar @amshekar @bbowyersmyth @iainnicol @EdwinEngelen @kaiukdevelopment @ocdtrekkie @ericmutter @MohammadHamdyGhanem

danmoseley commented 6 years ago

@craigajohnson @RealDotNetDave @VBDotNetCoder @Nukepayload2 @anthonydgreen

danmoseley commented 6 years ago

@franzalex @beppe9000 @govert

govert commented 6 years ago

@danmosemsft Thanks - that's going to be helpful.

I clicked on the Collection source... What might Telesto be? https://github.com/Microsoft/referencesource/search?q=TELESTO&unscoped_q=TELESTO

I'm presuming it's some limited platform, and we'd be in the NOT TELESTO world for .NET Core?

danmoseley commented 6 years ago

I think TELESTO was used to limit the size for Silverlight and we want to keep the extra size,

ghost commented 6 years ago

@danmosemsft I suggest that classes used with My object should be added to corefx to be available to all languages even if you move them to other namespaces (out of visualbasic namespace), as they have more functionality not offered by corefx. Adding clsses especially to one language doesn't make sense and is not efficient. Languages can only differ in syntax. These classes will still be accessed via My in vb.net but used directly as regular classes fromany language. Application framework also should be available in c#. I import it from vb just to make c# app a single instance.

hughbe commented 6 years ago

Can I grab ComClassAttribute, HideModuleNameAttribute, MyGroupCollectionAttribute, VBFixedArrayAttribute, VBFixedStringAttribute, DesignerGeneratedAttribute, OptionCompareAttribute, OptionTextAttribute and StandardModuleAttribute?

danmoseley commented 6 years ago

@MohammadHamdyGhanem could C# not access My like any other type? But otherwise, we have no plans to make My more widely available to C#. It was specifically designed for the coding style of VB developers. Languages do not differ only in syntax. They have different developer groups as do their accompanying libraries.

stephentoub commented 6 years ago

could C# not access My like any other type?

@danmosemsft, no, it's a figment of the VB compiler's imagination; it emits the "My" support into the binary at compile-time.

ghost commented 6 years ago

@danmosemsft I didn't ask to add My to C#. In fact My has some long expressions that I don't like in VB. I'm asking to relocate the classes use by My, from Microsofr.VisulBasic Namespace to the CoreFx namespaces suitable for thier functionality (i.e tracing classes go to the namespaces where other tracing classes are), so they can be used normally from any .NET lang.uage as any othe classes, whithout adding a reference to Microsofr.VisulBasic Namespace first (C# programmers hate this as hell).

weshaggard commented 6 years ago

@danmosemsft as we port these to corefx we need to be very careful it doesn't require us to put everything into the shared framework. Microsoft.VisualBasic.dll was one of the assemblies in .NET Framework that had static dependencies to almost everything.

KathleenDollard commented 6 years ago

@MohammadHamdyGhanem Can you list some of the classes you want to see in a non-VB namespace? That would go through CoreFx, probably with the involvement of CSharpLang.

ghost commented 6 years ago

@KathleenDollard FileSystem Financial ApplicationBase CantStartSingleInstanceException StartupEventArgs StartupNextInstanceEventArgs UnhandledExceptionEventArgs WindowsFormsApplicationBase () TextFieldParser AspLog FileLogTraceListener () Log ClipboardProxy (*) FileSystemProxy RegistryProxy SpecialDirectoriesProxy

KathleenDollard commented 6 years ago

That is pretty broad.

I wanted to determine if there were a few things that could be justified to the CoreFx with user stories, etc. This indicates you want the VB experience.

Since the goal is support for moving existing apps, these features need to stay in the VisualBasic namespace to avoid breaking people.

bbowyersmyth commented 6 years ago

I'll grab Microsoft.VisualBasic.Collection

reduckted commented 6 years ago

I know there's the backward-compatibility argument, but has there been any discussion around deprecating/not porting some of these classes? Take, for example, Collection. Why would anyone use that over say, List(Of T), or Dictionary(Of TKey, TValue)? It's not even a generic collection.

And then there's the ErrObject 😨 Does anyone still use that over Try/Catch blocks?

Information.IsArray(), Information.IsError() and Information.IsNothing(). What's the point of them?

And some of these aren't VB-specific as @MohammadHamdyGhanem points out. FileSystem.DeleteDirectory is a great example. It lets you move directories to the recycle bin. Surely that's something that a C# programmer would want to do.

paul1956 commented 6 years ago

@reduckted When I ported a VB App to Portable, lack of the complete FileSystem class caused issues, but the portable class and I assume .NetCore have the similar if not identical functionality but different API's. People using .NetCore don't need another way to do the same thing but VB needs a (thin) layer to keep doing what we were doing without changing lots of code. Hopefully it gets written once and everyone gets to use it. I have never looked at VB Collections but I assume the implementation would be a thin layer over the functions you list. Years ago I ported a library of VB6 string functions to use with VB.net over time I have stopped using them because it just confused people but it allowed a quicker port.

paul1956 commented 6 years ago

@danmosemsft I tried to clone the src project using Open in Visual Studio and I get the dialog below. I have the extension and have no issue Cloning other projects like DotNet/Roslyn. image

danmoseley commented 6 years ago

@paul1956 hmm in not sure what to suggest. If you pick some other random repo now, does it work?

danmoseley commented 6 years ago

@reduckted

has there been any discussion around deprecating/not porting some of these classes

There are many customers who have existing apps that use one or more existing libraries that happen to have been written in VB. They cannot migrate to..NET Core which is why we are porting this verbatim. You're definitely right I wouldn't recommend using some of these types in new code. Those types maybe could have some text in the docs.

paul1956 commented 6 years ago

@danmosemsft I have tried 2 others and they both work. I was able to clone starting with Visual Studio instead of the website link.

paul1956 commented 6 years ago

@danmosemsft I would like to try FileIO (4 files) since I really need it.

bbowyersmyth commented 6 years ago

I'll grab: Microsoft.VisualBasic.CompilerServices.DecimalType Microsoft.VisualBasic.CompilerServices.DoubleType Microsoft.VisualBasic.CompilerServices.Versioned (Partial) Microsoft.VisualBasic.Information (Partial)

paul1956 commented 6 years ago

@danmosemsft I noticed the ref file and all the tests are C#, if you want VB programmers to contribute I would hope at least the test are VB and there is some explanation of what to do with the ref file.

paul1956 commented 6 years ago

Before I started I tried to just compile the stuff that was already done and I get variations of the message below, I also get the same message in 2 of the 4 modules I was working on. It this a prerequisite to porting the rest or am I missing something?

Severity    Code    Description Project File    Line    Suppression State
Error   BC35000 Requested operation is not available because the runtime library function 'System.ParamArrayAttribute..ctor' is not defined.    Microsoft.VisualBasic   C:\Users\PaulM\Source\Repos\corefx\src\Common\src\System\SR.vb  65  Active
reduckted commented 6 years ago

There are many customers who have existing apps that use one or more existing libraries that happen to have been written in VB. They cannot migrate to..NET Core which is why we are porting this verbatim.

Fair enough.

You're definitely right I wouldn't recommend using some of these types in new code. Those types maybe could have some text in the docs.

Yeah, but who reads docs anyway? 🤣 Maybe put the ObsoleteAttribute on them?

danmoseley commented 6 years ago

@paul1956 that is a very fair criticism! The ref file is C# because it is in some cases regenerated by a tool, which emits C#. Hopefully in most cases its possible to just copy and modify existing entries. The tests are C# probably because all the others are (not a great reason). I know when I ported some of the VB last week I discovered my VB had become rather rusty so I'm right there with you..

danmoseley commented 6 years ago

@reduckted we rarely use [Obsolete] in the Framework itself anymore because customers that have warnings as errors enabled can no longer compile when they update to target a newer framework. Various solutions have been suggested for this apparently (probably the history is in the Roslyn repo) but nothing has stuck.

What we recommend instead is https://docs.microsoft.com/en-us/dotnet/standard/analyzers/api-analyzer

danmoseley commented 6 years ago

BC35000 Requested operation is not available because the runtime library function 'System.ParamArrayAttribute..ctor' is not defined. Microsoft.VisualBasic C:\Users\PaulM\Source\Repos\corefx\src\Common\src\System\SR.vb 65 Active

I do not know why you get this. @omartawfik could you please help?

jaredpar commented 6 years ago

That message means the compiler simply could not find that type hence it's likely an issue in what is being passed to the compiler (bad mscorlib reference). Can you share out an msbuild binary log?

danmoseley commented 6 years ago

@paul1956 another thing you can try is clean -all at the root of CoreFX, then build from the root again. Maybe the packages are in a bad state, and they are cleaned out with that command.

paul1956 commented 6 years ago

@danmosemsft I assume for the ref file, I need to translate the Function, Sub and Property declare lines to C# and then add {thrown null;} if not please let me know what I need to do. Also it looks like I can declare most inputs object because otherwise I get errors for things other than simple types.

The https://docs.microsoft.com/en-us/dotnet/standard/analyzers/api-analyzer only works with C# not VB according to the documentation.

Clean -all deletes all the work I did (it was saved)

@jaredpar Except for the errors I expect the command line build works correctly. The issue is when I build with Visual Studio. Do you want me to just paste the msbuild.log file here or something else?

paul1956 commented 6 years ago

I also noticed all the yellow ! in the dependency list and that both kinds of slashes are used and restore NuGet does nothing. image

danmoseley commented 6 years ago

@paul1956

I assume for the ref file, I need to translate the Function, Sub and Property declare lines to C# and then add {thrown null;} if not please let me know what I need to do.

That's right. Alternatively, there is a shortcut. Once you have the implementation building (in the src folder) you can execute this command in the src folder: msbuild /t:generatereferencesource. This will edit ref\Microsoft.VisualBasic.cs to match the implementation. This will make a bigger diff than you need, for various reasons (the implementation is not in sync with it already). However you may be able to copy out just the lines corresponding to what you did in the implementation, then revert the file, and paste in those lines you want.

Also it looks like I can declare most inputs object because otherwise I get errors for things other than simple types.

The ref file has no Import statements. All types in there that aren't already known to the compiler must be fully qualified eg System.ObsoleteAttribute.

danmoseley commented 6 years ago

I repro the problem building the .vbproj inside VS. I will break that out into a separate issue. Until that is fixed, it is necessary to build on the command line and VS squiggling will be not useful.

danmoseley commented 6 years ago

@paul1956 I think I found the problem that prevented the VB project building inside VS. See https://github.com/dotnet/corefx/pull/31303. By copying the change in that pull request to your machine, it should hopefully be fixed for you as well.

paul1956 commented 6 years ago

@danmosemsft I created a sub directory under src called FileIO to match the ref source, do I still execute msbuild /t:generatereferencesource in src or in src\FileIO. Also why are there 6 files directly under source instead of in an appropriate sub directory following ref source, do we really want all the files under src?

danmoseley commented 6 years ago

Also why are there 6 files directly under source instead of in an appropriate sub directory following ref source, do we really want all the files under src?

@paul1956 the .vb files under src should be in a folder matching their namespace, eg., src\Microsoft\VisualBasic\CompilerServices\IncompleteInitialization.vb. That should already be the case for all of them? There are a few files directly in src such as the project file and other non source files.

I created a sub directory under src called FileIO to match the ref source, do I still execute msbuild /t:generatereferencesource in src or in src\FileIO.

The trick is that you get src building first. You can do this in VS (after pasting in my fix above at least). Once you have done that, in a command prompt, go to the src folder itself and run msbuild /t:generatereferencesource. it should take only a second and leave you with an edited ref file. As you'll see there's a bunch of noise that you won't want to keep, but you should see lines corresponding to your src changes.

danmoseley commented 6 years ago

Oh I see, you're noting that referencesource doesn't follow this folder pattern. Right. In CoreFX, you would have a file laid out like eg src\Microsoft\VisualBasic\FileIO\FileSystem.vb.

danmoseley commented 6 years ago

The https://docs.microsoft.com/en-us/dotnet/standard/analyzers/api-analyzer only works with C# not VB according to the documentation.

@pjanotti is this correct?

paul1956 commented 6 years ago

@danmosemsft Question, for example I need an Enum in HostProtectionResource in System.Security.Permissions in mscorelib.dll that is not part yet part of corefx. Do I

  1. Make it a local in FileIO
  2. Create a new Folder System.Security.Permissions next to FileIO and port the file to VB (trivial for me since I have a faithful converter).
  3. Add existing C# file to corefx\src\Common\src\CoreLib\System\Security\
  4. Something else
paul1956 commented 6 years ago

Which Nuget Packages can I install? When I try to install System.Runtime.Serialization.Primitives I get the message below. I had assumed that 3.0x is a superset of at least one of the standards below. It also looks like the functions I want are in this repo but how do I reference them.

Restoring packages for C:\Users\PaulM\Source\Repos\corefx\src\Microsoft.VisualBasic\src\Microsoft.VisualBasic.vbproj...
NU1202: Package System.Runtime.Serialization.Primitives 4.3.0 is not compatible with netstandard (.NETStandard,Version=v0.0). Package System.Runtime.Serialization.Primitives 4.3.0 supports:
  - monoandroid10 (MonoAndroid,Version=v1.0)
  - monotouch10 (MonoTouch,Version=v1.0)
  - net45 (.NETFramework,Version=v4.5)
  - net46 (.NETFramework,Version=v4.6)
  - netcore50 (.NETCore,Version=v5.0)
  - netstandard1.0 (.NETStandard,Version=v1.0)
  - netstandard1.3 (.NETStandard,Version=v1.3)
  - portable-net45+win8+wp8+wpa81 (.NETPortable,Version=v0.0,Profile=Profile259)
  - win8 (Windows,Version=v8.0)
  - wp8 (WindowsPhone,Version=v8.0)
  - wpa81 (WindowsPhoneApp,Version=v8.1)
  - xamarinios10 (Xamarin.iOS,Version=v1.0)
  - xamarinmac20 (Xamarin.Mac,Version=v2.0)
  - xamarintvos10 (Xamarin.TVOS,Version=v1.0)
  - xamarinwatchos10 (Xamarin.WatchOS,Version=v1.0)
pjanotti commented 6 years ago

The https://docs.microsoft.com/en-us/dotnet/standard/analyzers/api-analyzer only works with C# not VB according to the documentation.

@pjanotti is this correct?

Yes, that is correct.

danmoseley commented 6 years ago

@eerhardt any idea about this (https://github.com/dotnet/corefx/issues/31181#issuecomment-407348802) .. I haven't seen it.

eerhardt commented 6 years ago

@paul1956 - dotnet/corefx is a "self-contained" repo, meaning that it doesn't reference NuGet packages like you would in a normal project. Instead, all of the reference assemblies are built in corefx, and then all the other code uses those "ref" assemblies to build.

So if you want to add a reference from Microsoft.VisualBasic to System.Runtime.Serialization.Primitives you would do something like the following:

https://github.com/dotnet/corefx/blob/0b0f63d47951773cc46393312d80b40176d3a413/src/System.Threading.Tasks.Dataflow/src/System.Threading.Tasks.Dataflow.csproj#L79

paul1956 commented 6 years ago

@eerhardt that worked, I hope someone is collecting all this issues in one place so every "external" developer does not have to go through this. A related question the code in some of the files I need to port have lots of conditional compiler flags, how do I know how to set them? I have no idea what these flags mean with respect to CoreFx For Example

#If Not FEATURE_CORECLR Then
    <Serializable>
    <SecurityPermissionAttribute(SecurityAction.InheritanceDemand, ControlEvidence:=True, ControlPolicy:=True)>
    <System.Runtime.InteropServices.ComVisible(True)>
    Public MustInherit Class CodeAccessPermission
        Implements IPermission, ISecurityEncodable, IStackWalk
#Else
    <Serializable>
    <System.Runtime.InteropServices.ComVisible(True)>
    Public MustInherit Class CodeAccessPermission
        Implements IPermission, ISecurityEncodable, IStackWalk
#End If

or 

#If FEATURE_CAS_POLICY Then

        static internal SecurityElement CreatePermissionElement( IPermission perm, String permname )        {            SecurityElement root = new SecurityElement( "IPermission" );            XMLUtil.AddClassAttribute( root, perm.GetType(), permname );            // If you hit this assert then most likely you are trying to change the name of this class.             // This is ok as long as you change the hard coded string above and change the assert below.            Contract.Assert( perm.GetType().FullName.Equals( permname ), "Incorrect class name passed in! Was: " + permname + " Should be " + perm.GetType().FullName);            root.AddAttribute( "version", "1" );            return root;        }                static internal void ValidateElement( SecurityElement elem, IPermission perm )        {            if (elem == null)                throw new ArgumentNullException( "elem" );            Contract.EndContractBlock();                            if (!XMLUtil.IsPermissionElement( perm, elem ))                throw new ArgumentException( Environment.GetResourceString( "Argument_NotAPermissionElement"));                            String version = elem.Attribute( "version" );                        if (version != null && !version.Equals( "1" ))                throw new ArgumentException( Environment.GetResourceString( "Argument_InvalidXMLBadVersion") );        }        abstract public SecurityElement ToXml();        abstract public void FromXml( SecurityElement elem );        //        // Unimplemented interface methods         // (as a reminder only)        //        public override String ToString()        {            return ToXml().ToString();        }
#End If
danmoseley commented 6 years ago

I'll look at the #if's and suggest approach for each..

paul1956 commented 6 years ago

@danmosemsft thanks, Still learning Git, what do I need to get to the proper fix for dotnet/corefx#31303 in my branch hopefully without losing all my changes.

danmoseley commented 6 years ago

@paul1956 as that fix is now committed into master, you need to rebase your working branch on master. Before doing that, you should either stash (git stash) or commit (git commit -am for example) your uncommitted changes. (If yo'ure learning git, maybe copy them somewhere else in case too...). Then, with your working branch checked out, you would do:

git fetch 
git rebase upstream/master

After doing this, if you had committed your temporary change, you would return to your editing state with git reset head~.

paul1956 commented 6 years ago

@danmosemsft something went wrong, should upstream be origin?

PS C:\Users\PaulM\Source\Repos\corefx> git fetch
PS C:\Users\PaulM\Source\Repos\corefx> git rebase upstream/master
fatal: Needed a single revision
invalid upstream 'upstream/master'
PS C:\Users\PaulM\Source\Repos\corefx> git remote -v
origin  https://github.com/paul1956/corefx.git (fetch)
origin  https://github.com/paul1956/corefx.git (push)
PS C:\Users\PaulM\Source\Repos\corefx>
danmoseley commented 6 years ago

Ah, yes, normally I would do something like this when I clone a repo so I have the fork and parent distinct:

git remote rename origin upstream
git remote add origin https://github.com/paul1956/corefx
git fetch --all