search

3F / DllExport

.NET DllExport with .NET Core support (aka 3F/DllExport aka DllExport.bat)
MIT License
965 stars 133 forks source link

Stop support of any standard NuGet clients #38

Closed 3F closed 7 years ago

3F commented 7 years ago

Since we have a lot of problems/inconvenience with new nuget features like a global caching, new PackageReference format, removed install/uninstall ps-scripts, and the overall focus on libraries only, I'll consider new destribution via our compact GetNuTool client (or something else, not important). That is, this package still will be placed via any nuget server, but without any guarantee for support of any standard client.

Because I'm tired from nuget project -_- in recent years it's very inconvenient for any tools, because of main focus to libraries (that's soundly not for all cases).

GetNuTool (batch file about 10kb) has been created primarily to provide tools and service of user projects, libraries, the build processes, etc. It especially used by vsSolutionBuildEvent projects (CIM library for support CI features) because of removed solution-level in 2015

Thus, probably we need to reconsider new way for distribution our tool for best support.

Related issue: https://github.com/3F/DllExport/issues/36#issuecomment-305979911

How it can be (draft)

gnt.core can be wrapped to 1 batch file (text-based file, who don't know) that will:

Requirements: no any, GetNuTool works via msbuild like your common build. And logic will be placed in 1 batch file, that you can store with your code as simple build-script.

Problems

But GetNuTool is oriented for solution-level only, therefore we need also provide support to add our settings & required targets for user project-files.

For this I can encapsulate logic of this inside our package from vsSolutionBuildEvent project (I think part of this I can open by new MIT license), and main scheme should be like:

batch -> package -> {installation} -> {logic from vsSolutionBuildEvent} -> {imported targets and configuration} -> done

so I think it should be fine.

Pros / cons

+

-

So what anyone think about this ? Your suggestions ?

3F commented 7 years ago

536bfbc contains the final draft-version of the new configurator that will work without amazing NuGet clients at all.

The final wrapper/installer will be soon...

dllexport_cfg_mvssln

3F commented 7 years ago

Anyone please also test it before final release ! https://ci.appveyor.com/project/3Fs/dllexport/build/build-53/artifacts

How to check it:

All current available commands:

>DllExport

DllExport - v1.5.2.34083 [ 6641d7e ]
Copyright (c) 2009-2015  Robert Giesecke
Copyright (c) 2016-2017  Denis Kuzmin [ entry.reg@gmail.com :: github.com/3F ]

Distributed under the MIT license
https://github.com/3F/DllExport
Wizard - based on hMSBuild logic and includes GetNuTool core - https://github.com/3F

Usage: DllExport [args to DllExport] [args to GetNuTool core]
------

Arguments:
----------
 -restore              - To restore configured DllExport.

 -sln-dir {path}       - Path to directory with .sln files to be processed.
 -sln-file {path}      - Optional predefined .sln file to process via the restore operations etc.
 -metalib {path}       - Relative path from PkgPath to DllExport meta library.
 -dxp-target {path}    - Relative path to .target file of the DllExport.
 -dxp-version {num}    - Specific version of DllExport. Where {num}:
                          * Versions: 1.6.0 ...
                          * Keywords:
                            `actual` to use unspecified local version or to get latest available;

 -msb {path}           - Full path to specific msbuild.
 -packages {path}      - A common directory for packages.
 -server {url}         - Url for searching remote packages.
 -wz-target {path}     - Relative path to .target file of the Wizard.
 -eng                  - Try to use english language for all build messages.
 -GetNuTool {args}     - Access to GetNuTool core. https://github.com/3F/GetNuTool
 -debug                - To show additional information from hMSBuild.
 -version              - Display version of hMSBuild.
 -build-info           - Display build information of DllExport.
 -help                 - Display this help. Aliases: -help -h -?

--------
Samples:
--------
 DllExport -restore
 DllExport -restore -sln-file "Conari.sln"

 DllExport -build-info
 DllExport -restore -sln-dir -sln-dir ..\ -debug

 DllExport -GetNuTool -unpack
 DllExport -GetNuTool /p:ngpackages="Conari;regXwild"

---------------------
Possible Error Codes: ERROR_FILE_NOT_FOUND (0x2), ERROR_PATH_NOT_FOUND (0x3), ERROR_SUCCESS (0x0)
---------------------
3F commented 7 years ago

Check latest from dev_1.6 branch

https://ci.appveyor.com/project/3Fs/dllexport/build/build-54/artifacts

How to check:

Misunderstanding for new users via current NuGet server - why install command not works properly.

I added information via \build feature:

    We'll support only information about new installer because:
    * The `\build` feature still does not guarantee of work for all cases. https://github.com/3F/DllExport/issues/36#issuecomment-305977523
    * We don't have any guaranteed information about uninstall operation for our package because of removed uninstall.ps1
      https://github.com/3F/DllExport/issues/38

It also prepares DllExport.bat with -action Configure key, thus you can still easily configure all your projects inside solution as you need.

yes, updated key:

 -action {type}        - Specified action for Wizard. Where {type}:
                          * Configure - To configure DllExport for specific projects.
                          * Restore   - To restore configured DllExport.

Side note:

\build\net20\DllExport.targets with:

  <Import Project="..." />

  <!-- ... and optionally -->
  <Target Name="DllExportLib" BeforeTargets="PrepareForBuild">
    <CallTarget Targets="...." />
  </Target>

Therefore, today I added only information about our new installer:

warn_ dllexport bat

As you see, DllExport.bat (about 20 Kb - now, we don't use powershell at all) also will be distributed with our packages to control installation when it possible. For other cases it can be received directly from GitHub and also stored inside any other your scripts for your repo.

I'll write compressor/encoder for this later. Seems should work from hMSBuild.

3F commented 7 years ago

.NET DllExport v1.6 Wizard - Demonstration of the new manager & distribution.

https://www.youtube.com/watch?v=okPThdWDZMM

dllexport_v1.6_wizard_manager_sample

3F commented 7 years ago

TODO:

https://github.com/3F/DllExport/issues/36#issuecomment-320534432

that is, we'll also try to provide DllExport.bat via std. nuget for access to our manager (via new Wizard) to configure data for projects, like you can see in video above.

The old Configurator is obsolete anyway because of caching etc, that's incompatible with ddNS etc.

3F commented 7 years ago

-_- damn, no any problems with local server, but after release, yes, of course...

Problem 1:

[12:31:52,51 ] dxpName = 'DllExport'
[12:31:52,51 ] dxpVersion = '1.6.0-beta'
[12:31:52,53 ] dxpTarget = '"packages\\DllExport.1.6.0-beta\\tools/net.r_eg.DllExport.Wizard.targets"'
[12:31:52,53 ] _remoteUrl = 'DllExport/1.6.0-beta'

  GetNuTool v1.6 - github.com/3F
  =========

Getting `DllExport/1.6.0-beta` ... Extracting into `D:\tmp\2017\DllExportClassLibrary1\packages\DllExport.1.6.0-beta`
gnt.core(1,713): error MSB4018: The "NGDownload" task failed unexpectedly.\r
gnt.core(1,713): error MSB4018: System.IO.FileFormatException: File contains corrupted data.\r
gnt.core(1,713): error MSB4018:    at MS.Internal.IO.Zip.ProgressiveCrcCalculatingStream.Read(Byte[] buffer, Int32 offset, Int32 count)\r
gnt.core(1,713): error MSB4018:    at MS.Internal.IO.Zip.ZipIOModeEnforcingStream.Read(Byte[] buffer, Int32 offset, Int32 count)\r

Problem 2 (via VS IDE):

Executing script file 'C:\Users\reg\.nuget\packages\dllexport\1.6.0-beta\tools\init.ps1'...
 : The term '' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spell
ing of the name, or if a path was included, verify that the path is correct and try again.
At C:\Users\reg\.nuget\packages\dllexport\1.6.0-beta\tools\init.ps1:1 char:1
+ + ~
    + CategoryInfo          : ObjectNotFound: (:String) [], CommandNotFoundException
    + FullyQualifiedErrorId : CommandNotFoundException

Something went wrong with package from nuget.org server ! I checked that released local .nupkg works perfectly via local server, but failed from nuget.org o_o

upd. problem has been isolated but still does not resolved because of bugs from nuget.org

Seems problematic symbols, but it placed via online editor on nuget.org after release ! what the... >(

I already edited this:

My first thoughts: I escaped this manually: <, >, &. ok ! it has been updated. But when I checked it again, then via local server, then I also found problem with ? symbol. New trying to edit and no result for .nupkg (both places: nuget.org & preview.nuget.org). > 40min and it still does not applied.

Even if I'm trying to remove all text, no result for .nupkg package.

An edit is pending for this package version. You are seeing the edited package description now. These changes will be applied to the package file and become visible to everybody soon

How long ?!

No any response after 3 hours of waiting via https://www.nuget.org/policies/Contact

Well, to reupload new package with new number 1.6.0.1-beta is the most easy way, and to update manager in latest branch, and probably update DllExport.bat in GitHub releases, and... new release probably much more better, omg -_-

Everyone: use correct .nupkg from GitHub while we have problem with nuget.org

upd2.

Incredible project in any time !

Unfortunately guys, you should wait fixes from nuget.org (maybe new data will be updated automatically after 10 hours, I don't know...), or wait new packages from us on the next week. Or use other server with valid package from https://github.com/3F/DllExport/releases/tag/v1.6-beta

https://www.nuget.org/packages/DllExport/1.6.0-beta/ReportMyPackage

... Please replace or remove 1.6.0-beta and tell me what's going on with releaseNotes ?! because I'm scared to use your online editor after this at all !

!

upd3:

As I see, this package has been updated now (seems automatically after 24 hours etc.).

3F commented 7 years ago

Ok, what I found:

The second problem ('corrupted data' via manager - see above) is more complex. The nuget.org repacks .nupkg with different compression method for some files:

cmp_package_from_nuget org

cmp_package_from_nuget org_2

the 0x08 means Deflate method, but nuget server does it with 0x00 that means without compression and with compressed length as 2 bytes (see value from 0x3F96 - 0x3F99)

Well, I don't know why nuget.org does this, but I know how to correctly process this via our GetNuTool.

The initial a correct package DllExport.1.6.0-beta.nupkg here: https://github.com/3F/DllExport/releases/download/v1.6-beta/DllExport.1.6.0-beta.nupkg

please be patient, I've plan to release beta2 after improving GetNuTool client. This week, as possible for my time. You can also test released beta version via any local server etc. because this problem for nuget.org server only.

Write here if you have any related questions etc.

3F commented 7 years ago

Ok, now all this changes has been merged into master, and we're ready to use this as the main features for this project.

I also implemented optional configuration via external storage and now it looks like this:

https://github.com/3F/DllExport/issues/49#issuecomment-338407802

dllexport

I also can to implement features for configuring without any modification of project files at all (like PackageReference features from nuget), it's not so hard for implementing, however, the main problem in comparison with PackageReference - nuget is already integrated into VS, but for our case the VS users should manually install any other plugin for VS (I don't no see major problems for build through msbuild, like for CIM we can catch and re-implement any nodes without any modifications or interaction from users.). Well, we can also interact with VS through COM interrupts instead of VSIX, but ... :) just create new issue if someone need this, we'll discuss about this in details later.

Before final release I've plan to release the latest beta-3 soon for public tests.

ArgusMagnus commented 6 years ago

Im sorry to say this, but this is a terrible choice from a user perspective. Suddenly I am prompted to download some .bat files from the internet with no clear and easy instructions to use them. I will have to find out how exactly I need to use every time I use it (since I have to install it rarely into new projects).

If I run DllExport_Configure.bat, the cmd window flashes up, Im guessing it crashes, because nothing happens. I guess at this point, Im faster at writing a C++/CLI layer than investigating this further.

Please consider minimal NuGet support again, no need for the configuration GUI, just deliver a default configuration (e.g. x86 + x64, etc..) via NuGet please.

3F commented 6 years ago

@ArgusMagnus

  1. GUI it's optional way, you can still configure all this as you like. We really need something like that because of diverse our audience, but moreover, it's very convenient.
  2. The new distribution may be configured flexibly with any server. You can still get it from NuGet server by default, or from any other remote or local source.
  3. New manager is embeddable text-based script (no powershell or dotnet-cli). You can store it in any place, or still get it from any remote server if you need.
  4. ! Restoring packages are fully automatic ! You do not need to do anything. It will be restored automatically when build / opened solution via VS IDE.

If I run DllExport_Configure.bat, the cmd window flashes up, Im guessing it crashes, because nothing happens.

it looks like a bug, please attach detailed log, but first, try latest changes: https://github.com/3F/DllExport/issues/56

I will have to find out how exactly I need to use every time I use it (since I have to install it rarely into new projects).

to be sure again, you don't need to do anything because all still should be automatically.

with no clear and easy instructions to use them

I agree, we need new illustration for 'how to'.

But today, please see this video-illustration: https://www.youtube.com/watch?v=okPThdWDZMM&t=46s (it was from first beta but conceptually it's same for actual version)

I'll update some documentation soon.

Please do not hesitate to contact with me if anyone else still have any problem for this.

ArgusMagnus commented 6 years ago

Thanks for the fast reply. It seems I was under a wrong impression based on a bug (I thought I had to run this scripts manually, etc.). I will take some time to post some details later.

ArgusMagnus commented 6 years ago

So I still don't quite understand, if I install the NuGet package, I don't need to run the DllExport_Configure.bat?

I opened an issue for DllExport_Configure.bat crashing: #58

3F commented 6 years ago

@ArgusMagnus

When your project will be configured, yes, you don't need to run the DllExport.bat (DllExport_Configure.bat - it just a wrapper to command DllExport -action Configure, remove it if you need)

if I install the NuGet package ...

Do not use std. nuget installation and also remove any old DllExport-packages. Only DllExport.bat

ArgusMagnus commented 6 years ago

So.. I do need to manually run a bat file. I stand by my statement that this is a big step back from a user perspective. Why is the .bat file which is download by the nuget package not also run by the nuget package?

3F commented 6 years ago

@ArgusMagnus I don't see difference. When you install v1.5.2 through nuget client you still need to configure it manually.

Again, you do not need to call DllExport.bat (neither you nor anyone else) after initial configuration.

When anyone else will get your src with configured DllExport, again, all will be restored automatically.

Seriously, while you're trying to solve problem with #58 why you don't want to check how it works via empty new projects.

3F commented 6 years ago

I added basic information for How to start.

dagostinelli commented 6 years ago

I agree with the sentiment. I uninstalled this 3x before I finally let myself run the batch file.

I realize that this is actually more of a special program to configure the project file. But all I did was choose the 64-bit option and click apply.

It's a big step.

dagostinelli commented 6 years ago

Also, Windows Defender popped up in my case and proudly blocked the batch file from executing at first.

3F commented 6 years ago

I uninstalled this 3x before I finally let myself run the batch file.

Please clarify, I'm not sure what do you mean.

all I did was choose the 64-bit option and click apply.

good, we'll need also to add a silent mode for installation, something like install/uninstall for all possible projects with default settings. thanks for feedback

Windows Defender popped up in my case and proudly blocked the batch file from executing at first.

not sure how to prevent this, probably only through report to MS about false positive detection

21.02.2018 6:46, Darryl T. Agostinelli:

I agree with the sentiment. I uninstalled this 3x before I finally let myself run the batch file.

I realize that this is actually more of a special program to configure the project file. But all I did was choose the 64-bit option and click apply.

It's a big step.

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/3F/DllExport/issues/38#issuecomment-367204735, or mute the thread https://github.com/notifications/unsubscribe-auth/ADGJ_a8STW9YuRpgr8zsN4Qtbbf2yMH2ks5tW5GlgaJpZM4NvJuE.

dagostinelli commented 6 years ago

Eek, I think I may have mis-explained. There is no bug. I apologize.

What I meant was that I installed DllExport, took one look at the batch file and uninstalled it without trying. I forgot about DllExport, rediscovered it, tried it and said no way again. I repeated all of that 3 times before I eventually read this GitHub issue and decided to do it. Windows Defender even reported this library as a potential red-flag.

It's the sentiment that I'm agreeing with. That this should just work from a NuGet package and not from the batch file. It's just very, very irregular to have a batch file.

3F commented 6 years ago

Unfortunately the modern NuGet (a new focus in development at all) is very inconvenient for projects like this. We are not a library, we're tool. And I already discussed about PackageReference support, here's my decision: https://github.com/3F/DllExport/issues/36#issuecomment-323581430

I have no plan for this at all, firstly and mainly - I just don't want. I'm tired of the fact that the support for important things from nuget project periodically are lost or canceled (suddenly but officially). The weird and unclear support of their product for all years. A contrast example: in past I had many urgent tasks to solve problems with canceled the solution-level for X products (the community should know about open vsSBE tool, as an example). Now it also ps-scripts. And I'm already silent about compatibility and functionality between their different clients - console tool, vsix-plugin etc.

And moreover, for this project I'm focused on other things at all: " I've plan to fully avoid the dependence on ILAsm and ILDasm. So probably it will be fully new project. "

But ! anyone else may try to support also this way. I'll review definitely your PR later. Welcome.

tried it and said no way 3 times before I eventually read this GitHub issue and decided to do it.

I'm still don't understand your "3 times"

3F commented 6 years ago

By the way, this script can be embedded inside any other place and freely distributed between repos, or just be downloaded from github before build via any script etc.

The main pros: this can be embedded or distributed via different servers and tools. Fully independent. The one setting, and then all in automatic mode (restoring, configuring, etc.)

Looking ahead, it also ready for different OS beyond of the Windows. Because as I already said in other discussion (I'll add more details for current wiki later), this batch script was based on GetNuTool core, that's Cross-Platform Embeddable Package Manager that requires only MSBuild. So it very easy to continue support for any other platform.

dagostinelli commented 6 years ago

I edited my "3 times" comment above for clarity. It's irrelevant anyway...