Package the exe as a self contained app so it can run cross platform with being re-built

[rfennell]

As you might remember I have an Azure DevOps extension that wrappers this tool.

This task runs fine on Windows based build agents, but I cannot get it to work on Linux and MacOS based ones using the EXE you provide in the release.

Given the error message, I think this is because the EXE is not built as a self-contained application.

Is this something

1. You have tested
2. Would consider packaging the EXE as in the release so the task can pull down the current version in a form that can be run on all platforms?

[MaxMelcher]

Hey Richard - sure!

I want to build & publish the converter with GitHub actions soon - doing a second build for a linux runtime should be easy. Not sure when I can do it, but yes thats something I would love to see.

1. tested - no; the path and downloading could be different on linux.
2. consider - yes

PR welcome :)

[rfennell]

I have done some initial research.

As you already have a single file windows distribution, I have focused on creating a Linux one.

To do this I used the command ( not 100% sure over the flag, see this post on errors seen if p:PublishReadyToRun=true

dotnet publish -r linux-x64 -p:PublishSingleFile=true -p:PublishReadyToRun=false

This resulted in a 122Mb file. I copied this to the Ubuntu VM and tried to run it, but it failed with a

bash: ./azuredevops-export-wiki; Permission denied

This was fixed with the command

chmod u+x azuredevops-export-wiki

The executable now ran, it showed the QT warning so it did start. But them gave an error

I tried copying over the libwkhtmltox.dll to the same folder. it made no difference.

I think the issue is related to the libwkhtmltox assembly being added to the project as a prebuilt assembly. Either it is not being packaged up correctly, or is built in the wrong format.

I will continue to work on this as time allows

[rfennell]

A bit more of an update, tried exactly the same publish command with but with a simple 'Hello World' console application. I was able to build a single file for Windows and another for Linux. These both ran without error on their appropriate platform.

So this does point to the issue being related libwkhtmltox assembly as I mentioned above.

Given this, I am not sure I can progress this much more while this assembly is loaded in this manner.

[MaxMelcher]

We could load it dynamically - download if not present. This would bring down the exe size a lot.

Get Outlook for iOShttps://aka.ms/o0ukef

---

From: Richard Fennell @.> Sent: Thursday, March 18, 2021 6:09:09 PM To: MaxMelcher/AzureDevOps.WikiPDFExport @.> Cc: Max Melcher @.>; Comment @.> Subject: Re: [MaxMelcher/AzureDevOps.WikiPDFExport] Package the exe as a self contained app so it can run cross platform with being re-built (#55)

A bit more of an update, tried exactly the same publish command with but with a simple 'Hello World' console application. I was able to build a single file for Windows and another for Linux. These both ran without error on their appropriate platform.

So this does point to the issue being related libwkhtmltox assembly as I mentioned above.

Given this, I am not sure I can progress this much more while this assembly is loaded in this manner.

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/MaxMelcher/AzureDevOps.WikiPDFExport/issues/55#issuecomment-802127734, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AAH3LN3UOIJ4GBVHRKDKDFTTEIXTLANCNFSM4ZMRZVGA.

[rfennell]

I think the issue could be that the wkhtmlopdf library is platform specific. So you can't load the Windows version on Linux, and I assume MacOS. No sure how to resolve this as I have no experience of writing dotnet core apps for Linux

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[MaxMelcher]

@rfennell the latest version runs on ubuntu - how to best provide this to your Azure DevOps task? 89f5399583a8f22de5ee55ddb50ba38fa4270800

[rfennell]

It may 'just' work, I will test it and report back

[MaxMelcher]

I briefly scrolled through your code to see how it works - it pulls the exe file from the release. That's fine for windows, I guess.

Should I just add a Linux binary without .exe extension to the release to make it easier for you?

[rfennell]

After a bit of testing, the issue I see is that on non-windows (I was using Ubuntu) you can't package or run the tool as an EXE.If you try to run the exe you get

So I need to run the command

dotnet AzureDevOpsPDFExport.dll.

If I run just the above command line, the tool runs and exits as expected. However, to be useful I need to pass in command line parameters and I can't get this to work. I have tried to use the -- (as detailed here) but it has no effect, I get the out of range exception.

Packaging

But that can all be worked out. The major point is that within the release we also need the DLL, as well as the Windows EXE. I will then alter my task to run the EXE for windows or the dotnet AzureDevOpsPDFExport.dll for other platforms.

The only alternative is for me to build the DLL on the fly, but this requires the DOTNET SDK is on the build agent, which though likely is not guaranteed.

Timescale

A final question, what is the timescale to release this new version. At present my task does not support pre-view releases, I have modified my private version so it can pull down the new version as a test. If the cross-platform version is going to be a while, I will add the option to use pre-release versions via a new task parameter. If it is about to come out, I will no bother, I will just wait

[rfennell]

Sorry, only just re-read your last comment, yes, adding the DLL as well as the EXE to the release package, would really help.

Also on the parameter front, the problem appears to be -p folder option. If I miss this out it work

If you add the -p to a valid folder it does not error, but is ignored

I tried building the DLL locally and I can see the issue is the way the parameters are passing on .NET. There are pre-pended spaces in the parameters that are causing problems with relative paths.

The fix for me was to just always pass fully qualified paths and the problems went away.

The alternatives to add something like the following assuming you enter paths on the form -p ./somepath

var path = _options.Path.Trim();
var single = _options.Sigle.Trm()

Anyway, all this means we end up with exactly what you suggested, if you include the DLL, as well as the EXE, in the package. I just need to make some command line generation changes to get it to work on Mac & Linux. So back to question, when do you intend to publish this new version, do I need to alter the task to use pre-release versions?

[rfennell]

@MaxMelcher I now have a private version of my task that builds the correct command line for Linux/Mac and also provides a flag to allow the us of pre-releases of the tool. Should work as soon as the DLL is present in the release package

[MaxMelcher]

Ok - I think it's safe to say that dotnet is installed on all the build servers and that should work. I'll ping you once I have a release candidate (and github actions to release future versions with same naming and all that).

Thank you!

[rfennell]

Any ETA on releasing the cross-platform version?

I have a working Azure DevOps cross-platform task that used the beta, I have just been holding off release for the actual release.

If it is going to be a while I think will publish my task with an option to use beta versions of this tool

[MaxMelcher]

Thanks for the reminder, trying to release it very soon. next week very likely.

Get Outlook for iOShttps://aka.ms/o0ukef

---

From: Richard Fennell @.> Sent: Tuesday, October 26, 2021 5:39:33 PM To: MaxMelcher/AzureDevOps.WikiPDFExport @.> Cc: Max Melcher @.>; Mention @.> Subject: Re: [MaxMelcher/AzureDevOps.WikiPDFExport] Package the exe as a self contained app so it can run cross platform with being re-built (#55)

Any ETA on releasing the cross-platform version?

I have a working Azure DevOps cross-platform task that used the beta, I have just been holding off release for the actual release.

If it is going to be a while I think will publish my task with an option to use beta versions of this tool

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/MaxMelcher/AzureDevOps.WikiPDFExport/issues/55#issuecomment-952065122, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AAH3LN7WS7HXOXB2Z5YGTS3UI3KUJANCNFSM4ZMRZVGA.

[MaxMelcher]

.NET 6 release is in 9 days - I think it makes sense to release afterwards so we do not have two breaking changes. @rfennell do you support .NET 6?

[rfennell]

Makes perfect sense to wait.

My task is written in Node.js so as long as the agent has a suitable ru time installed all should be good

[MaxMelcher]

Thanks for confirming! I will release a beta next week to see if NET6 breaks anything.

Get Outlook for iOShttps://aka.ms/o0ukef

---

From: Richard Fennell @.> Sent: Sunday, October 31, 2021 9:02:03 AM To: MaxMelcher/AzureDevOps.WikiPDFExport @.> Cc: Max Melcher @.>; Mention @.> Subject: Re: [MaxMelcher/AzureDevOps.WikiPDFExport] Package the exe as a self contained app so it can run cross platform with being re-built (#55)

Makes perfect sense to wait.

My task is written in Node.js so as long as the agent has a suitable ru time installed all should be good

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/MaxMelcher/AzureDevOps.WikiPDFExport/issues/55#issuecomment-955654719, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AAH3LN3A5AVMJCTPQ3SSQOLUJTZXXANCNFSM4ZMRZVGA.

[MaxMelcher]

Just release the v4 version @rfennell - can you please test?

[rfennell]

I am on holiday this week , but will check it as soon as I get back at the end of the week

Just to check the new release contains the exe, is this now OK for linux/mac, you used to need a dell for these?

[MaxMelcher]

the executable for linux does not have an extension: https://github.com/MaxMelcher/AzureDevOps.WikiPDFExport/releases/download/4.0.0/azuredevops-export-wiki

enjoy the holidays!

[rfennell]

Just released V2 of my extension. It is tested on Windows and Linux. I can't see any release it should not work on Mac too if we had a suitably packaged executable.

The only changes I had to make were

• Alter the task to generate the correct command line to run for Windows and Linux
• Run CHMOD on Linux to set the execution permission

I need to make no changes for your updated command line parameter (header etc) as I just pass this information through as string parameter.

The only gotcha is the need for .NET6 on the agent. In many cases this runtime will be present on a build agent e.g if VS2022 is installed. If it is not present it can be installed as a step in a build pipeline using the usedotnet task before this task is called.

Note: this technique will work for both V1 and V2 of this task

steps:
- task: UseDotNet@2
  displayName: 'Use .NET Core runtime'
  inputs:
    packageType: 'runtime'
    version: '6.0.x'
    includePreviewVersions: false
- task: richardfennellBM.BM-VSTS-WikiPDFExport-Tasks.WikiPDFExportTask.WikiPdfExportTask@2
  displayName: 'Export Single File'
  inputs:
    cloneRepo: false
    usePreRelease: false
    localpath: '$(System.DefaultWorkingDirectory)'
    singleFile: 'infile.md'
    outputFile: '$(Build.ArtifactStagingDirectory)/singleFile.pdf'
    ...

Thanks @MaxMelcher for all your work on getting this tool ports to .NET6

[MaxMelcher]

I can't see any release it should not work on Mac too if we had a suitably packaged executable.

I cant test the Mac version - can you? How should the extension be? I release with GitHub Actions now, so adding this should be easy :)

[rfennell]

I have a couple if options.

There are a couple of Macs the office used for testing. Just need to pick one up next time I go in

The other option is I just but test on a hosted Azure DevOps build agent.

I will get back to you

[rfennell]

@MaxMelcher, good news, as suspected it works fine on a Mac. I tested on an old Mac Mini running macOS High Sierra 10.13.6.

I tested both using dotnet run (after cloning the repo) and packaged up after running dotnet build -- runtime osx-x64 --self-contained; both worked as expected.

Not sure how you want to add the Mac packaged version to release?

Once, I know I can alter my task to download the correct version. At present, if not on Windows, it tries to use the Linux build, which won't work on the Mac (I tried you get a cannot execute binary file error which is not unreasonable)

[MaxMelcher]

ok thats easy. I just publish the third binary and add -osx to the filename. https://github.com/MaxMelcher/AzureDevOps.WikiPDFExport/releases/tag/4.0.3

Ok?

[rfennell]

Perfect

[rfennell]

Just released 2.1.2 of my extension that now fully support Macs as well as Windows and Linux

[MaxMelcher]

Very cool - thanks! :)