Correct way of calling PHP from PowerShell Core

fMichaleczek commented 5 years ago

I'm studying a bridge betweek PowerShell Core and PeachPie.

My aim is to be able to write a cmdlet in php and be able to import it from PowerShell :

<?php

namespace PSPeachPie;

use System\Management\Automation\{Cmdlet, CmdletAttribute, Parameter};

[Cmdlet("Send", "Greeting")]
class SendGreetingCommand extends Cmdlet
{
    // Declare the parameters for the cmdlet.
    [Parameter(Mandatory=true)]
    public $Name;

    // Declare the parameters for the cmdlet.
    [Parameter(Mandatory=false)]
    public $FirstName = 'Flavien';

    public function SendGreetingCommand() {

    }

    // Override the ProcessRecord method to process
    // the supplied user name and write out a
    // greeting to the user by calling the WriteObject
    // method.
    public function ProcessRecord() : void
    {
        $this->WriteObject("Hello " + $this.FirstName + " " + $this.Name + "!");
    }
}

After compiling the previous code with Peachpie, I make a module manifest (psd1) for PowerShell to load the dll. I have no error, it seems PowerShell engine doesn't discover the SendGreetingCommand class.

PS D:\PSPeachie> import-module .\PSPeachPie.psd1
PS D:\PSPeachie> get-command -module PSPeachPie # nothing

So, I use a little reflection to declare the class manually inside a PowerShell script module (psm1) :

Add-Type -Path "$PSScriptRoot\PSPeachPie.dll"
$cmdletEntry = [System.Management.Automation.Runspaces.SessionStateCmdletEntry]::new(
    <# name:             #> 'Send-Greeting',
    <# implementingType: #> [PSPeachPie.SendGreetingCommand],
    <# helpFileName:     #> $null
)

$internal = $ExecutionContext.SessionState.GetType().
    GetProperty('Internal', [System.Reflection.BindingFlags]'Instance, NonPublic').
    GetValue($ExecutionContext.SessionState)

$internal.GetType().InvokeMember(
    <# name:       #> 'AddSessionStateEntry',
    <# invokeAttr: #> [System.Reflection.BindingFlags]'InvokeMethod, Instance, NonPublic',
    <# binder:     #> $null,
    <# target:     #> $internal,
    <# args:       #> @(
        <# entry: #> $cmdletEntry
        <# local: #> $true
    )
)
Export-ModuleMember -Cmdlet *

So, I have an error that confirm the constructor less is a requirement to the PowerShell side.

PS D:\PSPeachie> import-module .\PSPeachPie.psd1
PS D:\PSPeachie> get-command -module PSPeachPie

CommandType     Name                                               Version    Source
-----------     ----                                               -------    ------
Cmdlet          Send-Greeting                                      0.0.1      PSPeachPie

PS D:\PSPeachie> send-greeting
send-greeting : Type 'PSPeachPie.SendGreetingCommand' does not have a default constructor
Parameter name: type
At line:1 char:1
+ send-greeting
+ ~~~~~~~~~~~~~
+ CategoryInfo          : NotSpecified: (:) [], ArgumentException
+ FullyQualifiedErrorId : System.ArgumentException

I open this issue because as a php/C#/powershell developper, i think at a very reasonable cost, the bridge is very interesting. ( peachpie is based on a patched Roslyn, powershell on a patched DLR).

PowerShell has a cmdletadapter class (ObjectModelWrapper integrates OM-specific operations into generic cmdletization framework) but it seems it need also a constructor less class.

Have you got some ideas ?

Resources: PowerShell#9535:Writing Cmdlets in other languages PeachPie#443:Correct way of calling PHP from C# PowerShellDocs:How to write a cmdlet PeachPie#259:Constructor with no parameters MSDocs:CmdletAdapter

jakubmisek commented 5 years ago

Very nice idea! You're right, currently the PeachPie compiler generates constructors with one implicit parameter Context ctx (https://docs.peachpie.io/api/assembly/compiled-class/#constructors). We are in discussion on how to emit classes without this parameter.

jakubmisek commented 5 years ago

parameter-less ctors have been implemented in PeachPie 0.9.43, please check @fMichaleczek :)

fMichaleczek commented 5 years ago

@jakubmisek First, thank you for your quick response ! I understand this issue can sound like crazy :) As a little summary, it's a first move that let me begin to explorer the crossover. As a sample scenario, I can work to implement a cmdlet that execute a query on mysql and return back a formatted object. I let you with a detailed report. Let me know what do you think about the issues :)

Report

I confirm the parameter-less constructor solve the problem when instanciante a cmdlet. I am able to import a dll compiled with Peachpie SDK as a PowerShell Module with a PowerShell Module Manifest (psd1) that declare the dll and provide PowerShell options.

But it seems that :

An Attribute on a property doesn't produce any error or MSIL code. After reading the documentation, it seems the feature is missing. In consequence, the cmdlet can't have parameters :-( .

There is a casting problem near class property (and maybe more around type).

    $secund = "foostring";
    $this->WriteObject($secund); # work

    $this->WriteObject($this.Name); # null

I can create an issue for the attribute feature after you confirm me it's not my fault. About the casting, I need to go deeper because if it's work for you in C#, it must work here. PowerShell has many mecanisms to serialize and decorate object ( Adapted and Extended Type Systems ATS/ETS ). EDIT : I think this is related to PowerShell LanguagePrimitives LanguagePrimitives.AsPSObjectOrNull

Aplication Mode

After, I will test PowerShell Application Mode in a PHP application ( C# Sample)

Can you confirm the PeachPie Application Mode (for hosting Peachpie Compiler in C# ) is not ready ? SimpleAnalyzerAssemblyLoader

Peachpie dll in PowerShell

PS > Get-Content PSPeachPie.psd1 | ? { $_ -match 'PSPeachpie.dll' }
RootModule = 'PSPeachPie.dll'
PS > Import-Module .\PSPeachPie.psd1
PS > Get-Command -module PSPeachPie

CommandType     Name                                               Version    Source
-----------     ----                                               -------    ------
Cmdlet          Send-Greeting                                      0.0.1      PSPeachPie

PS > Send-Greeting -Verbose

WARNING: SendGreetingCommand BeginProcessing
VERBOSE: SendGreetingCommand ProcessRecord Start
foostring
VERBOSE: SendGreetingCommand ProcessRecord End
VERBOSE: SendGreetingCommand EndProcessing Start
VERBOSE: SendGreetingCommand EndProcessing End

PS D:\leXPec\Code\PSPeachPie\PSPeachPie> Send-Greeting -?

NAME
    Send-Greeting

SYNTAX
    Send-Greeting [<CommonParameters>]

ALIASES
    None

REMARKS
    None

Source code :

<?php
// <PackageReference Include="PowerShellStandard.Library" Version="5.1.0" />
namespace PSPeachPie;

use System\Management\Automation\{CmdletAttribute, ParameterAttribute, PSCmdlet, PSObject};

// Declare cmdlet with the verb 'Send' and the noun 'Greeting'.
// Verb need to respect the Get-Verb list
// See https://docs.microsoft.com/en-us/dotnet/api/system.management.automation.pscmdlet?view=pscore-6.2.0
[Cmdlet("Send", "Greeting")]
class SendGreetingCommand extends PSCmdlet
{
    // Declare a mandatory parameter for the cmdlet.
    [Parameter(Mandatory=true)]
    public $Name = "Bill";

    // Declare a default parameter for the cmdlet.
    [Parameter()]
    public $FirstName;

    // Declare a default parameter for the cmdlet.
    [Parameter()]
    public $Age = "23";

    public function __construct() 
    {
        $this->FirstName = "Roger";
    }

    // Override the BeginProcessing which execute BEFORE the pipeline 
    public function BeginProcessing() : void
    {
        $this->WriteWarning("SendGreetingCommand BeginProcessing");
    }

    // Override the ProcessRecord which execute the pipeline
    // with write out an object by calling the WriteObject method.
    public function ProcessRecord() : void
    {
        // https://docs.microsoft.com/en-us/dotnet/api/system.management.automation.cmdlet.writeobject?view=pscore-6.2.0

        $this->WriteVerbose("SendGreetingCommand ProcessRecord Start");

        $first = $this.FirstName;
        $this->WriteObject($first); # null

        $secund = "foostring";
        $this->WriteObject($secund); # work

        $this->WriteObject($this.Name); # null

        $this->WriteVerbose("SendGreetingCommand ProcessRecord End");

    }

    // Override the EndProcessing which execute AFTER the pipeline 
    public function EndProcessing() : void
    {
        $this->WriteVerbose("SendGreetingCommand EndProcessing Start");

        $this->WriteObject((string) $this.Name); # null

        $this->WriteVerbose("SendGreetingCommand EndProcessing End");
    }
}

liesauer commented 5 years ago

@jakubmisek First, thank you for your quick response ! I understand this issue can sound like crazy :) As a little summary, it's a first move that let me begin to explorer the crossover. As a sample scenario, I can work to implement a cmdlet that execute a query on mysql and return back a formatted object. I let you with a detailed report. Let me know what do you think about the issues :)

Report

I confirm the parameter-less constructor solve the problem when instanciante a cmdlet. I am able to import a dll compiled with Peachpie SDK as a PowerShell Module with a PowerShell Module Manifest (psd1) that declare the dll and provide PowerShell options.

But it seems that :

An Attribute on a property doesn't produce any error or MSIL code. After reading the documentation, it seems the feature is missing. In consequence, the cmdlet can't have parameters :-( .

There is a casting problem near class property (and maybe more around type).
        $secund = "foostring";
        $this->WriteObject($secund); # work

        $this->WriteObject($this.Name); # null
I can create an issue for the attribute feature after you confirm me it's not my fault. About the casting, I need to go deeper because if it's work for you in C#, it must work here. PowerShell has many mecanisms to serialize and decorate object ( Adapted and Extended Type Systems ATS/ETS ). EDIT : I think this is related to PowerShell LanguagePrimitives LanguagePrimitives.AsPSObjectOrNull

Aplication Mode

After, I will test PowerShell Application Mode in a PHP application ( C# Sample)

Can you confirm the PeachPie Application Mode (for hosting Peachpie Compiler in C# ) is not ready ? SimpleAnalyzerAssemblyLoader

Peachpie dll in PowerShell
PS > Get-Content PSPeachPie.psd1 | ? { $_ -match 'PSPeachpie.dll' }
RootModule = 'PSPeachPie.dll'
PS > Import-Module .\PSPeachPie.psd1
PS > Get-Command -module PSPeachPie

CommandType     Name                                               Version    Source
-----------     ----                                               -------    ------
Cmdlet          Send-Greeting                                      0.0.1      PSPeachPie

PS > Send-Greeting -Verbose

WARNING: SendGreetingCommand BeginProcessing
VERBOSE: SendGreetingCommand ProcessRecord Start
foostring
VERBOSE: SendGreetingCommand ProcessRecord End
VERBOSE: SendGreetingCommand EndProcessing Start
VERBOSE: SendGreetingCommand EndProcessing End

PS D:\leXPec\Code\PSPeachPie\PSPeachPie> Send-Greeting -?

NAME
    Send-Greeting

SYNTAX
    Send-Greeting [<CommonParameters>]

ALIASES
    None

REMARKS
    None
Source code :
<?php
// <PackageReference Include="PowerShellStandard.Library" Version="5.1.0" />
namespace PSPeachPie;

use System\Management\Automation\{CmdletAttribute, ParameterAttribute, PSCmdlet, PSObject};

// Declare cmdlet with the verb 'Send' and the noun 'Greeting'.
// Verb need to respect the Get-Verb list
// See https://docs.microsoft.com/en-us/dotnet/api/system.management.automation.pscmdlet?view=pscore-6.2.0
[Cmdlet("Send", "Greeting")]
class SendGreetingCommand extends PSCmdlet
{
    // Declare a mandatory parameter for the cmdlet.
    [Parameter(Mandatory=true)]
    public $Name = "Bill";

    // Declare a default parameter for the cmdlet.
    [Parameter()]
    public $FirstName;

    // Declare a default parameter for the cmdlet.
    [Parameter()]
    public $Age = "23";

    public function __construct() 
    {
        $this->FirstName = "Roger";
    }

    // Override the BeginProcessing which execute BEFORE the pipeline 
    public function BeginProcessing() : void
    {
        $this->WriteWarning("SendGreetingCommand BeginProcessing");
    }

    // Override the ProcessRecord which execute the pipeline
    // with write out an object by calling the WriteObject method.
    public function ProcessRecord() : void
    {
        // https://docs.microsoft.com/en-us/dotnet/api/system.management.automation.cmdlet.writeobject?view=pscore-6.2.0

        $this->WriteVerbose("SendGreetingCommand ProcessRecord Start");

        $first = $this.FirstName;
        $this->WriteObject($first); # null

        $secund = "foostring";
        $this->WriteObject($secund); # work

        $this->WriteObject($this.Name); # null

        $this->WriteVerbose("SendGreetingCommand ProcessRecord End");

    }

    // Override the EndProcessing which execute AFTER the pipeline 
    public function EndProcessing() : void
    {
        $this->WriteVerbose("SendGreetingCommand EndProcessing Start");

        $this->WriteObject((string) $this.Name); # null

        $this->WriteVerbose("SendGreetingCommand EndProcessing End");
    }
}

$this->Name, not $this.Name

liesauer commented 5 years ago

btw，A class which not implement __toString can not cast to string

fMichaleczek commented 5 years ago

@liesauer thanks, it was so obvious 😱 WriteObject is a method that take system.object and transform it to a PSObject through the DLR.So, I'm now testing the cast between php et dotnet world.

jakubmisek commented 5 years ago

@fMichaleczek great progress! We'll add custom attributes for PHP fields (https://github.com/peachpiecompiler/peachpie/issues/452). Then we'll see how it will handle casting between PhpValue and whatever PSObject needs.

fMichaleczek commented 5 years ago

@jakubmisek this is my report on method and casting phpValue. From my side, it's not a problem, I make an override and move on. I can create a separate issue with not pseudo code.

Summary : In an instance class, when I use a Phcp variable as an argument of a dotnet method, if the ParameterType is a System.Object, i see unconsistency in the casting and receive null value except for type string and array.

class PSCmdlet 
{
    public void WriteObject(System.Object)
    {
        ...
    }
}

$this->WriteObject(true); // null
$this->WriteObject(1234); // null
$this->WriteObject("test"); // OK
...
$this->WriteObject(array(
            "foo" => "foovalue",
            "bar" => "barvalue",
)); // OK

So, I create a PchpCmdlet which herits from PSCmdlet

class PhcpCmdlet : PSCmdlet 
{
       public void WriteObject(PhpValue sendToPipeline)
       {
        switch (sendToPipeline.TypeCode)
            {
        case PhpTypeCode.Object:
                    obj = PhpConvert.AsObject(sendToPipeline);
                    break;
                default:
                    obj = new PSObject(sendToPipeline.Object);
                    break;
            }
            base.WriteObject(obj);
       }
}

but it's not working for double and PhpArray. I try to overload WriteObject with them, but it's worst, it break everything ( the casting don't make the best choise ), so I create 2 methods :

class PhcpCmdlet : PSCmdlet 
{
        ...

    public void WriteArrayObject(PhpArray sendToPipeline)
        {
            WriteObject((PhpValue) sendToPipeline);
        }

        public void WriteDoubleObject(double sendToPipeline)
        {
            WriteObject((PhpValue) sendToPipeline);
        }
}

Php :

$this->WriteArrayObject(array(
       "foo" => "foovalue",
       "bar" => "barvalue",
));
$this->WriteDoubleObject(1.234);

jakubmisek commented 5 years ago

You can just call phpvalue.ToClr() // https://docs.peachpie.io/api/ref/phpvalue/#from-phpvalue

       public void WriteObject(PhpValue sendToPipeline) {
            base.WriteObject(sendToPipeline.ToClr());
       }

this works for all the types.

Anyways; the problem is object in PHP means a class instance. So if you pass array, double or boolean ... it cannot be converted to a PHP class.

fMichaleczek commented 5 years ago

This is the reproductible errors that apply only on array and double (not on all scalar). I think it's more a method overloads problem that fail back into a casting error.

class PhcpCmdlet : PSCmdlet 
{
     public void WriteObject(PhpValue sendToPipeline) {
           ...
     }
}

$this->WriteObject(1.234); 
>> Expected no exception to be thrown, but an exception "costof(double -> System.Object)"

$this->WriteObject(array(
            "foo" => "foovalue",
            "bar" => "barvalue",
)); 
>> Expected no exception to be thrown, but an exception "costof(array -> System.Object)" was thrown

class PhcpCmdlet : PSCmdlet 
{
        public void WriteObject(PhpArray sendToPipeline)
        {
                 ...
        }
}

$this->WriteObject(array(
            "foo" => "foovalue",
            "bar" => "barvalue",
)); 
>> Expected no exception to be thrown, but an exception "costof(array -> System.Object)" was thrown

jakubmisek commented 5 years ago

thanks, this should be fixed in the recent commit :)

jakubmisek commented 5 years ago

so the remaining issue is passing a value of any type into CLRs System.Object without having to workaround it in C#

fMichaleczek commented 5 years ago

I don't know if it's an issue. Priority is coherence between PHP and dotnet, and it's a little bit hard for me to see the consequence on your bridge. We could check that later.

For the side, PowerShell call PHP, I only need tests after you next release.

For the other side, PHP call PowerShell, there is 2 issues, one with a small workaround around APM (invoke, begininvoke, endinvoke) and an error (fatal) around IDynamicMetaObjectProvider (Not implemented). I can only invoke ToString() on a psobject.

Let me know if you want a report. I can't think you could handle that without installing the PowerShell SDK and have an environment.

I don't know your progression in your roadmap about dynamic part. If Peachpie wants a bridge to 3rd parties API like VMWare, Azure, Microsoft, AWS, PowerShell can be used like a SQL API in PHP code. PowerShell can be see as a set of static method with lazy parameters for dot net.

My project name is PSPeachie, suggestions are welcome before i upload to Github.

jakubmisek commented 5 years ago

@fMichaleczek thank you, there should be some fixes in the recent release 0.9.44. It would be great if you could try and give some feedback on what is not working or what could be better - the goal is to implement the PHP class overriding the C# class without much workarounds and hacks :)

jakubmisek commented 5 years ago

closing for now, if there would be more issues with C# <-> PHP interop, please open a new issue :) thanks!

peachpiecompiler / peachpie