iRon7 / Join-Object

Combines two objects lists based on a related property between them.
MIT License
107 stars 13 forks source link

MissingLeftProperty: Join-Object : The property 'xxx' cannot be found on the left object. #27

Closed grryf closed 3 years ago

grryf commented 3 years ago

Hello :) Could you please investigate. I have upgraded from VERSION 3.3.0 (using module ".psm1") to VERSION 3.5.2 (import module or using module ".psm1" yeld same results) and my script stopped to work. Reverted and it started to work normally.

`Join-Object : The property 'manager' cannot be found on the left object. At C:\scripts-task_scheduler\class_Powershell\class_AD_Users.psm1:25 char:22

my script: $this.list = Join-Object -LeftObject $this.list -RightObject $list_AD_manager -On 'manager' -JoinType Left # -Equals 'manager.DistinguishedName'

I have confirmed in debug that every list element has 'manager' property lowercase by doing so: `foreach ($k in $list_AD_manager) { # or $this.list $list_p = $k | Get-Member -MemberType NoteProperty | select -ExpandProperty Name if ("manager" -in $list_p) {

write-host "." -NoNewline

} else {
    write-host "ERROR"
}

}`

Help

iRon7 commented 3 years ago

Is it possible to create a Minimal, Reproducible Example for this issue? What is the type and contained by $this.list and $list_AD_manager? To get the type use: $this.list.PSTypeNames and $list_AD_manager.PSTypeNames. To share (a part of) the contents, you might do something like:

$this.list |Select-Object -First 5 |ConvertTo-Csv -NoTypeInformation

and

$list_AD_manager |Select-Object -First 5 |ConvertTo-Csv -NoTypeInformation

btw, does it also error when you do something like:

$this.list |Select-Object  Manager, ..., ... |LeftJoin $list_AD_manager -On Manager
iRon7 commented 3 years ago

I suspect that there is actually a $Null record in your $this.list, this will produce a similar error and will not be captured by your testing.

To workaround (confirm) this, you might try do:

$this.list = $this.list.Where({ $null -ne $_ }) |% { $_ }

Prior the join operation

I am not sure whether this qualifies as a (Join-Object) bug (presuming that this suggestion is indeed is the case) as a "fix" might break a side-by-side join (a join by omitting the -on parameter). I need to think about this...

grryf commented 3 years ago

This worked: $this.list | Select-Object @{n = "Manager"; e = { $_.manager } }, * -ExcludeProperty Manager | LeftJoin $list_AD_manager -On Manager

$list_AD_manager is [Object[]] with PSObject properties: manager and manager.mail $this.list is [Object[]] with String properties: manager, etc

Now I see that I tried to join String to PSObject property with same name.

Tried to make it work without above line but somehow it's very hard to cast the properties to String as I'm using calculated properties which always yeld PSObject whatever I cast [String] and .toString() it... This is more frustrating as workaround is to use calculated property too.

[Object[]] $list_AD_manager = $this.list | Select-Object @{Name = "manager"; expression = { "$($.DistinguishedName)" } }, ` @{Name = "manager.mail"; expression = { "$($.mail)" } }`

Don't know what I could do more. Maybe should I really make working data sample for you to analyse?

iRon7 commented 3 years ago

I am not sure whether I understand the issue. The type of the Manager property Value isn't related to the The property 'manager' cannot be found on the left object error. But if you want to change all your properties to a String type, you might consider something (ugly) like this:

$this.list |ConvertTo-Csv |ConvertFrom-Csv |...

I did some analyzes were it might go wrong and it comes actually down to the properties names (PSObject.Properties.Name) of the first object in the lists. So the question is, what does this reveal:

$this.list |Select -first 1 |% { $_.PSObject.Properties.Name }

and

$list_AD_manager |Select -first 1 |% { $_.PSObject.Properties.Name }
iRon7 commented 3 years ago

A difference with version 3.3.0 is that it doesn't anymore automatically unroll embedded object lists as it now also support ("scalar") arrays as an input. (See StackOverflow question: Is there a PowerShell equivalent of paste (i.e., horizontal file concatenation)?)
If the properties of the first object ($this.list |Select -first 1 |% { $_.PSObject.Properties.Name }) are actually:

Length
LongLength
Rank
SyncRoot
IsReadOnly
IsFixedSize
IsSynchronized
Count

Your object list is probably embedded in a parent array. To unroll e.g. the $this.list, you might do this:

$this.list |ForEach-Object { $_ } |LeftJoin $list_AD_manager -On Manager

Note that Select-Object @{n = "Manager"; e = { $_.manager } }, * -ExcludeProperty Manager has the same effect.

grryf commented 3 years ago

Properties listed from first element. Then again two samples of code, where one works, another not.


Write-Host ($this.list | select -First 1 | % { $_.PSObject.Properties.Name })
SamAccountName mail xCompanyOfOrigin PasswordLastSet PasswordNeverExpires xStartDate xEndDate LastLogonDate DistinguishedName manager GivenName     
Surname l Enabled ObjectGUID

Write-Host ($list_AD_manager | select -First 1 | % { $_.PSObject.Properties.Name })
manager manager.mail

$null = $this.list | LeftJoin $list_AD_manager -On Manager # this works
$null = Join-Object -LeftObject $this.list -RightObject $list_AD_manager -On Manager -JoinType Left # this fails

Join-Object : The property 'Manager' cannot be found on the left object.
At line:1 char:9
+ $null = Join-Object -LeftObject $this.list -RightObject $list_AD_mana ...
+         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : SyntaxError: (:) [Join-Object], ArgumentException
    + FullyQualifiedErrorId : MissingLeftProperty,Join-Object

$this.list = $this.list | LeftJoin $list_AD_manager -On Manager # works again
iRon7 commented 3 years ago

Thanks,

I was too convinced that there would be any difference between the two syntaxes, that I never tried (and I don't have any checks in my pester tests either). This is indeed a bug (related to embedded list difference with earlier versions, I explained earlier). I will further investigate and let you know when it is fixed.

grryf commented 3 years ago

Good luck and take care. This was so much fun working with you.

iRon7 commented 3 years ago

FYI, I have updated the script/module a few hours ago. This should fix this issue using the -LeftObject parameter. Needless to say: I still recommend using the pipeline ("|") though as that uses considerable less memory when streamed correctly. Something like:

import-csv .\left.csv |LeftJoin (import-csv .\right.csv) -On Manager |Export-Csv .\results.csv
grryf commented 3 years ago

Thank you, just updated and tested - it works now with -LeftJoin too. I wasn't aware about piping performance difference. I was always against piping as $list.where({}) is significantly faster than $list | ?. Thank you!

On Tue, 6 Jul 2021 at 16:07, iRon7 @.***> wrote:

FYI, I have updated the script/module a few hours ago. This should fix this issue using the -LeftObject parameter. Needless to say: I still recommend using the pipeline ("|") though as that uses considerable less memory when streamed correctly. Something like:

import-csv .\left.csv |LeftJoin (import-csv .\right.csv) -On Manager |Export-Csv .\results.csv

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/iRon7/Join-Object/issues/27#issuecomment-874792546, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJWWF7BKLXYOUI5KPUOEKQLTWME3RANCNFSM47T5EOBA .

iRon7 commented 3 years ago

I was always against piping as $list.where({ ... }) is significantly faster than $list |Where-Object { ... }

Yes, that is because (in most cases) performance testing done on a $List that already resides in memory which isn't normally a used case. Besides, the execution is often deferred until you capture the actual result which causes another bias. See StackOverflow answer: Fastest Way to get a uniquely index item from the property of an array