search

codedthinking / Kezdi.jl

An umbrella of Julia packages for data analysis, in loving memory of Gábor Kézdi
Other
9 stars 0 forks source link

Look into why manipulations give back missings #63

Closed gergelyattilakiss closed 6 days ago

gergelyattilakiss commented 1 week ago

Sometimes it seems that manipulations with columns with type gives back type?.

korenmiklos commented 1 week ago
image
gergelyattilakiss commented 6 days ago

There are sure no missings in the df after running it. So it might be some default behavior? image

gergelyattilakiss commented 6 days ago

Transform does not have this... image

gergelyattilakiss commented 6 days ago

Also, pandas like indexing approach does not have this. image

gergelyattilakiss commented 6 days ago

Not view creation is at fault. image

gergelyattilakiss commented 6 days ago

I think I have where we introduce the missing type into the workflow. Using `df.x+df.y' in an assingment on SubDataFrames causes this. I check in the related package if this is intentional.

gergelyattilakiss commented 6 days ago

Further checks show that sdf level definition causes the missing type inclusion. Another thing that comes is if something is created in a df it flows down to sdf.

korenmiklos commented 6 days ago

Wow, this is deep.

It is indeed the expected behavior. If you assign some values to a SubDataFrame, the rest of the dataframe may be missing.

We do this manually by creating a fully missing vector first. But maybe not even needed.

korenmiklos commented 6 days ago

Because this works as expected, closing as wontfix.