search

queryverse / Query.jl

Query almost anything in julia
Other
394 stars 49 forks source link

Docs for missing values #306

Open baggepinnen opened 4 years ago

baggepinnen commented 4 years ago

I got confused by the following query

metrics = @from i in data0 begin
    @where i.happy != "NA" && i.energetic != "NA" && !ismissing(i.ActualStartTime)
    @select {i.ParticipantID, happy=round(Int, i.happy), energy=round(Int, i.energetic), i.sex, time=i.ActualStartTime}
    @collect DataFrame
end

and I expected the resulting dataframe to not contain any records for which time is missing, but no filtering took place. I had it explained for me on Zulip, that missing are translated to DataValues. The docs https://www.queryverse.org/Query.jl/stable/gettingstarted/#Missing-values-1 are not super clear on this and I did not expect this behavior even after having read that part of the docs again. In particular, the docs do not really mention what to use instead of ismissing or how I can opt out of the automatic translation if it all possible.

Paethon commented 4 years ago

This DataValue instead of missing business has confused me as well. Especially since I think there is no explanation in the documentation why the package is not using the standard Julia missing type.