biolab / orange3

🍊 :bar_chart: :bulb: Orange: Interactive data analysis
https://orangedatamining.com
Other
4.85k stars 1.01k forks source link

Preprocess widget Remove sparse features does nor remove rows with sparse features #6297

Closed mcapurso closed 1 year ago

mcapurso commented 1 year ago

What's wrong?

On Orange3 3.32 and 3.34 the Preprocess widget, Select Relevant Features dows NOT removes rows with null values. Moreover, if you have two instances of Select Relevant Features in the same Preprocess widget, one with zeros and the other with null, and save the ows and reload it, you find the two instances with null values. How can we reproduce the problem?

Run Orange3, open error_walmart.ows , in the FeatureFile widget open file features.csv . Look at FeatureStatistics and find that there are five features with null values more than 50% (Markdown1 - Markdown5) I build a feature X1 with all values zero Go to the Preprocess widget and you find two instances with null values. The first was zero and the second was null. Put the first at zero, save the ows, close and reload ad you find again the two at null: the widget does NOT save setting correctly or does NOT reload the setting correctly. Now set again the first at zero and the second at null, and in Feature_datatable(2) you will find that the X1 variable with zero values HAS BEEN deleted while the Markdown1 - Markdown5 columns with more than 50% of null values HAVE NOT been deleted. What's your environment?

janezd commented 1 year ago

I see you have Apply disabled. Changes will not be saved unless applied.

Beyond this, the filter indeed did not work properly: the user interface did not show the actual state, and sometimes a change of the state did not trigger an update. Should be fixed in #6308.