skip option not working in metadata upload

[hsosik]

Uploading a metadata file with some '1' values in the skip column is not resulting in the "skipped" option appearing correctly in the dashboard (still "Skipped: No" in Basic Info and the bins appear in the timeline).

Here is a test file to try (these are real metadata values so no problem uploading to the live dashboard) metadata_nbp2202_skip_test.csv

[joefutrelle]

1 and 0 are not currently accepted as values for the skip column in metadata uploads; the following code defines the accepted values. I think we should add 1 and 0 to the accepted values list. In the meantime you can use true and false and it should work.

https://github.com/WHOIGit/ifcbdb/blob/dfe8798eaf1a5bc87e526089426c8650849b2a8f/ifcbdb/dashboard/accession.py#L275

[hsosik]

1 and 0 are the skip values returned from the export metadata api call, so it does seem consistent to allow upload in that format!

[hsosik]

I tried using true and it still doesn't seem to work for me. Here's the file I'm trying for upload: "\sosiknas1\IFCB_data\NBP2202_holiver\metadata_nbp2202_skip.csv" What am I doing wrong?

[joefutrelle]

For the moment, use true (lowercase) instead of TRUE (uppercase) and that should work until the fix is in place.

[hsosik]

"\sosiknas1\IFCB_data\NBP2202_holiver\metadata_nbp2202_skip.csv" does have true lower case. If I open that file in a text editor, I see true (the values appear as TRUE if I open that csv in excel which decides to make it upper, interpreting as boolean I guess in the default csv load). If I upload that file, it does not work.

[joefutrelle]

I see what's going on now. Using y works but not true because Python/Pandas is inferring a boolean type as well.

There are two issues needing to be fixed:

• 1 and 0 are not in the allowed values list
• true is causing Pandas to treat the column as boolean, so the comparison to the allowed value true is failing.

[hsosik]

Indeed! I changed true to y in my file and it works now!!