Taxon Pages DwC export complications.

mabecabrera commented 11 months ago

Feature or enhancement

From my PC at the museum, when I click on the DWC button (https://sfg-taxonpages.github.io/orthoptera/#/otus/930579/overview),

I get this:

Do not know how to make it look like organized in columns. Having a hard time here...

The same thing happens when downloading in .CSV from the filters results.

Location

DwC download of specimens

Screenshot, napkin sketch of interface, or conceptual description

No response

Your role

No response

mjy commented 11 months ago

@mabecabrera Excel :(. Try this:

Do not click to open the file directly
Open Excel
Choose to import a file, ensure it's Tab delimited as an option

This should placed the data into columns?

debpaul commented 11 months ago

@mabecabrera to explain a bit more. Tab separated (tab delimited) files work well (better than others like "comma-separated" files) to keep data in the expected columns when data get shared. Sadly, Excel doesn't really understand .tsv files if you "double-click" to open. (And there are other reasons why "double-clicking" to open a data file in Excel is not a good idea. I can explain more later). And as @mjy explained, if you import data from the file instead, it will work as you expect it to. Do let us know if you need more help with this.

LocoDelAssembly commented 11 months ago

Still, the choice of .csv extension rather than .txt, .tsv, or .tab complicates this a bit more. When opening from within Excel instead of file explorer it asks for the fields separator only if the text file is not .csv, otherwise assumes field separator according to system locale (which in Spanish it is ";" instead of traditional "," of Comma Separated Values).

debpaul commented 11 months ago

Still, the choice of .csv extension rather than .txt, .tsv, or .tab complicates this a bit more. When opening from within Excel instead of file explorer it asks for the fields separator only if the text file is not .csv, otherwise assumes field separator according to system locale (which in Spanish it is ";" instead of traditional "," of Comma Separated Values).

Thanks @LocoDelAssembly for adding this info. Were you able then, to help Belén get the data in Excel as she expects? the joys of using Excel!

mjy commented 11 months ago

, otherwise assumes field separator according to system locale (which in Spanish it is ";" instead of traditional "," of Comma Separated Values)

Exactly, so we personally can't depend on .csv meaning anything. IBM used "character seperated values before", wikipedia notes https://en.wikipedia.org/wiki/Comma-separated_values#Specification.

But, if we want to serve the Excel gods, we should likely move to .tsv so people have to pause.

mabecabrera commented 11 months ago

Thak you all! I'm home now and from the mac it opens fine. I'll try soon those steps from windows.

@mjy as we are talking about this... I couldn't find the column in that DwC file to know if there are or not images for that collection object. Could you? MetadataAssociated... or anything like that...

mjy commented 11 months ago

@mabecabrera could several things:

We need to regenerate the DwC (likely reason). To test, find a specimen with images, and re-save it. It may now have values in associatedMedia.
I disabled associatedMedia since it wasn't exporting what people expected (links to binaries), but rather links to metadata

Please test the first to see if this is the issue.

mabecabrera commented 11 months ago

@mabecabrera Excel :(. Try this:

Do not click to open the file directly

Open Excel

Choose to import a file, ensure it's Tab delimited as an option

This should placed the data into columns?

YES! That works! Thank you very much.

mabecabrera commented 11 months ago

@mabecabrera could several things:

We need to regenerate the DwC (likely reason). To test, find a specimen with images, and re-save it. It may now have values in associatedMedia.

I disabled associatedMedia since it wasn't exporting what people expected (links to binaries), but rather links to metadata

Please test the first to see if this is the issue.

About this: That column is present...

But all the links I checked give me this error message...

And the file has so many empty columns. I would love to know from the public page which specimen does have images and see them without downloading this file... Imagine looking to this file from a mobile...

mjy commented 11 months ago

Add the project_token[]= to see the images.

Right, definitely not mobile.

Temp/intermediate solutions while specimenPages gets imagined:

I did play with OpenRefine, I think in a pinch we could write a recipe when played over the file quickly summarized it down to a few set of rows and data and maybe clickable links to see the images.
Another thought- does GBIF have an OSF DwC dump? They would pickup images there.
Also, I hate to sound like that guy- a rusty, crusty, mean cranky old dude- but why not add people to the project or practice db?!

mabecabrera commented 11 months ago

Also, I hate to sound like that guy- a rusty, crusty, mean cranky old dude- but why not add people to the project or practice db?!

ALSO THAT!!! YES! We will do! But in case we need to use a mobile view... visiting a museum... Every one will be guided and invited to the practice db. No exception. Then it will depend on them.

mjy commented 11 months ago

use a mobile view... visiting a museum...

Understood. A quick way to work in the collection to check for needed images.

SpeciesFileGroup / taxonpages