hestiaAI / hestialabs-experiences

HestiaLabs Data Experiences & Digipower Academy
https://digipower.academy
Other
7 stars 1 forks source link

Add buttons to view .TXT extension files as Text OR JSON OR Table (TSV(+CSV?)) #373

Open alexbfree opened 2 years ago

alexbfree commented 2 years ago

We see a text file in Washington Post data whose extension is .TXT but entire content is JSON data. This should be parsed as JSON.

valentinoli commented 2 years ago

Ok, shouldn't this be more generic? What about CSV data? What about HTML? Or if we take this further, an encoded image as a data URI? What about computational considerations?

alexbfree commented 2 years ago

for now just JSON within TXT is fine - for a specific usecase. contact me if you need sample data

alexbfree commented 2 years ago

Comment from Lua:

373 is trickier than I thought. Right now the file type is only determined through the extension, and changing this would require a major refactor. One option that I see is to make the user choose how to load the file, giving them options like "(try to) load as a table", "(try to) load as a JSON object", "(try to) load text", or something of the kind.

My comment:

yes that would be great as an intermediate option maybe just 3 little buttons 'view as JSON' 'view as table' or 'view as text' which you can click between. (text being default)

andreaskundig commented 2 years ago

This is related to #359 for which we probably need to guess the format of an unknown file

alexbfree commented 2 years ago

@andreaskundig not quite; given the update to this ticket, we sidestep the detection problem by giving buttons to the user so they can decide.

Perhaps we can try implementing this one and that might then help us advance #359 once done ?

andreaskundig commented 2 years ago

Yes we might end up using buttons like these for dealing with #359

alexbfree commented 2 years ago

Right, but i'm saying let's implement those buttons here and now for #373 (this remains high priority). Then we can re-assess what is still needed for #359. I have put #359 on hold until this is implemented.

streitl commented 2 years ago

Here's what came up during a discussion on this morning's stand-up:

The current solution (#472) with the buttons is great for the file viewer, but it's a very local solution. It does not cause a txt file to be interpreted as a json everywhere, just on the file viewer. This means that in the timed observation viewer for instance, no dated events will be retrieved as the file is interpreted as a txt file and not a json. One possible solution (that would imply a major refactor) is to determine the type of every file before doing anything else, and then tagging the file object with this metadata so that all pipelines and everything that we do consider the correct type for each file.

alexbfree commented 2 years ago

to be tested - francois thinks already done

alexbfree commented 2 years ago

I tested this and in fact the .TXT files don't render at all.

c788346.js:2 TypeError: Failed to execute 'createObjectURL' on 'URL': Overload resolution failed.
    at f.downloadPath (512e795.js:8:1709623)
    at En.get (c788346.js:2:27382)
    at En.evaluate (c788346.js:2:28522)
    at f.downloadPath (c788346.js:2:30411)
    at f.<anonymous> (512e795.js:8:1710669)
    at t._render (c788346.js:2:36074)
    at f.r (c788346.js:2:65839)
    at En.get (c788346.js:2:27382)
    at En.run (c788346.js:2:28266)
    at Sn (c788346.js:2:26365)

so this bug (or something closely related) is still active.

alexbfree commented 2 years ago

Moving to backlog on the basis it is probably not a priority