search

quintel / refinery

Calculates node demands and edge shares for sparse energy graphs.
2 stars 0 forks source link

From where is refinery going to read CSV's? #26

Closed wmeyers closed 10 years ago

wmeyers commented 11 years ago

@antw, we are now thinking about the file structure in which the Excel analyses are going to output the csv's with EB, shares, etc. We are thinking of a separate repository, but I'm not sure if that's what you intended. Have you and @dennisschoenmakers thought about this?

@ChaelKruip and @jorisberkhout should also be aware of this ticket.

wmeyers commented 11 years ago

Refinery reads from etsource/data/datasets. @antw and @dennisschoenmakers, is this the final location from which atlas and refinery read the csv's, or is this just a convenient location for now?

antw commented 11 years ago

is this the final location from which atlas and refinery read the csv's, or is this just a convenient location for now?

I don't know if this exact location is permanent or temporary, but I do think the CSVs belong in ETSource. After all, the queries live in ETSource...

When we finally kill InputExcel, everything in ETSource's data directory will be moved to the top directory, i.e. data/nodes becomes nodes, data/datasets becomes datasets, etc.

If the current arrangement isn't ideal, does anyone have suggestions for what we could improve?

wmeyers commented 11 years ago

On the other hand:

  1. The CSV's are created by our Excel analyses, which I think should not really be part of etsource.
  2. Also, these Excel analyses are producing several CSV's which are passed between the Excel analyses, but never actually used by Refinery, so they should also not be part of etsource

It's a bit of a dilemma, because on the one hand it makes complete sense to put the CSV's in etsource, since that's where the queries live. On the other hand it seems logical to keep them with wherever the Excel analyses are going to be living. I think keeping them in two places produces some risk of deviation between what we think is used by Refinery, between what is actually there. So I guess we should choose.

@antw, I'm not trying to get a definite answer from you at the moment on what it should be, but just would like to hear your ideas. I think this is also something that @dennisschoenmakers and @AlexanderWirtz have an important say in.

jorisberkhout commented 10 years ago

I believe this issue can be closed, @wmeyers ?

wmeyers commented 10 years ago

Seems like a good idea. Closing!