search

BONSAMURAIS / correspondence_tables

Work space for the correspondence tables working group for BONSAI
BSD 3-Clause "New" or "Revised" License
7 stars 6 forks source link

CorTables: Pre-hackathon release #3

Open cmutel opened 5 years ago

cmutel commented 5 years ago
mfastudillo commented 5 years ago

Hi! does anybody have a suggestion on how to get/make a correspondence between exiobase and US EPA elementary flow list?

konstantinstadler commented 5 years ago

Hi,

We have all available EXIOBASE concordances here:

https://ntnu.app.box.com/s/ziox4zmkgt3cdsg549brr0qaecskgjsd

perhaps that helps

best

Konstantin

From: Miguel notifications@github.com Sent: 15 March 2019 01:54 To: BONSAMURAIS/Correspondence-tables Cc: Subscribed Subject: Re: [BONSAMURAIS/Correspondence-tables] CorTables: Pre-hackathon release (#3)

Hi! does anybody have a suggestion on how to get/make a correspondence between exiobase and US EPA elementary flow list?

- You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://github.com/BONSAMURAIS/Correspondence-tables/issues/3#issuecomment-473119833, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AGd_y8LtS8cO09XWwNjNoyIgyQWMUb9Aks5vWu9ggaJpZM4b09J7.

tgmorais commented 5 years ago

Hi all,

Miguel, I'm preparing the correspondence between US EPA elementary flow and EXIOBASE emission, but unfortunately I'am having to much to do and I didn't finish yet. I will try finish it today/tomorrow.

Cheers Tiago

mfastudillo commented 5 years ago

Hi,

With respect to bibliographic data. I was thinking that we could use frictionless data to create resource descriptors of the existing csv files. I can give it a go, what do you think?

cmutel commented 5 years ago

@michele would need to give a definitive answer, but that sounds good. Even getting actual sources for the correspondence (i.e. who did it, how, when) would already be an improvement. Probably we will use JSON-LD as the "definitive" storage format, but for now we have CSVs so data packages make a lot of sense.

Feel free to make adjustments to get all CSVs in the same format, btw.

On Wed, 20 Mar 2019 at 16:02, Miguel notifications@github.com wrote:

Hi,

With respect to bibliographic data. I was thinking that we could use frictionless data to create resource descriptors of the existing csv files. I can give it a go, what do you think?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/BONSAMURAIS/Correspondence-tables/issues/3#issuecomment-474870895, or mute the thread https://github.com/notifications/unsubscribe-auth/AAnYYYV9nYmZu5jql4-Yp_itPTB3Fh7Kks5vYk1ogaJpZM4b09J7 .

-- ############################ Chris Mutel Technology Assessment Group, LEA Paul Scherrer Institut OHSA D22 5232 Villigen PSI Switzerland http://chris.mutel.org Telefon: +41 56 310 5787 ############################

mfastudillo commented 5 years ago

Sounds good. There is a lot of tables but I guess many will not be needed for the Hackaton.

I will start with exiobase to EPA but if somebody has tables in mind that will be needed, please let me know.

mfastudillo commented 5 years ago

I've just added a notebook where I started to do some data cleaning and first attempt to create data descriptors. @Stefano-MRC , please have look to the notebook , in the exiobase2_to_NACE2008_v2.csv table there is a almost empty column with some text, that I am not able to judge if it is relevant.

MicDr commented 5 years ago

@cmutel and @mfastudillo Good idea to try the conversion to frictionless data format but I would definitely put the output file in a different and dedicated folder rather than together with the other correspondence file as now (see my reply here too about the discussion above.)

Stefano-MRC commented 5 years ago

Hi Miguel,

I've modified the initial version https://github.com/BONSAMURAIS/Correspondence-tables/blob/master/raw/exiobase2_to_NACE2008.csv that included this 4th column. I don't know why was there, I just left it.

Stefano On 21/03/2019 22:54, Miguel wrote:

I've just added a notebook where I started to do some data cleaning and first attempt to create data descriptors. @Stefano-MRC https://github.com/Stefano-MRC , please have look to the notebook https://github.com/BONSAMURAIS/Correspondence-tables/blob/master/data%20exploration.ipynb , in the |exiobase2_to_NACE2008_v2.csv| table there is a almost empty column with some text, that I am not able to judge if it is relevant.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/BONSAMURAIS/Correspondence-tables/issues/3#issuecomment-475417629, or mute the thread https://github.com/notifications/unsubscribe-auth/AX5Wfc3iP6dT8vz35R65P6xzzUtQGRaHks5vY_9_gaJpZM4b09J7.

-- Best, S.

mfastudillo commented 5 years ago

when talking about vocabulary and correspondence tables. Do we need a common vocabulary for energy-related stuff so exiobase and entso-e can talk? it seems like the most urgent one for the hackathon. I recently discovered the Standard International Energy Product Classification (SIEC) used by UN. I think it would make sense to have a correspondence table with this and exiobase / entso-e energy stuff. I have not seen yet found a machine-readable version of the SIEC classification

mfastudillo commented 5 years ago

An update of what was done and what is missing:

an update of what needs to be done. I note that we have "moved to goalpost" because some of the suggested outputs just came after the hackathon.

US-EPA to chemspider (only for GHG) #18

exiobase units to units of measure (OM) #13

exiobase to entso-e activities #11

exiobase to US-EPA (ghg only) #8

entsoe technology name to "uri" is there. We create it when we were tasked to create the turtle files (later json). but that is probably not the right way to do it and they probably should not be there. @tgmorais1 moved USEPA_temperature_increase, and USEPA to the _LCIAModule (following issue #21 ) so I think we can erase them.

we are not sure which data should go in the descriptor. #17 (also raised by @tgmorais1 in #18). I think we need some clarification. @massimopizzol did an example of how to create descriptors, and it is quite straight forward.

cmutel commented 5 years ago

Please consider the examples in arborist as a path towards writing basic functions that can be reused to make many of these tasks easy. It has gone through several iterations already.

For example, https://github.com/BONSAMURAIS/arborist/blob/master/arborist/graph_common.py defines both a low-level function add_common_elements which every graph dataset needs, and a higher-level function generate_generic_graph which is enough for most common cases (see e.g. https://github.com/BONSAMURAIS/arborist/blob/master/arborist/climate_change.py ).

The JSON metadata is basically the same each time, and can be abstracted away easily. Some predicates can be determined automatically, with a default for best guess 1-1 relationships.

On Tue, 2 Apr 2019 at 18:57, Miguel notifications@github.com wrote:

An update of what was done and what is missing:

an update of what needs to be done. I note that we have been "moving to goalpost" because some of the suggested outputs just came after the hackathon.

US-EPA to chemspider http://www.chemspider.com/ (only for GHG) #18 https://github.com/BONSAMURAIS/Correspondence-tables/issues/18

  • metadata as json file
  • csv
  • csv with predicate
  • scriptable way to create it from raw data (e.g. using the API)

exiobase units to units of measure (OM) #13 https://github.com/BONSAMURAIS/Correspondence-tables/issues/13

  • metadata as json file
  • csv
  • csv with predicate
  • scriptable way to create it from raw data

exiobase to entso-e activities #11 https://github.com/BONSAMURAIS/Correspondence-tables/issues/11

  • metadata as json file
  • csv
  • csv with predicate and aggregator/dissagregator

exiobase to US-EPA (ghg only) #8 https://github.com/BONSAMURAIS/Correspondence-tables/issues/8

  • metadata as json file
  • csv
  • csv with predicate and aggregator/dissagregator

entsoe technology name to "uri" https://github.com/BONSAMURAIS/Correspondence-tables/blob/master/final_tables/tables/bentso_activity_types_uri.csv is there. We create it when we were tasked to create the turtle files (later json). but that is probably not the right way to do it and they probably should not be there. @tgmorais1 https://github.com/tgmorais1 moved USEPA_temperature_increase, and USEPA to the LCIA_Module (following issue #21 https://github.com/BONSAMURAIS/Correspondence-tables/issues/21 ) so I think we can erase them.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/BONSAMURAIS/Correspondence-tables/issues/3#issuecomment-479093470, or mute the thread https://github.com/notifications/unsubscribe-auth/AAnYYRbuLhyyFSNFfUUMW6MDl4FcQ-flks5vc4vdgaJpZM4b09J7 .

-- ############################ Chris Mutel Technology Assessment Group, LEA Paul Scherrer Institut OHSA D22 5232 Villigen PSI Switzerland http://chris.mutel.org Telefon: +41 56 310 5787 ############################